Browse Source
Phase 21: Fix Image Input & Deduplication Issues - 1 plan created - 2 tasks + 1 checkpoint defined - Ready for execution Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>handoff-20260429-1057
shrimbly
6 months ago
3 changed files with 181 additions and 11 deletions
@ -0,0 +1,153 @@ |
|||||
|
--- |
||||
|
phase: 21-fix-image-input-deduplication |
||||
|
plan: 01 |
||||
|
type: execute |
||||
|
--- |
||||
|
|
||||
|
<objective> |
||||
|
Fix Gemini nano-banana-pro model ignoring image inputs and resolve duplicate image saving across input/generated images. |
||||
|
|
||||
|
Purpose: Ensure image-to-image generation works correctly with Gemini and that the deduplication system consistently prevents duplicate files in the generations folder. |
||||
|
Output: Working image-to-image generation with Gemini Pro, consistent hashing across all image saving operations. |
||||
|
</objective> |
||||
|
|
||||
|
<execution_context> |
||||
|
~/.claude/get-shit-done/workflows/execute-phase.md |
||||
|
~/.claude/get-shit-done/templates/summary.md |
||||
|
~/.claude/get-shit-done/references/checkpoints.md |
||||
|
</execution_context> |
||||
|
|
||||
|
<context> |
||||
|
@.planning/PROJECT.md |
||||
|
@.planning/ROADMAP.md |
||||
|
@.planning/STATE.md |
||||
|
|
||||
|
# Key source files: |
||||
|
@src/app/api/generate/route.ts |
||||
|
@src/app/api/save-generation/route.ts |
||||
|
@src/utils/imageStorage.ts |
||||
|
|
||||
|
# Prior phase with related hashing decision: |
||||
|
@.planning/phases/13-fix-duplicate-generations/13-01-SUMMARY.md |
||||
|
|
||||
|
**Tech stack available:** Next.js 16, Zustand, @google/genai SDK |
||||
|
**Established patterns:** MD5 content hashing for deduplication (Phase 13) |
||||
|
|
||||
|
**Issues being addressed:** |
||||
|
1. nano-banana-pro model generates without considering image inputs (image data sent but not used) |
||||
|
2. Input and generated images have duplicate files despite different hashes - need consistent hashing approach |
||||
|
3. Generated images should be prepended with prompt details like generated videos |
||||
|
|
||||
|
**Research findings:** |
||||
|
- Gemini SDK docs show that for image editing, the order matters: image should come BEFORE text in the parts array |
||||
|
- Current implementation sends [text, ...images] but should send [...images, text] for image-to-image |
||||
|
- imageStorage.ts uses position-based sampling hash while save-generation uses MD5 - these are incompatible |
||||
|
</context> |
||||
|
|
||||
|
<tasks> |
||||
|
|
||||
|
<task type="auto"> |
||||
|
<name>Task 1: Fix Gemini image-to-image part ordering</name> |
||||
|
<files>src/app/api/generate/route.ts</files> |
||||
|
<action> |
||||
|
In generateWithGemini(), change the requestParts array construction to place images BEFORE text prompt. The Gemini SDK documentation shows image editing works with [inlineData, text] order, not [text, inlineData]. |
||||
|
|
||||
|
Current code (around line 59): |
||||
|
``` |
||||
|
const requestParts = [ |
||||
|
{ text: prompt }, // Text first - WRONG for image-to-image |
||||
|
...imageData.map(...) // Images second |
||||
|
]; |
||||
|
``` |
||||
|
|
||||
|
Change to: |
||||
|
``` |
||||
|
const requestParts = [ |
||||
|
...imageData.map(...) // Images first - correct for image editing |
||||
|
{ text: prompt }, // Text second |
||||
|
]; |
||||
|
``` |
||||
|
|
||||
|
WHY: Gemini's image generation models treat the first content type as primary input. For text-to-image, text should be first. For image-to-image (editing), images should be first so the model knows it's modifying existing images. |
||||
|
|
||||
|
IMPORTANT: Only reorder when images are present. If no images, text should still be the only part. |
||||
|
</action> |
||||
|
<verify> |
||||
|
Build succeeds with `npm run build`. Manual test: Create workflow with ImageInput → GenerateImage (nano-banana-pro) → Output, connect an image, add prompt like "make it blue", run workflow. Output should visually reference the input image. |
||||
|
</verify> |
||||
|
<done> |
||||
|
When images are connected to a Gemini GenerateImage node, the generated output incorporates/transforms the input image based on the prompt. |
||||
|
</done> |
||||
|
</task> |
||||
|
|
||||
|
<task type="auto"> |
||||
|
<name>Task 2: Unify image hashing to MD5 across all save operations</name> |
||||
|
<files>src/utils/imageStorage.ts</files> |
||||
|
<action> |
||||
|
Replace the position-based sampling hash in imageStorage.ts with MD5 content hashing (matching save-generation/route.ts pattern). |
||||
|
|
||||
|
The current approach at line 209-211 uses a weak hash based on string length and sampled substrings: |
||||
|
``` |
||||
|
const hash = `${folder}-${len}-${imageData.substring(50, 100)}-${imageData.substring(mid, mid + 50)}-${imageData.substring(Math.max(0, len - 50))}`; |
||||
|
``` |
||||
|
|
||||
|
This is problematic because: |
||||
|
1. Different hash method than save-generation API (MD5) |
||||
|
2. Same image can get different hashes if saved via different paths |
||||
|
3. Position-based sampling is weak for dedup (similar images may have same samples) |
||||
|
|
||||
|
Replace with: |
||||
|
1. Import crypto at top: `import crypto from "crypto";` |
||||
|
2. Create helper: `function computeContentHash(data: string): string { return crypto.createHash("md5").update(data).digest("hex"); }` |
||||
|
3. Replace the hash computation: `const hash = `${folder}-${computeContentHash(imageData)}`;` (folder prefix still needed to separate inputs/generations) |
||||
|
|
||||
|
WHY: MD5 is already used in save-generation (Phase 13 decision), provides reliable deduplication, and is fast enough for our use case. |
||||
|
</action> |
||||
|
<verify> |
||||
|
Build succeeds with `npm run build`. TypeScript types check with no errors. |
||||
|
</verify> |
||||
|
<done> |
||||
|
imageStorage.ts uses MD5 hashing, consistent with save-generation API. Same image saved twice gets same hash and is deduplicated. |
||||
|
</done> |
||||
|
</task> |
||||
|
|
||||
|
<task type="checkpoint:human-verify" gate="blocking"> |
||||
|
<what-built> |
||||
|
Fixed Gemini image-to-image generation (part ordering) and unified MD5 hashing for deduplication. |
||||
|
</what-built> |
||||
|
<how-to-verify> |
||||
|
1. Run: `npm run dev` |
||||
|
2. Create a new workflow with: |
||||
|
- ImageInput node (load any test image) |
||||
|
- GenerateImage node (select nano-banana-pro model) |
||||
|
- Prompt node with text like "add a sunset background" or "make it black and white" |
||||
|
- Output node |
||||
|
3. Connect: ImageInput → GenerateImage (image handle), Prompt → GenerateImage (text handle) |
||||
|
4. Run the workflow |
||||
|
5. Verify: The generated image should visually show the transformation applied to the input image (not a completely new unrelated image) |
||||
|
6. Check generations folder: No duplicate files for the same generated content |
||||
|
</how-to-verify> |
||||
|
<resume-signal>Type "approved" if image-to-image works correctly, or describe issues</resume-signal> |
||||
|
</task> |
||||
|
|
||||
|
</tasks> |
||||
|
|
||||
|
<verification> |
||||
|
Before declaring phase complete: |
||||
|
- [ ] `npm run build` succeeds without errors |
||||
|
- [ ] `npm test` passes all existing tests |
||||
|
- [ ] Gemini image-to-image generation produces output that transforms the input image |
||||
|
- [ ] No TypeScript errors in modified files |
||||
|
</verification> |
||||
|
|
||||
|
<success_criteria> |
||||
|
- All tasks completed |
||||
|
- All verification checks pass |
||||
|
- Image-to-image generation with nano-banana-pro produces visually related output |
||||
|
- Deduplication uses consistent MD5 hashing across all image save paths |
||||
|
- No regression in text-to-image generation (still works when no image connected) |
||||
|
</success_criteria> |
||||
|
|
||||
|
<output> |
||||
|
After completion, create `.planning/phases/21-fix-image-input-deduplication/21-01-SUMMARY.md` |
||||
|
</output> |
||||
Loading…
Reference in new issue