Browse Source
Phase 21: Fix Image Input & Deduplication Issues - 1 plan created - 2 tasks + 1 checkpoint defined - Ready for execution Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>handoff-20260429-1057
shrimbly
6 months ago
3 changed files with 181 additions and 11 deletions
@ -0,0 +1,153 @@ |
|||
--- |
|||
phase: 21-fix-image-input-deduplication |
|||
plan: 01 |
|||
type: execute |
|||
--- |
|||
|
|||
<objective> |
|||
Fix Gemini nano-banana-pro model ignoring image inputs and resolve duplicate image saving across input/generated images. |
|||
|
|||
Purpose: Ensure image-to-image generation works correctly with Gemini and that the deduplication system consistently prevents duplicate files in the generations folder. |
|||
Output: Working image-to-image generation with Gemini Pro, consistent hashing across all image saving operations. |
|||
</objective> |
|||
|
|||
<execution_context> |
|||
~/.claude/get-shit-done/workflows/execute-phase.md |
|||
~/.claude/get-shit-done/templates/summary.md |
|||
~/.claude/get-shit-done/references/checkpoints.md |
|||
</execution_context> |
|||
|
|||
<context> |
|||
@.planning/PROJECT.md |
|||
@.planning/ROADMAP.md |
|||
@.planning/STATE.md |
|||
|
|||
# Key source files: |
|||
@src/app/api/generate/route.ts |
|||
@src/app/api/save-generation/route.ts |
|||
@src/utils/imageStorage.ts |
|||
|
|||
# Prior phase with related hashing decision: |
|||
@.planning/phases/13-fix-duplicate-generations/13-01-SUMMARY.md |
|||
|
|||
**Tech stack available:** Next.js 16, Zustand, @google/genai SDK |
|||
**Established patterns:** MD5 content hashing for deduplication (Phase 13) |
|||
|
|||
**Issues being addressed:** |
|||
1. nano-banana-pro model generates without considering image inputs (image data sent but not used) |
|||
2. Input and generated images have duplicate files despite different hashes - need consistent hashing approach |
|||
3. Generated images should be prepended with prompt details like generated videos |
|||
|
|||
**Research findings:** |
|||
- Gemini SDK docs show that for image editing, the order matters: image should come BEFORE text in the parts array |
|||
- Current implementation sends [text, ...images] but should send [...images, text] for image-to-image |
|||
- imageStorage.ts uses position-based sampling hash while save-generation uses MD5 - these are incompatible |
|||
</context> |
|||
|
|||
<tasks> |
|||
|
|||
<task type="auto"> |
|||
<name>Task 1: Fix Gemini image-to-image part ordering</name> |
|||
<files>src/app/api/generate/route.ts</files> |
|||
<action> |
|||
In generateWithGemini(), change the requestParts array construction to place images BEFORE text prompt. The Gemini SDK documentation shows image editing works with [inlineData, text] order, not [text, inlineData]. |
|||
|
|||
Current code (around line 59): |
|||
``` |
|||
const requestParts = [ |
|||
{ text: prompt }, // Text first - WRONG for image-to-image |
|||
...imageData.map(...) // Images second |
|||
]; |
|||
``` |
|||
|
|||
Change to: |
|||
``` |
|||
const requestParts = [ |
|||
...imageData.map(...) // Images first - correct for image editing |
|||
{ text: prompt }, // Text second |
|||
]; |
|||
``` |
|||
|
|||
WHY: Gemini's image generation models treat the first content type as primary input. For text-to-image, text should be first. For image-to-image (editing), images should be first so the model knows it's modifying existing images. |
|||
|
|||
IMPORTANT: Only reorder when images are present. If no images, text should still be the only part. |
|||
</action> |
|||
<verify> |
|||
Build succeeds with `npm run build`. Manual test: Create workflow with ImageInput → GenerateImage (nano-banana-pro) → Output, connect an image, add prompt like "make it blue", run workflow. Output should visually reference the input image. |
|||
</verify> |
|||
<done> |
|||
When images are connected to a Gemini GenerateImage node, the generated output incorporates/transforms the input image based on the prompt. |
|||
</done> |
|||
</task> |
|||
|
|||
<task type="auto"> |
|||
<name>Task 2: Unify image hashing to MD5 across all save operations</name> |
|||
<files>src/utils/imageStorage.ts</files> |
|||
<action> |
|||
Replace the position-based sampling hash in imageStorage.ts with MD5 content hashing (matching save-generation/route.ts pattern). |
|||
|
|||
The current approach at line 209-211 uses a weak hash based on string length and sampled substrings: |
|||
``` |
|||
const hash = `${folder}-${len}-${imageData.substring(50, 100)}-${imageData.substring(mid, mid + 50)}-${imageData.substring(Math.max(0, len - 50))}`; |
|||
``` |
|||
|
|||
This is problematic because: |
|||
1. Different hash method than save-generation API (MD5) |
|||
2. Same image can get different hashes if saved via different paths |
|||
3. Position-based sampling is weak for dedup (similar images may have same samples) |
|||
|
|||
Replace with: |
|||
1. Import crypto at top: `import crypto from "crypto";` |
|||
2. Create helper: `function computeContentHash(data: string): string { return crypto.createHash("md5").update(data).digest("hex"); }` |
|||
3. Replace the hash computation: `const hash = `${folder}-${computeContentHash(imageData)}`;` (folder prefix still needed to separate inputs/generations) |
|||
|
|||
WHY: MD5 is already used in save-generation (Phase 13 decision), provides reliable deduplication, and is fast enough for our use case. |
|||
</action> |
|||
<verify> |
|||
Build succeeds with `npm run build`. TypeScript types check with no errors. |
|||
</verify> |
|||
<done> |
|||
imageStorage.ts uses MD5 hashing, consistent with save-generation API. Same image saved twice gets same hash and is deduplicated. |
|||
</done> |
|||
</task> |
|||
|
|||
<task type="checkpoint:human-verify" gate="blocking"> |
|||
<what-built> |
|||
Fixed Gemini image-to-image generation (part ordering) and unified MD5 hashing for deduplication. |
|||
</what-built> |
|||
<how-to-verify> |
|||
1. Run: `npm run dev` |
|||
2. Create a new workflow with: |
|||
- ImageInput node (load any test image) |
|||
- GenerateImage node (select nano-banana-pro model) |
|||
- Prompt node with text like "add a sunset background" or "make it black and white" |
|||
- Output node |
|||
3. Connect: ImageInput → GenerateImage (image handle), Prompt → GenerateImage (text handle) |
|||
4. Run the workflow |
|||
5. Verify: The generated image should visually show the transformation applied to the input image (not a completely new unrelated image) |
|||
6. Check generations folder: No duplicate files for the same generated content |
|||
</how-to-verify> |
|||
<resume-signal>Type "approved" if image-to-image works correctly, or describe issues</resume-signal> |
|||
</task> |
|||
|
|||
</tasks> |
|||
|
|||
<verification> |
|||
Before declaring phase complete: |
|||
- [ ] `npm run build` succeeds without errors |
|||
- [ ] `npm test` passes all existing tests |
|||
- [ ] Gemini image-to-image generation produces output that transforms the input image |
|||
- [ ] No TypeScript errors in modified files |
|||
</verification> |
|||
|
|||
<success_criteria> |
|||
- All tasks completed |
|||
- All verification checks pass |
|||
- Image-to-image generation with nano-banana-pro produces visually related output |
|||
- Deduplication uses consistent MD5 hashing across all image save paths |
|||
- No regression in text-to-image generation (still works when no image connected) |
|||
</success_criteria> |
|||
|
|||
<output> |
|||
After completion, create `.planning/phases/21-fix-image-input-deduplication/21-01-SUMMARY.md` |
|||
</output> |
|||
Loading…
Reference in new issue