Browse Source
Phase 13: Fix Duplicate Generations - 1 plan created - 2 tasks defined - Ready for executionhandoff-20260429-1057
1 changed files with 133 additions and 0 deletions
@ -0,0 +1,133 @@ |
|||||
|
--- |
||||
|
phase: 13-fix-duplicate-generations |
||||
|
plan: 01 |
||||
|
type: execute |
||||
|
--- |
||||
|
|
||||
|
<objective> |
||||
|
Add content-based deduplication to the auto-save generations flow. |
||||
|
|
||||
|
Purpose: Prevent duplicate images/videos in the generations folder when the same content is generated multiple times. Currently every generation is saved regardless of whether identical content already exists. |
||||
|
|
||||
|
Output: Modified save-generation API that checks content hash before writing, returns existing file if duplicate found. |
||||
|
</objective> |
||||
|
|
||||
|
<execution_context> |
||||
|
@~/.claude/get-shit-done/workflows/execute-phase.md |
||||
|
@~/.claude/get-shit-done/templates/summary.md |
||||
|
</execution_context> |
||||
|
|
||||
|
<context> |
||||
|
@.planning/PROJECT.md |
||||
|
@.planning/ROADMAP.md |
||||
|
@.planning/STATE.md |
||||
|
|
||||
|
**Key source files:** |
||||
|
@src/app/api/save-generation/route.ts - Current save logic (no deduplication) |
||||
|
@src/utils/imageStorage.ts - Existing hash logic for workflow images (lines 206-211) |
||||
|
|
||||
|
**Current behavior:** |
||||
|
- `workflowStore.ts` calls `/api/save-generation` with `imageId = Date.now()` for every generation |
||||
|
- `/api/save-generation` saves to `{directoryPath}/{imageId}.{ext}` unconditionally |
||||
|
- No content hash check - identical images get duplicate files |
||||
|
|
||||
|
**Target behavior:** |
||||
|
- Hash image/video content on server before saving |
||||
|
- Check if file with same hash already exists in generations folder |
||||
|
- If duplicate: return existing file path, don't create new file |
||||
|
- If unique: save file with hash-based filename, return new path |
||||
|
|
||||
|
**Prior art:** |
||||
|
- `imageStorage.ts:saveImageAndGetId` uses client-side hash of substring samples (lines 206-211) |
||||
|
- Server-side crypto hash is more reliable and works for URL-fetched content |
||||
|
</context> |
||||
|
|
||||
|
<tasks> |
||||
|
|
||||
|
<task type="auto"> |
||||
|
<name>Task 1: Add content hashing to save-generation API</name> |
||||
|
<files>src/app/api/save-generation/route.ts</files> |
||||
|
<action> |
||||
|
Add server-side content hashing using Node.js crypto: |
||||
|
|
||||
|
1. Import `crypto` module at top |
||||
|
2. After buffer is created (line ~95-108), compute content hash: |
||||
|
```typescript |
||||
|
const contentHash = crypto.createHash('md5').update(buffer).digest('hex'); |
||||
|
``` |
||||
|
3. Check if file with this hash already exists in directory: |
||||
|
- List files in directoryPath |
||||
|
- Check if any filename starts with the hash |
||||
|
- If found, return early with existing file info |
||||
|
4. Use hash-prefixed filename instead of timestamp-based: |
||||
|
- Change filename format to: `{hash}_{promptSnippet}.{extension}` |
||||
|
- Keep prompt snippet for human readability |
||||
|
- Hash ensures uniqueness without timestamp |
||||
|
|
||||
|
Why MD5: Fast for binary data, collision resistance not critical for dedup (worst case: rare false duplicate). SHA256 is overkill for local file dedup. |
||||
|
|
||||
|
Why hash in filename: Enables O(1) duplicate check via glob/fs.existsSync pattern match instead of reading all files. |
||||
|
</action> |
||||
|
<verify> |
||||
|
Build passes: `npm run build` succeeds without errors. |
||||
|
Manual test: Generate same image twice, second generation should return existing file path without creating duplicate. |
||||
|
</verify> |
||||
|
<done> |
||||
|
- MD5 hash computed from buffer content |
||||
|
- Filename uses hash prefix: `{hash}_{prompt}.{ext}` |
||||
|
- Duplicate check finds existing files by hash prefix |
||||
|
- Returns existing file info if duplicate found |
||||
|
- New files saved with hash-prefixed name |
||||
|
</done> |
||||
|
</task> |
||||
|
|
||||
|
<task type="auto"> |
||||
|
<name>Task 2: Add duplicate detection response handling</name> |
||||
|
<files>src/app/api/save-generation/route.ts</files> |
||||
|
<action> |
||||
|
Enhance response to indicate whether file was deduplicated: |
||||
|
|
||||
|
1. Add `isDuplicate: boolean` field to response JSON |
||||
|
2. When returning existing file (duplicate found): |
||||
|
- Set `isDuplicate: true` |
||||
|
- Return existing `filePath` and `filename` |
||||
|
- Log deduplication event for observability |
||||
|
3. When saving new file: |
||||
|
- Set `isDuplicate: false` |
||||
|
- Return new `filePath` and `filename` as before |
||||
|
|
||||
|
This allows callers (workflowStore) to know if deduplication occurred, useful for debugging and future UI feedback. |
||||
|
</action> |
||||
|
<verify> |
||||
|
Test duplicate scenario: Generate same content twice, second call returns `{ success: true, isDuplicate: true, filePath: existing }`. |
||||
|
Test unique scenario: Generate new content, returns `{ success: true, isDuplicate: false, filePath: new }`. |
||||
|
</verify> |
||||
|
<done> |
||||
|
- Response includes `isDuplicate` boolean field |
||||
|
- Duplicate files logged: "Generation deduplicated: existing file found" |
||||
|
- Unique files logged: "Generation saved" (existing log) |
||||
|
</done> |
||||
|
</task> |
||||
|
|
||||
|
</tasks> |
||||
|
|
||||
|
<verification> |
||||
|
Before declaring phase complete: |
||||
|
- [ ] `npm run build` succeeds without errors |
||||
|
- [ ] Generate same prompt twice → second returns existing file (no new file created) |
||||
|
- [ ] Generate different prompt → new file created with hash prefix |
||||
|
- [ ] Response includes `isDuplicate` field |
||||
|
- [ ] Existing generations folder functionality preserved |
||||
|
</verification> |
||||
|
|
||||
|
<success_criteria> |
||||
|
- All tasks completed |
||||
|
- All verification checks pass |
||||
|
- No duplicate files created for identical content |
||||
|
- Backward compatible: existing generations still accessible |
||||
|
- Hash-based filenames enable fast duplicate lookup |
||||
|
</success_criteria> |
||||
|
|
||||
|
<output> |
||||
|
After completion, create `.planning/phases/13-fix-duplicate-generations/13-01-SUMMARY.md` |
||||
|
</output> |
||||
Loading…
Reference in new issue