---
phase: 13-fix-duplicate-generations
plan: 01
type: execute
---
Add content-based deduplication to the auto-save generations flow.
Purpose: Prevent duplicate images/videos in the generations folder when the same content is generated multiple times. Currently every generation is saved regardless of whether identical content already exists.
Output: Modified save-generation API that checks content hash before writing, returns existing file if duplicate found.
@~/.claude/get-shit-done/workflows/execute-phase.md
@~/.claude/get-shit-done/templates/summary.md
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
**Key source files:**
@src/app/api/save-generation/route.ts - Current save logic (no deduplication)
@src/utils/imageStorage.ts - Existing hash logic for workflow images (lines 206-211)
**Current behavior:**
- `workflowStore.ts` calls `/api/save-generation` with `imageId = Date.now()` for every generation
- `/api/save-generation` saves to `{directoryPath}/{imageId}.{ext}` unconditionally
- No content hash check - identical images get duplicate files
**Target behavior:**
- Hash image/video content on server before saving
- Check if file with same hash already exists in generations folder
- If duplicate: return existing file path, don't create new file
- If unique: save file with hash-based filename, return new path
**Prior art:**
- `imageStorage.ts:saveImageAndGetId` uses client-side hash of substring samples (lines 206-211)
- Server-side crypto hash is more reliable and works for URL-fetched content
Task 1: Add content hashing to save-generation API
src/app/api/save-generation/route.ts
Add server-side content hashing using Node.js crypto:
1. Import `crypto` module at top
2. After buffer is created (line ~95-108), compute content hash:
```typescript
const contentHash = crypto.createHash('md5').update(buffer).digest('hex');
```
3. Check if file with this hash already exists in directory:
- List files in directoryPath
- Check if any filename starts with the hash
- If found, return early with existing file info
4. Use hash-prefixed filename instead of timestamp-based:
- Change filename format to: `{hash}_{promptSnippet}.{extension}`
- Keep prompt snippet for human readability
- Hash ensures uniqueness without timestamp
Why MD5: Fast for binary data, collision resistance not critical for dedup (worst case: rare false duplicate). SHA256 is overkill for local file dedup.
Why hash in filename: Enables O(1) duplicate check via glob/fs.existsSync pattern match instead of reading all files.
Build passes: `npm run build` succeeds without errors.
Manual test: Generate same image twice, second generation should return existing file path without creating duplicate.
- MD5 hash computed from buffer content
- Filename uses hash prefix: `{hash}_{prompt}.{ext}`
- Duplicate check finds existing files by hash prefix
- Returns existing file info if duplicate found
- New files saved with hash-prefixed name
Task 2: Add duplicate detection response handling
src/app/api/save-generation/route.ts
Enhance response to indicate whether file was deduplicated:
1. Add `isDuplicate: boolean` field to response JSON
2. When returning existing file (duplicate found):
- Set `isDuplicate: true`
- Return existing `filePath` and `filename`
- Log deduplication event for observability
3. When saving new file:
- Set `isDuplicate: false`
- Return new `filePath` and `filename` as before
This allows callers (workflowStore) to know if deduplication occurred, useful for debugging and future UI feedback.
Test duplicate scenario: Generate same content twice, second call returns `{ success: true, isDuplicate: true, filePath: existing }`.
Test unique scenario: Generate new content, returns `{ success: true, isDuplicate: false, filePath: new }`.
- Response includes `isDuplicate` boolean field
- Duplicate files logged: "Generation deduplicated: existing file found"
- Unique files logged: "Generation saved" (existing log)
Before declaring phase complete:
- [ ] `npm run build` succeeds without errors
- [ ] Generate same prompt twice → second returns existing file (no new file created)
- [ ] Generate different prompt → new file created with hash prefix
- [ ] Response includes `isDuplicate` field
- [ ] Existing generations folder functionality preserved
- All tasks completed
- All verification checks pass
- No duplicate files created for identical content
- Backward compatible: existing generations still accessible
- Hash-based filenames enable fast duplicate lookup