From fc10173490f6296e64a32407d79de14699f9395f Mon Sep 17 00:00:00 2001 From: shrimbly Date: Fri, 9 Jan 2026 22:32:06 +1300 Subject: [PATCH] docs(05): create phase plan for image URL server Phase 05: Image URL Server - 2 plans created - 6 total tasks defined - Request-scoped cleanup (no TTL) to prevent memory accumulation - Ready for execution --- .../phases/05-image-url-server/05-01-PLAN.md | 123 ++++++++++++++++ .../phases/05-image-url-server/05-02-PLAN.md | 134 ++++++++++++++++++ 2 files changed, 257 insertions(+) create mode 100644 .planning/phases/05-image-url-server/05-01-PLAN.md create mode 100644 .planning/phases/05-image-url-server/05-02-PLAN.md diff --git a/.planning/phases/05-image-url-server/05-01-PLAN.md b/.planning/phases/05-image-url-server/05-01-PLAN.md new file mode 100644 index 00000000..25632880 --- /dev/null +++ b/.planning/phases/05-image-url-server/05-01-PLAN.md @@ -0,0 +1,123 @@ +--- +phase: 05-image-url-server +plan: 01 +type: execute +--- + + +Create local image serving infrastructure for URL-based provider inputs. + +Purpose: Enable providers like Replicate and fal.ai to receive images as URLs instead of large base64 payloads, improving reliability for larger images and meeting provider API requirements. +Output: In-memory image store, serving endpoint, and utility function for URL generation. + + + +~/.claude/get-shit-done/workflows/execute-phase.md +~/.claude/get-shit-done/templates/summary.md + + + +@.planning/PROJECT.md +@.planning/ROADMAP.md +@.planning/STATE.md + +# Prior phase context: +@.planning/phases/03-generate-node-refactor/03-02-SUMMARY.md + +# Relevant source files: +@src/app/api/generate/route.ts +@src/lib/providers/cache.ts + +**Tech stack available:** Next.js API routes, in-memory caching pattern from providers/cache.ts +**Established patterns:** TTL-based caching, API route structure + +**Constraining decisions:** +- Phase 03-02: Server-side provider execution, not client-side +- Phase 02-03: 10-minute cache TTL pattern established + +**Provider requirements (from discovery):** +- Replicate: Accepts data URIs but recommends URLs for files >256KB. Parameter name varies by model (`image`, `input_image`, etc.) +- fal.ai: Accepts both URLs and data URIs via `image_url` parameter + + + + + + Task 1: Create in-memory image store with explicit cleanup + src/lib/images/store.ts + +Create an in-memory store for temporary image hosting with request-scoped lifecycle: +- Store structure: Map +- `storeImage(base64DataUrl: string): string` - stores image, returns unique ID (use crypto.randomUUID()) +- `getImage(id: string): { data: Buffer, mimeType: string } | null` - retrieves image by ID +- `deleteImage(id: string): boolean` - removes image, returns true if existed +- `deleteImages(ids: string[]): void` - bulk delete for cleanup + +No TTL - callers are responsible for cleanup after use. This prevents memory accumulation since images are deleted immediately after provider fetches them. + +Parse base64 data URL format: `data:{mimeType};base64,{data}` to extract mimeType and Buffer. +Export store functions. + + TypeScript compiles without errors: `npx tsc --noEmit` + Image store module exists with storeImage, getImage, deleteImage, deleteImages exports + + + + Task 2: Create image serving API endpoint + src/app/api/images/[id]/route.ts + +Create Next.js API route to serve stored images: +- GET handler that extracts `id` from params +- Call `getImage(id)` from store +- If not found: return 404 with JSON error +- If found: return image with correct Content-Type header and binary data +- Set Cache-Control: no-store (images are temporary) + +Route structure follows Next.js App Router dynamic route pattern. +Import getImage from @/lib/images/store. + + Build succeeds: `npm run build` (route compiles correctly) + GET /api/images/[id] returns stored images or 404 + + + + Task 3: Create uploadImageForUrl utility + src/lib/images/index.ts + +Create utility function that stores an image and returns its URL plus ID for cleanup: +- `uploadImageForUrl(base64DataUrl: string, baseUrl: string): { url: string, id: string }` +- Calls storeImage() to store the image +- Returns full URL `${baseUrl}/api/images/${id}` AND the id for later cleanup +- Export from index.ts along with re-exports from store.ts + +The baseUrl parameter allows the caller to provide the server's base URL. Returning the ID enables callers to delete the image after use. + +Also create helper to detect if an image should use URL (size threshold): +- `shouldUseImageUrl(base64DataUrl: string): boolean` +- Returns true if base64 data exceeds 256KB (Replicate's recommendation threshold) +- This allows callers to decide whether to use URL or pass base64 directly + + TypeScript compiles: `npx tsc --noEmit` + uploadImageForUrl (returns url+id) and shouldUseImageUrl functions exported from @/lib/images + + + + + +Before declaring plan complete: +- [ ] `npm run build` succeeds without errors +- [ ] `npx tsc --noEmit` passes +- [ ] Files exist: src/lib/images/store.ts, src/lib/images/index.ts, src/app/api/images/[id]/route.ts + + + + +- All tasks completed +- All verification checks pass +- No TypeScript errors +- Image store, serving endpoint, and utility functions ready for integration + + + +After completion, create `.planning/phases/05-image-url-server/05-01-SUMMARY.md` + diff --git a/.planning/phases/05-image-url-server/05-02-PLAN.md b/.planning/phases/05-image-url-server/05-02-PLAN.md new file mode 100644 index 00000000..c197aa2e --- /dev/null +++ b/.planning/phases/05-image-url-server/05-02-PLAN.md @@ -0,0 +1,134 @@ +--- +phase: 05-image-url-server +plan: 02 +type: execute +--- + + +Integrate image URL serving with Replicate and fal.ai providers. + +Purpose: Enable img2img workflows by passing input images to providers as URLs, completing the multi-provider image generation pipeline. +Output: Working img2img support for Replicate and fal.ai models. + + + +~/.claude/get-shit-done/workflows/execute-phase.md +~/.claude/get-shit-done/templates/summary.md + + + +@.planning/PROJECT.md +@.planning/ROADMAP.md +@.planning/STATE.md + +# Prior plan in this phase: +@.planning/phases/05-image-url-server/05-01-SUMMARY.md + +# Relevant source files: +@src/app/api/generate/route.ts +@src/lib/images/index.ts +@src/store/workflowStore.ts + +**Tech stack available:** Image URL utilities from 05-01, provider dispatch pattern in generate route +**Established patterns:** Header-based API key passing, server-side provider execution + +**Provider image parameter requirements:** +- Replicate: Parameter name varies by model (commonly `image`, `input_image`). Format: URI (URL or data URI). +- fal.ai: Parameter is `image_url`. Format: URL string or data URI. + + + + + + Task 1: Update generateWithReplicate to pass image input + src/app/api/generate/route.ts + +Modify generateWithReplicate() to include image input in prediction: +- Accept images array in GenerationInput (already present) +- If images array has items, add first image to predictionInput as `image` parameter +- Use the image URL directly (caller will provide URL or data URI) +- Log image input presence for debugging + +Note: Different Replicate models use different parameter names (`image`, `input_image`, `image_prompt`). Start with `image` as it's most common for img2img models. Future enhancement could detect from model schema. + +Do NOT convert base64 to URL here - that happens at the call site (workflow execution). + + TypeScript compiles: `npx tsc --noEmit` + generateWithReplicate passes image parameter to prediction input when images provided + + + + Task 2: Update generateWithFal to pass image_url parameter + src/app/api/generate/route.ts + +Modify generateWithFal() to include image_url in request: +- Accept images array in GenerationInput (already present) +- If images array has items, add first image as `image_url` in requestBody +- fal.ai accepts both URLs and data URIs in this field +- Log image input presence for debugging + +fal.ai is consistent: always use `image_url` parameter name. + + TypeScript compiles: `npx tsc --noEmit` + generateWithFal passes image_url parameter when images provided + + + + Task 3: Upload images to URL server with cleanup + src/app/api/generate/route.ts + +Update the POST handler to convert large images to URLs and clean up after use: +- Import uploadImageForUrl, shouldUseImageUrl, deleteImages from @/lib/images +- Get base URL from request: `new URL(request.url).origin` +- For Replicate and fal providers, process images array: + - Track uploaded image IDs in an array for cleanup + - If shouldUseImageUrl(image) returns true, call uploadImageForUrl(image, baseUrl), store ID + - Otherwise, pass the base64 data URI directly (small images work fine as data URIs) +- Pass processed images array to generateWithReplicate/generateWithFal +- In finally block: call deleteImages(uploadedIds) to clean up stored images + +Pattern: +```typescript +const uploadedImageIds: string[] = []; +try { + // Process images, track IDs + // Call provider +} finally { + deleteImages(uploadedImageIds); +} +``` + +This ensures: +- Small images (<256KB) use efficient data URIs +- Large images use temporary URLs served by /api/images/[id] +- Images cleaned up immediately after provider fetches them (success or failure) +- No memory accumulation +- Gemini path unchanged (it handles base64 directly) + + Build succeeds: `npm run build` + Large images converted to URLs and cleaned up after provider call completes + + + + + +Before declaring plan complete: +- [ ] `npm run build` succeeds without errors +- [ ] `npx tsc --noEmit` passes +- [ ] generateWithReplicate includes image in prediction input +- [ ] generateWithFal includes image_url in request body +- [ ] Large images converted to URLs before provider calls + + + + +- All tasks completed +- All verification checks pass +- No TypeScript errors +- Replicate and fal.ai can receive image inputs for img2img workflows +- Phase 5 complete + + + +After completion, create `.planning/phases/05-image-url-server/05-02-SUMMARY.md` +