From ee8ade26a7e1330b9907131b03318ee0dc36abab Mon Sep 17 00:00:00 2001 From: shrimbly Date: Fri, 9 Jan 2026 23:27:52 +1300 Subject: [PATCH] docs(06-01): complete GenerateVideo node plan Tasks completed: 2/2 - Create GenerateVideoNode component - Add video generation to API route SUMMARY: .planning/phases/06-video-and-polish/06-01-SUMMARY.md --- .planning/ROADMAP.md | 6 +- .planning/STATE.md | 29 ++--- .../06-video-and-polish/06-01-SUMMARY.md | 105 ++++++++++++++++++ 3 files changed, 124 insertions(+), 16 deletions(-) create mode 100644 .planning/phases/06-video-and-polish/06-01-SUMMARY.md diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 112c5b96..d4c1a380 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -19,7 +19,7 @@ None - [x] **Phase 3: Generate Node Refactor** - Rename NanoBanana to generic Generate with multi-model support ✓ - [x] **Phase 4: Model Search Dialog** - Floating action bar icons and searchable model browser ✓ - [x] **Phase 5: Image URL Server** - Local endpoint serving images as URLs for providers ✓ -- [ ] **Phase 6: Video & Polish** - Video playback, parameter customization, edge cases +- [ ] **Phase 6: Video & Polish** - Video playback, parameter customization, edge cases (In progress) ## Phase Details @@ -84,7 +84,7 @@ Plans: **Design decision**: GenerateVideo as separate node type (not combined with GenerateImage) Plans: -- [ ] 06-01: GenerateVideo node with video-capable model selector +- [x] 06-01: GenerateVideo node with video-capable model selector - [ ] 06-02: Video playback in output node - [ ] 06-03: Custom model parameters from provider schemas - [ ] 06-04: Edge case handling and final polish @@ -101,4 +101,4 @@ Phases execute in numeric order: 1 → 2 → 3 → 4 → 5 → 6 | 3. Generate Node Refactor | 3/3 | Complete | 2026-01-09 | | 4. Model Search Dialog | 2/2 | Complete | 2026-01-09 | | 5. Image URL Server | 2/2 | Complete | 2026-01-09 | -| 6. Video & Polish | 0/4 | Not started | - | +| 6. Video & Polish | 1/4 | In progress | - | diff --git a/.planning/STATE.md b/.planning/STATE.md index 1eebab71..ff932a41 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -5,23 +5,23 @@ See: .planning/PROJECT.md (updated 2026-01-09) **Core value:** Provider infrastructure that dynamically discovers models from external APIs, enabling users to access hundreds of image/video generation models without hardcoding schemas. -**Current focus:** Phase 5 - Image URL Server +**Current focus:** Phase 6 - Video & Polish ## Current Position -Phase: 5 of 6 (Image URL Server) -Plan: 2 of 2 in current phase -Status: Phase complete -Last activity: 2026-01-09 - Completed 05-02-PLAN.md (Provider image integration) +Phase: 6 of 6 (Video & Polish) +Plan: 1 of 4 in current phase +Status: In progress +Last activity: 2026-01-09 - Completed 06-01-PLAN.md (GenerateVideo node) -Progress: [========================================================================] 80% (12/15 plans) +Progress: [=================================================================================] 87% (13/15 plans) ## Performance Metrics **Velocity:** -- Total plans completed: 12 -- Average duration: 5.1 min -- Total execution time: 1.01 hours +- Total plans completed: 13 +- Average duration: 5.5 min +- Total execution time: 1.18 hours **By Phase:** @@ -32,10 +32,11 @@ Progress: [===================================================================== | 3. Generate Node Refactor | 3/3 | 13 min | 4.3 min | | 4. Model Search Dialog | 2/2 | 17 min | 8.5 min | | 5. Image URL Server | 2/2 | 5 min | 2.5 min | +| 6. Video & Polish | 1/4 | 10 min | 10 min | **Recent Trend:** -- Last 5 plans: 3 min, 2 min, 15 min, 3 min, 2 min -- Trend: fast (05-02 was integration only) +- Last 5 plans: 2 min, 15 min, 3 min, 2 min, 10 min +- Trend: stable ## Accumulated Context @@ -70,6 +71,8 @@ Recent decisions affecting current work: - Extract variant suffix from fal.ai model IDs for display name - No TTL for image store - explicit cleanup pattern (callers delete after use) - 256KB threshold for shouldUseImageUrl (Replicate recommendation) +- Gemini excluded from video node (doesn't support video generation) +- Large videos (>20MB) return URL instead of base64 to avoid memory issues ### Deferred Issues @@ -82,6 +85,6 @@ Recent decisions affecting current work: ## Session Continuity Last session: 2026-01-09 -Stopped at: Completed 05-02-PLAN.md (Provider image integration) +Stopped at: Completed 06-01-PLAN.md (GenerateVideo node) Resume file: None -Next action: Plan Phase 6 (Video & Polish) +Next action: Execute 06-02-PLAN.md (Video playback in output node) diff --git a/.planning/phases/06-video-and-polish/06-01-SUMMARY.md b/.planning/phases/06-video-and-polish/06-01-SUMMARY.md new file mode 100644 index 00000000..1c8ff204 --- /dev/null +++ b/.planning/phases/06-video-and-polish/06-01-SUMMARY.md @@ -0,0 +1,105 @@ +--- +phase: 06-video-and-polish +plan: 01 +subsystem: ui, api +tags: [video, react, nextjs, replicate, fal.ai, react-flow] + +# Dependency graph +requires: + - phase: 03 + provides: GenerateImageNode pattern, provider selection UI + - phase: 05 + provides: Image URL serving for providers +provides: + - GenerateVideoNode component with provider/model selection + - Video generation API support in /api/generate + - Video output display with HTML5 video element +affects: [06-02, 06-03] + +# Tech tracking +tech-stack: + added: [] + patterns: [video content-type detection, video URL fallback for large files] + +key-files: + created: [src/components/nodes/GenerateVideoNode.tsx] + modified: [src/types/index.ts, src/store/workflowStore.ts, src/components/WorkflowCanvas.tsx, src/components/nodes/index.ts, src/app/api/generate/route.ts, src/lib/quickstart/validation.ts] + +key-decisions: + - "Video node excludes Gemini provider (Gemini doesn't support video generation)" + - "Large videos (>20MB) return URL instead of base64 to avoid memory issues" + - "Video content-type detection via HTTP headers for Replicate, result.video.url for fal.ai" + +patterns-established: + - "Video response format: { video?: string, videoUrl?: string, contentType: 'video' | 'image' }" + - "GenerateVideoNode follows same provider/model selector pattern as GenerateImageNode" + +issues-created: [] + +# Metrics +duration: 10min +completed: 2026-01-09 +--- + +# Phase 6 Plan 1: GenerateVideo Node Summary + +**GenerateVideoNode component with fal.ai/Replicate provider selection and video generation API support** + +## Performance + +- **Duration:** 10 min +- **Started:** 2026-01-09T10:16:14Z +- **Completed:** 2026-01-09T10:25:55Z +- **Tasks:** 2 +- **Files modified:** 7 + +## Accomplishments + +- Created GenerateVideoNode component following GenerateImageNode pattern +- Extended /api/generate route to handle video outputs from both Replicate and fal.ai +- Added video playback display with HTML5 video element and controls +- Integrated video node into workflow execution and regeneration + +## Task Commits + +Each task was committed atomically: + +1. **Task 1: Create GenerateVideoNode component** - `0c29ad1` (feat) +2. **Task 2: Add video generation to API route** - `937480f` (feat) + +**Plan metadata:** Pending + +## Files Created/Modified + +- `src/components/nodes/GenerateVideoNode.tsx` - New video generation node with provider/model selectors +- `src/types/index.ts` - Added generateVideo to NodeType, GenerateVideoNodeData interface, extended GenerateResponse +- `src/store/workflowStore.ts` - Default data, execution logic, regeneration support +- `src/components/WorkflowCanvas.tsx` - Node registration, handles, minimap color (#9333ea purple) +- `src/components/nodes/index.ts` - Exported GenerateVideoNode +- `src/app/api/generate/route.ts` - Video detection for Replicate/fal.ai, content-type handling +- `src/lib/quickstart/validation.ts` - Added generateVideo to valid types + +## Decisions Made + +- Gemini excluded from video node (doesn't support video generation) +- Large videos (>20MB) return URL directly to avoid memory issues with base64 +- Video detection: Replicate uses content-type headers, fal.ai uses result.video.url field +- Video capabilities filter: text-to-video, image-to-video + +## Deviations from Plan + +None - plan executed exactly as written. + +## Issues Encountered + +None + +## Next Phase Readiness + +- GenerateVideoNode ready for testing with actual video models +- Next plan (06-02) will add video playback to OutputNode for viewing generated videos +- Video execution path untested with real providers (will be tested in 06-02) + +--- +*Phase: 06-video-and-polish* +*Completed: 2026-01-09*