diff --git a/.planning/phases/03-generate-node-refactor/03-01-PLAN.md b/.planning/phases/03-generate-node-refactor/03-01-PLAN.md index 742b7971..65d9c525 100644 --- a/.planning/phases/03-generate-node-refactor/03-01-PLAN.md +++ b/.planning/phases/03-generate-node-refactor/03-01-PLAN.md @@ -7,10 +7,10 @@ type: execute Rename NanoBanana node to GenerateImage and add multi-provider model selector. -Purpose: Transform the hardcoded Gemini-only node into a multi-provider image generation node that filters to image-capable models only. -Output: GenerateImage node component with provider/model dropdown (image models only), updated types, store functions renamed. +Purpose: Transform the Gemini-only node into a multi-provider image generation node. +Output: GenerateImageNode component with provider/model dropdowns filtering to image-capable models only. -Note: GenerateVideo node will be added in Phase 6 (Video & Polish) as a separate node type. +Note: Video generation will use a separate GenerateVideo node in Phase 6. @@ -22,127 +22,93 @@ Note: GenerateVideo node will be added in Phase 6 (Video & Polish) as a separate @.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md - -# Prior phase context: @.planning/phases/01-provider-infrastructure/01-02-SUMMARY.md @.planning/phases/02-model-discovery/02-03-SUMMARY.md - -# Key source files: @src/components/nodes/NanoBananaNode.tsx @src/types/index.ts @src/store/workflowStore.ts @src/lib/providers/types.ts -**Tech stack available:** Zustand, React, ProviderModel interface, /api/models endpoint -**Established patterns:** Provider self-registration, localStorage settings persistence -**Constraining decisions:** -- Keep nanoBanana node type internally for backward compatibility -- Gemini models use internal names (nano-banana, nano-banana-pro) +**Tech available:** ProviderModel interface, /api/models endpoint, provider registry +**Patterns:** Provider self-registration, localStorage settings persistence +**Constraints:** Keep "nanoBanana" node type string for backward compatibility - Task 1: Update types for multi-provider model selection + Task 1: Add SelectedModel type and update NanoBananaNodeData src/types/index.ts - 1. Add new types for selected model: - - `SelectedModel` interface with { provider: ProviderType, modelId: string, displayName: string } - - Keep existing ModelType for backward compatibility - - 2. Update NanoBananaNodeData interface: - - Add optional `selectedModel?: SelectedModel` field - - Keep existing `model: ModelType` field for Gemini backward compatibility - - 3. Add "gemini" to ProviderType union if not already present (it should be the Gemini provider identifier) - - WHY: NanoBananaNodeData needs to store either legacy Gemini model selection OR new multi-provider selection. The optional selectedModel field enables graceful migration. + Add SelectedModel interface: + ```typescript + export interface SelectedModel { + provider: ProviderType; + modelId: string; + displayName: string; + } + ``` + + Update NanoBananaNodeData to add optional selectedModel field (keep existing model field for Gemini backward compatibility). + + Add "gemini" to ProviderType if not present. - npx tsc --noEmit passes - Types compile, SelectedModel interface exists, NanoBananaNodeData has selectedModel field + npx tsc --noEmit + SelectedModel type exists, NanoBananaNodeData has selectedModel field, types compile - Task 2: Rename component to GenerateImageNode with model selector UI + Task 2: Create GenerateImageNode with provider/model selector src/components/nodes/NanoBananaNode.tsx, src/components/nodes/index.ts - 1. Rename file from NanoBananaNode.tsx to GenerateImageNode.tsx - - 2. Rename component from NanoBananaNode to GenerateImageNode - - 3. Add model selector UI that shows: - - Provider dropdown (gemini, replicate, fal) - only show enabled providers - - Model dropdown (fetched from /api/models for selected provider) - - **Filter to image capabilities only**: text-to-image, image-to-image - - For Gemini, show existing hardcoded models (Nano Banana, Nano Banana Pro) - - For other providers, fetch models dynamically with capability filter - - 4. When provider/model changes: - - Update node data with selectedModel: { provider, modelId, displayName } - - For Gemini, also update legacy model field for backward compatibility - - 5. Keep existing aspect ratio/resolution controls for Gemini models - - Hide them for non-Gemini models (parameters will come from provider schema later) - - 6. Update index.ts to export GenerateImageNode and alias as NanoBananaNode: + 1. Rename NanoBananaNode.tsx to GenerateImageNode.tsx + 2. Rename component function to GenerateImageNode + 3. Add provider dropdown showing enabled providers (gemini always, others if configured) + 4. Add model dropdown: + - For gemini: hardcoded Nano Banana / Nano Banana Pro + - For replicate/fal: fetch from /api/models?provider=X&capabilities=text-to-image,image-to-image + 5. Filter to image capabilities only (text-to-image, image-to-image) + 6. Keep aspect ratio/resolution controls for Gemini, hide for others + 7. Update index.ts exports: ```typescript export { GenerateImageNode } from "./GenerateImageNode"; export { GenerateImageNode as NanoBananaNode } from "./GenerateImageNode"; ``` - - WHY: The component rename reflects its image-specific purpose. Video generation will be a separate GenerateVideoNode in Phase 6. The alias maintains backward compatibility. - npm run build succeeds, component renders in browser - GenerateImageNode.tsx exists, shows provider selector, model dropdown shows only image-capable models + npm run build + GenerateImageNode renders with provider/model dropdowns, shows only image models Task 3: Update store and canvas references - src/store/workflowStore.ts, src/components/WorkflowCanvas.tsx + src/store/workflowStore.ts, src/components/WorkflowCanvas.tsx, src/components/ConnectionDropMenu.tsx, src/components/FloatingActionBar.tsx - 1. In workflowStore.ts: - - Rename saveNanoBananaDefaults to saveGenerateImageDefaults (keep old name as alias) - - Rename loadNanoBananaDefaults to loadGenerateImageDefaults (keep old name as alias) - - Update localStorage key comment but keep actual key for migration - - Update createDefaultNodeData for "nanoBanana" case to include default selectedModel for Gemini - - 2. In WorkflowCanvas.tsx: - - Update nodeTypes registration to use GenerateImageNode - - Keep "nanoBanana" as the type key (internal type name unchanged for now) - - Update minimap color mapping if it references "nanoBanana" - - 3. In any file using NanoBananaNode import: - - Update import to use GenerateImageNode - - Files: ConnectionDropMenu.tsx, FloatingActionBar.tsx, etc. - - WHY: Internal references update to new names while keeping the node type string "nanoBanana" for workflow file compatibility. + 1. workflowStore.ts: Rename helper functions (keep aliases for compatibility): + - loadNanoBananaDefaults → loadGenerateImageDefaults + - saveNanoBananaDefaults → saveGenerateImageDefaults + 2. Update createDefaultNodeData "nanoBanana" case to set default selectedModel for Gemini + 3. WorkflowCanvas.tsx: Update nodeTypes to use GenerateImageNode + 4. Update imports in ConnectionDropMenu.tsx, FloatingActionBar.tsx - npm run build succeeds, existing workflows load correctly - All imports updated, store functions renamed with aliases, workflows still load + npm run build succeeds, app loads without errors + All references updated, existing workflows load correctly -Before declaring plan complete: -- [ ] `npm run build` succeeds without errors -- [ ] `npx tsc --noEmit` passes type checking +- [ ] `npm run build` succeeds - [ ] GenerateImageNode renders with provider dropdown -- [ ] Model selector shows only image-capable models (text-to-image, image-to-image) -- [ ] Gemini models show in model selector -- [ ] Existing workflows with nanoBanana nodes still load +- [ ] Model selector shows only image-capable models +- [ ] Existing workflows with nanoBanana nodes load - -- All tasks completed -- All verification checks pass - GenerateImageNode component exists with multi-provider UI -- Model selector filters to image capabilities only -- Types support both legacy Gemini and new multi-provider selection -- Backward compatibility maintained for existing workflows +- Model selector filters to text-to-image and image-to-image only +- Backward compatibility maintained -After completion, create `.planning/phases/03-generate-node-refactor/03-01-SUMMARY.md` +Create `.planning/phases/03-generate-node-refactor/03-01-SUMMARY.md` diff --git a/.planning/phases/03-generate-node-refactor/03-02-PLAN.md b/.planning/phases/03-generate-node-refactor/03-02-PLAN.md index 78957209..3456ade3 100644 --- a/.planning/phases/03-generate-node-refactor/03-02-PLAN.md +++ b/.planning/phases/03-generate-node-refactor/03-02-PLAN.md @@ -5,12 +5,10 @@ type: execute --- -Add provider-specific execution logic to the generate API route for image generation. +Implement provider-specific image generation in API route. -Purpose: Enable the generate endpoint to route requests to the correct provider (Gemini, Replicate, fal.ai) based on the selected model, filtering to image outputs only. -Output: Updated /api/generate route that dispatches to provider-specific image generation logic. - -Note: Video generation will use a separate endpoint in Phase 6. +Purpose: Route generation requests to correct provider (Gemini, Replicate, fal.ai) based on selected model. +Output: Working generate endpoint that dispatches to provider implementations. @@ -20,156 +18,82 @@ Note: Video generation will use a separate endpoint in Phase 6. @.planning/PROJECT.md -@.planning/ROADMAP.md @.planning/STATE.md - -# Prior plan context: @.planning/phases/03-generate-node-refactor/03-01-SUMMARY.md - -# Key source files: @src/app/api/generate/route.ts @src/lib/providers/types.ts @src/lib/providers/replicate.ts @src/lib/providers/fal.ts -@src/lib/providers/index.ts -**Tech stack available:** ProviderInterface with generate() method, provider registry -**Established patterns:** API routes return { success, data/error }, provider self-registration -**Constraining decisions:** -- Gemini generation logic already works, preserve it -- Provider implementations have generate() method defined in ProviderInterface +**Tech available:** ProviderInterface.generate(), provider registry +**Constraints:** Preserve existing Gemini logic, image output only - Task 1: Implement generate() method in Replicate provider + Task 1: Implement Replicate provider generate() method src/lib/providers/replicate.ts - 1. Implement the generate() method that currently throws "not implemented": - - Accept GenerationInput with model, prompt, images, parameters - - Call Replicate's prediction API: POST https://api.replicate.com/v1/predictions - - Handle input format: { version: model.id, input: { prompt, image?, ...parameters } } - - Poll for completion (Replicate is async) or use webhook pattern - - Return GenerationOutput with image/video data - - 2. Handle image input: - - If images provided, check if Replicate model expects URLs - - For now, skip image input (Phase 5 adds image URL server) - - Log warning if images provided but skipped - - 3. Fetch output: - - Poll GET /predictions/{id} until status is "succeeded" or "failed" - - Convert output URL to base64 data URL - - Return in GenerationOutput format with type: "image" - - 4. Filter to image models only: - - Only accept models with text-to-image or image-to-image capabilities - - Return error if video model passed (video generation is Phase 6) - - WHY: Replicate requires polling for async predictions. The generate() method encapsulates this complexity. Image-only filter keeps this endpoint focused. + Implement generate() that currently throws "not implemented": + 1. POST to https://api.replicate.com/v1/predictions with { version: modelId, input: { prompt } } + 2. Poll GET /predictions/{id} until status is "succeeded" or "failed" + 3. Fetch output image URL and convert to base64 data URL + 4. Return GenerationOutput with type: "image" + 5. Skip image inputs for now (Phase 5 adds URL server) - TypeScript compiles, generate() method exists and doesn't throw immediately - Replicate provider has working generate() method that calls prediction API for image models + TypeScript compiles + Replicate generate() calls prediction API and returns image - Task 2: Implement generate() method in fal.ai provider + Task 2: Implement fal.ai provider generate() method src/lib/providers/fal.ts - 1. Implement the generate() method: - - Accept GenerationInput with model, prompt, images, parameters - - Call fal.ai's run endpoint: POST https://fal.run/{model.id} - - Include prompt and parameters in request body - - fal.ai is synchronous (waits for result) - - 2. Handle auth: - - Use "Key {apiKey}" header format (already established) - - API key optional but rate limited without it - - 3. Parse response: - - Extract image URL from response - - Convert to base64 data URL for consistency - - Handle images array output format - - 4. Filter to image models only: - - Only accept models with text-to-image or image-to-image capabilities - - Return error if video model passed (video generation is Phase 6) - - WHY: fal.ai's synchronous API is simpler than Replicate. The generate() method normalizes the response format. Image-only filter keeps this endpoint focused. + Implement generate(): + 1. POST to https://fal.run/{modelId} with { prompt, ...parameters } + 2. Use "Key {apiKey}" auth header + 3. Extract image URL from response, convert to base64 + 4. Return GenerationOutput with type: "image" - TypeScript compiles, generate() method exists - fal.ai provider has working generate() method that calls fal.run endpoint for image models + TypeScript compiles + fal.ai generate() calls fal.run and returns image - Task 3: Update generate API route to dispatch by provider + Task 3: Update generate API route for multi-provider dispatch src/app/api/generate/route.ts - 1. Update request type to accept either: - - Legacy format: { model: ModelType, prompt, images, aspectRatio, resolution, useGoogleSearch } - - New format: { selectedModel: SelectedModel, prompt, images, parameters } - - 2. Add provider dispatch logic: - ```typescript - if (body.selectedModel) { - // New multi-provider path - const { provider, modelId } = body.selectedModel; - if (provider === "gemini") { - // Use existing Gemini logic - } else { - // Get provider and call generate() - const providerImpl = getProvider(provider); - const model = await providerImpl.getModel(modelId); - const result = await providerImpl.generate({ - model, - prompt: body.prompt, - images: body.images, - parameters: body.parameters - }); - return NextResponse.json(result); - } - } else { - // Legacy Gemini path (existing code) - } - ``` - - 3. Extract Gemini-specific logic into helper function for clarity - - 4. Ensure API key handling: - - For Gemini: use GEMINI_API_KEY from env - - For Replicate/fal: get from provider.getApiKey() (localStorage via client header) - - 5. Add client headers for API keys: - - Accept X-Replicate-API-Key and X-Fal-API-Key headers - - Pass to provider implementations - - WHY: The route becomes a dispatcher that preserves existing Gemini behavior while enabling new providers. + 1. Accept both formats: + - Legacy: { model, prompt, images, aspectRatio, resolution } + - New: { selectedModel, prompt, images, parameters } + 2. If selectedModel provided and provider !== "gemini": + - Get provider from registry + - Get API key from request headers (X-Replicate-API-Key, X-Fal-API-Key) + - Call provider.generate() + - Return result + 3. Otherwise use existing Gemini logic + 4. Extract Gemini code into helper function for clarity - npm run build succeeds, existing Gemini generation still works - Generate route dispatches to correct provider, Gemini still works as before + npm run build, test Gemini generation still works + API route dispatches to correct provider, Gemini unchanged -Before declaring plan complete: -- [ ] `npm run build` succeeds without errors -- [ ] Existing Gemini generation works (test with existing workflow) -- [ ] Replicate provider generate() method implemented -- [ ] fal.ai provider generate() method implemented -- [ ] API route accepts both legacy and new request formats +- [ ] `npm run build` succeeds +- [ ] Gemini generation works as before +- [ ] Replicate provider has generate() implementation +- [ ] fal.ai provider has generate() implementation - -- All tasks completed -- All verification checks pass -- Generate API routes to correct provider based on selectedModel -- Gemini generation unchanged for backward compatibility -- Replicate and fal.ai providers have generate() implementations +- Generate API routes requests to correct provider +- Provider generate() methods implemented +- Existing Gemini functionality preserved -After completion, create `.planning/phases/03-generate-node-refactor/03-02-SUMMARY.md` +Create `.planning/phases/03-generate-node-refactor/03-02-SUMMARY.md` diff --git a/.planning/phases/03-generate-node-refactor/03-03-PLAN.md b/.planning/phases/03-generate-node-refactor/03-03-PLAN.md index c19b682a..6ddc6a23 100644 --- a/.planning/phases/03-generate-node-refactor/03-03-PLAN.md +++ b/.planning/phases/03-generate-node-refactor/03-03-PLAN.md @@ -5,10 +5,10 @@ type: execute --- -Ensure backward compatibility for existing workflows containing nanoBanana nodes. +Ensure backward compatibility for existing workflows. -Purpose: Existing saved workflows must load and execute correctly without modification. -Output: Migration logic, verified backward compatibility, updated workflow execution. +Purpose: Existing saved workflows must load and execute without modification. +Output: Migration logic, working execution with both legacy and new formats. @@ -18,137 +18,88 @@ Output: Migration logic, verified backward compatibility, updated workflow execu @.planning/PROJECT.md -@.planning/ROADMAP.md @.planning/STATE.md - -# Prior plan context: @.planning/phases/03-generate-node-refactor/03-01-SUMMARY.md @.planning/phases/03-generate-node-refactor/03-02-SUMMARY.md - -# Key source files: @src/store/workflowStore.ts -@src/types/index.ts @src/components/nodes/GenerateImageNode.tsx -**Tech stack available:** Zustand, workflow JSON persistence -**Established patterns:** Workflow load/save in workflowStore -**Constraining decisions:** -- Node type "nanoBanana" must remain unchanged in workflow files -- Legacy ModelType ("nano-banana", "nano-banana-pro") must still work +**Constraints:** Node type "nanoBanana" unchanged, legacy ModelType must work - Task 1: Add migration logic for loaded workflows + Task 1: Add workflow migration logic src/store/workflowStore.ts - 1. In loadWorkflow() or where workflow JSON is parsed, add migration: - - When loading a nanoBanana node without selectedModel field - - Automatically set selectedModel based on legacy model field: - ```typescript - if (node.type === "nanoBanana" && node.data.model && !node.data.selectedModel) { - node.data.selectedModel = { - provider: "gemini", - modelId: node.data.model, - displayName: node.data.model === "nano-banana" ? "Nano Banana" : "Nano Banana Pro" - }; - } - ``` - - 2. Keep legacy model field synced for saves: - - When selectedModel is Gemini, also update the model field - - This ensures old versions can still read the workflow - - 3. Add version field to workflow if not present: - - Set workflowVersion in saved JSON for future migrations - - Not strictly necessary but good practice - - WHY: Migration on load allows old workflows to work without manual editing. Syncing both fields maintains bidirectional compatibility. + In workflow loading (loadWorkflow or where JSON is parsed): + 1. For nanoBanana nodes without selectedModel, derive it from legacy model field: + ```typescript + if (node.type === "nanoBanana" && node.data.model && !node.data.selectedModel) { + node.data.selectedModel = { + provider: "gemini", + modelId: node.data.model, + displayName: node.data.model === "nano-banana" ? "Nano Banana" : "Nano Banana Pro" + }; + } + ``` + 2. Keep both fields synced on save for bidirectional compatibility - Load an existing workflow file, check node data has both model and selectedModel - Old workflows load with selectedModel populated, new saves include both fields + Load existing workflow file, check node has selectedModel populated + Old workflows load with selectedModel auto-populated - Task 2: Update executeWorkflow for multi-provider generation + Task 2: Update executeWorkflow for multi-provider src/store/workflowStore.ts - 1. In executeWorkflow()'s nanoBanana case: - - Check if node has selectedModel - - If selectedModel exists and provider !== "gemini", pass API keys in headers: - ```typescript - const headers: Record = { - "Content-Type": "application/json" - }; - if (nodeData.selectedModel?.provider === "replicate") { - const settings = get().providerSettings; - const replicateConfig = settings.providers.find(p => p.id === "replicate"); - if (replicateConfig?.apiKey) { - headers["X-Replicate-API-Key"] = replicateConfig.apiKey; - } - } - // Similar for fal - ``` - - 2. Build request payload: - - Include selectedModel in request body for new providers - - Keep existing fields for Gemini backward compatibility - - 3. Handle response format: - - Provider generate() returns GenerationOutput with outputs array - - Extract first image output as outputImage - - Validate output type is "image" (video handled by separate node in Phase 6) - - WHY: The store's execution logic needs to pass provider API keys (stored client-side) to the server route. + In executeWorkflow nanoBanana case: + 1. Check for selectedModel + 2. If non-Gemini provider, add API key headers from providerSettings: + ```typescript + const headers = { "Content-Type": "application/json" }; + if (nodeData.selectedModel?.provider === "replicate") { + const config = get().providerSettings.providers.find(p => p.id === "replicate"); + if (config?.apiKey) headers["X-Replicate-API-Key"] = config.apiKey; + } + // Similar for fal + ``` + 3. Include selectedModel in request body + 4. Handle GenerationOutput response format - Execute workflow with Gemini model, verify generation still works - Workflow execution handles both legacy and new model selection formats + Execute workflow with Gemini model + Execution works for both legacy and new model selection - Task 3: Update GenerateImageNode to handle legacy data + Task 3: Handle legacy data in GenerateImageNode UI src/components/nodes/GenerateImageNode.tsx - 1. When component mounts or data changes: - - If model exists but not selectedModel, derive selectedModel from model - - Display the correct provider/model in dropdowns - - 2. Ensure UI state reflects actual node data: - - Provider dropdown shows "gemini" for legacy nodes - - Model dropdown shows correct Gemini model name - - 3. When saving defaults, save in both formats: - - saveGenerateImageDefaults should save both model and selectedModel - - Ensures new nodes get correct defaults - - WHY: The UI must correctly display legacy nodes that were migrated on load. + 1. On mount/data change: if model exists but not selectedModel, derive selectedModel + 2. Ensure dropdowns reflect correct provider/model for migrated nodes + 3. Save defaults in both formats for compatibility - Open workflow with old nanoBanana node, verify dropdowns show correct selection - GenerateImageNode displays correct provider/model for both legacy and new nodes + Open workflow with old nanoBanana node, verify correct selection shown + UI correctly displays legacy nodes -Before declaring plan complete: -- [ ] `npm run build` succeeds without errors -- [ ] Load existing workflow with nanoBanana nodes - nodes display correctly -- [ ] Execute existing workflow - Gemini generation works -- [ ] Create new Generate node - defaults are correct -- [ ] Save and reload workflow - data persists correctly +- [ ] `npm run build` succeeds +- [ ] Existing workflow loads without errors +- [ ] Legacy Gemini execution works +- [ ] New provider selection persists correctly - -- All tasks completed -- All verification checks pass -- Existing workflows load and execute without errors -- Migration logic populates selectedModel from legacy model field -- Both old and new format nodes work in same workflow +- Existing workflows load and execute unchanged +- Migration auto-populates selectedModel +- Both formats work in same workflow - Phase 3 complete -After completion, create `.planning/phases/03-generate-node-refactor/03-03-SUMMARY.md` +Create `.planning/phases/03-generate-node-refactor/03-03-SUMMARY.md`