From d1b4b0213ef6725658b4a1dcb8efb282238e48e2 Mon Sep 17 00:00:00 2001 From: shrimbly Date: Mon, 2 Feb 2026 23:42:24 +1300 Subject: [PATCH] chore: remove .planning files from git tracking These files are already in .gitignore but were committed before the rule was added. Kept locally, removed from repo. Co-Authored-By: Claude Opus 4.5 --- .planning/ROADMAP.md | 697 ------------------ .planning/STATE.md | 240 ------ .../31-01-SUMMARY.md | 97 --- .../31-02-SUMMARY.md | 118 --- .../32-chat-ui-foundation/32-02-PLAN.md | 168 ----- .../34-02-SUMMARY.md | 96 --- .../34-03-SUMMARY.md | 130 ---- .../35-01-SUMMARY.md | 207 ------ .../35-02-SUMMARY.md | 102 --- .../35-03-SUMMARY.md | 125 ---- 10 files changed, 1980 deletions(-) delete mode 100644 .planning/ROADMAP.md delete mode 100644 .planning/STATE.md delete mode 100644 .planning/phases/31-workflow-proposal-system/31-01-SUMMARY.md delete mode 100644 .planning/phases/31-workflow-proposal-system/31-02-SUMMARY.md delete mode 100644 .planning/phases/32-chat-ui-foundation/32-02-PLAN.md delete mode 100644 .planning/phases/34-agentic-workflow-editing/34-02-SUMMARY.md delete mode 100644 .planning/phases/34-agentic-workflow-editing/34-03-SUMMARY.md delete mode 100644 .planning/phases/35-large-workflow-handling/35-01-SUMMARY.md delete mode 100644 .planning/phases/35-large-workflow-handling/35-02-SUMMARY.md delete mode 100644 .planning/phases/35-large-workflow-handling/35-03-SUMMARY.md diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md deleted file mode 100644 index 7922ac41..00000000 --- a/.planning/ROADMAP.md +++ /dev/null @@ -1,697 +0,0 @@ -# Roadmap: Node Banana Multi-Provider Support - -## Overview - -Transform Node Banana from a Gemini-only image generator into a multi-provider platform supporting Replicate and fal.ai. The journey builds provider infrastructure first, then adds dynamic model discovery, refactors the generate node for flexibility, creates a searchable model browser, adds local image serving for URL-based providers, and finishes with video support and polish. - -## Domain Expertise - -None - -## Milestones - -- ✅ **v1.0 Multi-Provider Support** - Phases 1-6 (shipped 2026-01-11) -- ✅ **v1.1 Improvements** - Phases 7-14 (shipped 2026-01-12) -- ✅ **v1.2 Improvements** - Phases 15-24 (shipped 2026-01-17) -- ✅ **v1.3 Improvements** - Phases 25-30 (shipped 2026-01-24) -- ✅ **v1.4 Features** - Phases 31-35, 40 (shipped 2026-02-01) -- 📋 **v1.5 Store Refactoring** - Phases 36-39 (planned) - -## Phases - -**Phase Numbering:** -- Integer phases (1, 2, 3): Planned milestone work -- Decimal phases (2.1, 2.2): Urgent insertions (marked with INSERTED) - -
-✅ v1.0 Multi-Provider Support (Phases 1-6) - SHIPPED 2026-01-11 - -### Phase 1: Provider Infrastructure -**Goal**: Users can configure Replicate and fal.ai API keys in settings, with keys securely stored -**Depends on**: Nothing (first phase) -**Research**: Unlikely (internal UI patterns, Zustand state management) -**Plans**: 2 plans - -Plans: -- [x] 01-01: Provider settings UI in ProjectSetupModal -- [x] 01-02: Provider abstraction layer and types - -### Phase 2: Model Discovery -**Goal**: App can fetch and cache available models from enabled providers at runtime -**Depends on**: Phase 1 -**Research**: Likely (external APIs) -**Research topics**: Replicate model listing API endpoints, fal.ai model discovery endpoints, response schemas and pagination -**Plans**: 3 plans - -Plans: -- [x] 02-01: Replicate model fetching API route -- [x] 02-02: fal.ai model fetching API route -- [x] 02-03: Model caching and unified model interface - -### Phase 3: Generate Node Refactor -**Goal**: NanoBanana node becomes GenerateImage node supporting any provider's image models -**Depends on**: Phase 2 -**Research**: Unlikely (internal refactoring using existing patterns) -**Design decision**: Separate nodes for image vs video generation (GenerateVideo added in Phase 6) -**Plans**: 3 plans - -Plans: -- [x] 03-01: Rename NanoBanana to GenerateImage, add model selector (image models only) -- [x] 03-02: Provider-specific execution in generate API route -- [x] 03-03: Backward compatibility for existing workflows - -### Phase 4: Model Search Dialog -**Goal**: Users can browse models via floating action bar icons and searchable dialog -**Depends on**: Phase 3 -**Research**: Unlikely (internal UI using React patterns) -**Plans**: 2 plans - -Plans: -- [x] 04-01: Floating action bar with provider icons -- [x] 04-02: Model search dialog with filtering and selection - -### Phase 5: Image URL Server -**Goal**: Local API endpoint serves workflow images as URLs for providers requiring URL inputs -**Depends on**: Phase 3 -**Research**: Unlikely (Next.js API routes, existing patterns) -**Plans**: 2 plans - -Plans: -- [x] 05-01: Image serving endpoint and URL generation -- [x] 05-02: Integration with generate node for URL-based providers - -### Phase 6: Video & Polish -**Goal**: GenerateVideo node for video generation, video playback, custom parameters, edge cases -**Depends on**: Phase 5 -**Research**: Likely (video handling) -**Research topics**: HTML5 video element for base64/blob URLs, provider response formats for video content -**Design decision**: GenerateVideo as separate node type (not combined with GenerateImage) -**Plans**: 4 plans - -Plans: -- [x] 06-01: GenerateVideo node with video-capable model selector -- [x] 06-02: Video playback in output node -- [x] 06-03: Custom model parameters from provider schemas -- [x] 06-04: Edge case handling and final polish - -
- -
-✅ v1.1 Improvements (Phases 7-14) - SHIPPED 2026-01-12 - -**Milestone Goal:** Fix connection issues, improve error visibility, add video history, auto-size nodes, and polish UI - -#### Phase 7: Video Connections ✅ - -**Goal**: Fix video handle connections to only allow valid targets (generateVideo, output) -**Depends on**: Phase 6 -**Research**: Unlikely (internal connection validation patterns) -**Plans**: 1 plan - -Plans: -- [x] 07-01: Add video handle type and connection validation - -#### Phase 8: Error Display ✅ - -**Goal**: Better error visibility with overlay display, not hidden by previous output -**Depends on**: Phase 7 -**Research**: Unlikely (internal UI patterns) -**Plans**: 1 plan - -Plans: -- [x] 08-01: Error notifications with persistent toast and node overlays - -#### Phase 9: Video History ✅ - -**Goal**: Add history carousel for generated videos matching image history pattern -**Depends on**: Phase 8 -**Research**: Unlikely (existing image history pattern) -**Plans**: 1 plan - -Plans: -- [x] 09-01: Video history types, store support, load API, and carousel UI - -#### Phase 10: Node Autosizing ✅ - -**Goal**: Auto-size nodes to output dimensions and aspect ratio -**Depends on**: Phase 9 -**Research**: Unlikely (React Flow node sizing) -**Plans**: 1 plan - -Plans: -- [x] 10-01: Node dimension utility and auto-resize on output - -#### Phase 11: UI Polish ✅ - -**Goal**: Flora UI alignment, header UI improvements, project settings, provider logos on nodes -**Depends on**: Phase 10 -**Research**: Unlikely (internal UI work) -**Plans**: 1 plan - -Plans: -- [x] 11-01: Provider badges on nodes and header UI streamlining - -#### Phase 12: Model Improvements ✅ - -**Goal**: Verify Replicate image models work, extend model cache TTL -**Depends on**: Phase 11 -**Research**: Likely (Replicate API behavior verification) -**Research topics**: Replicate image model endpoints, cache invalidation strategies -**Plans**: 1 plan - -Plans: -- [x] 12-01: Extended cache TTL, fixed isImageInput, fixed stale node data in execution - -#### Phase 13: Fix Duplicate Generations ✅ - -**Goal**: Fix image deduplication - generations folder has duplicate images due to hashing failure -**Depends on**: Phase 12 -**Research**: Likely (investigate current hashing implementation) -**Research topics**: Current save-generation hashing logic, why duplicates are being created -**Plans**: 1 plan - -Plans: -- [x] 13-01: Add MD5 content hashing and deduplication to save-generation API - -#### Phase 14: Fix Drag-Connect Node Creation Bugs ✅ - -**Goal**: Fix bugs with nodes created via drag-connect: (1) connection vanishes after selecting model from browser, (2) defaults to Gemini with missing model selector in header -**Depends on**: Phase 13 -**Research**: Unlikely (React Flow connection state, node creation flow) -**Research topics**: Connection state during node creation, model selection callbacks, createDefaultNodeData initialization -**Plans**: 1 plan - -Plans: -- [x] 14-01: Normalize dynamic handle IDs and fix connection persistence - -
- -### 🚧 v1.2 Improvements (In Progress) - -**Milestone Goal:** Add automated testing across the application, modularize large monolithic files for better maintainability, and improve cost tracking for multi-provider support - -#### Phase 15: Test Infrastructure - -**Goal**: Set up testing framework with Vitest and React Testing Library for Next.js 16 -**Depends on**: Phase 14 -**Research**: Likely (Vitest + Next.js 16 App Router setup) -**Research topics**: Vitest configuration for Next.js 16, React Testing Library setup, test file organization -**Plans**: TBD - -Plans: -- [x] 15-01: React Testing Library setup and configuration - -#### Phase 16: Store Modularization ✅ - -**Goal**: Break up workflowStore.ts into focused modules (execution, nodes, edges, persistence) -**Depends on**: Phase 15 -**Research**: Unlikely (internal Zustand patterns) -**Plans**: 1 plan - -Plans: -- [x] 16-01: Extract localStorage helpers and node defaults to utility modules - -#### Phase 17: Component Tests - -**Goal**: Add tests for all 34 React components covering nodes, toolbars, modals, and edges -**Depends on**: Phase 16 -**Research**: Unlikely (established React Testing Library patterns) -**Plans**: 11 plans - -Plans: -- [x] 17-01: Core Nodes (BaseNode, PromptNode, ImageInputNode) -- [x] 17-02: Display Nodes (OutputNode, SplitGridNode, GroupNode) -- [x] 17-03: Generate Nodes (GenerateImageNode, GenerateVideoNode) -- [x] 17-04: Processing Nodes (LLMGenerateNode, AnnotationNode) -- [x] 17-05: Toolbars (Header, FloatingActionBar, MultiSelectToolbar) -- [x] 17-06: Canvas & Edges (WorkflowCanvas, EditableEdge, ReferenceEdge, EdgeToolbar) -- [x] 17-07: Menus & Notifications (ConnectionDropMenu, Toast, CostIndicator) -- [x] 17-08: Core Modals (ModelSearchDialog, ProjectSetupModal, CostDialog) -- [x] 17-09: Editor Modals (PromptEditorModal, SplitGridSettingsModal, AnnotationModal) -- [x] 17-10: Quickstart (WelcomeModal, QuickstartInitialView, QuickstartTemplatesView, PromptWorkflowView, QuickstartBackButton) -- [x] 17-11: Utilities (GlobalImageHistory, GroupsOverlay, ModelParameters) - -#### Phase 18: API Route Tests - -**Goal**: Add tests for generate, llm, models, and workflow API routes -**Depends on**: Phase 17 -**Research**: Unlikely (Next.js API route testing patterns) -**Plans**: 5 plans - -Plans: -- [x] 18-01: File I/O routes (workflow, save-generation) -- [x] 18-02: LLM route (Google, OpenAI providers) -- [x] 18-03: Generate route (Gemini provider) -- [x] 18-04: Generate route (Replicate, fal.ai providers) -- [x] 18-05: Models route (caching, aggregation) - -#### Phase 19: Type Refactoring - -**Goal**: Split types/index.ts into domain-specific type files (nodes, providers, workflow) -**Depends on**: Phase 18 -**Research**: Unlikely (internal refactoring) -**Plans**: 2 plans - -Plans: -- [x] 19-01: Extract node and annotation types -- [x] 19-02: Extract provider, workflow, API, and model types - -#### Phase 20: Integration Tests - -**Goal**: End-to-end workflow execution tests covering node connections and data flow -**Depends on**: Phase 19 -**Research**: Unlikely (internal testing patterns) -**Plans**: 2 plans - -Plans: -- [x] 20-01: Store integration tests (getConnectedInputs, validateWorkflow, topological sort) -- [x] 20-02: Workflow execution tests (data flow, error handling, connection validation) - -#### Phase 21: Fix Image Input & Deduplication Issues - -**Goal**: Fix Gemini nano-banana-pro model ignoring image inputs and resolve duplicate image saving across input/generated images -**Depends on**: Phase 20 -**Research**: Likely (Gemini API image input handling, current hashing implementation) -**Research topics**: Gemini 3 Pro image generation API requirements, why image inputs are ignored, current save logic for inputs vs generations -**Plans**: 1 plan - -**Issues:** -1. ~~nano-banana-pro model generates without considering image inputs~~ - RESOLVED (no longer an issue) -2. Input and generated images have duplicate files despite different hashes - need consistent hashing approach matching video saving -3. Generated images should be prepended with prompt details like generated videos - -Plans: -- [x] 21-01: Unify MD5 hashing for image deduplication - -#### Phase 22: Generate Node Dynamic Input Tests - -**Goal**: Test that generate nodes properly validate and render dynamic inputs from provider schemas, and that all inputs/parameters are correctly included in API calls -**Depends on**: Phase 21 -**Research**: Unlikely (existing test patterns from Phase 17) -**Plans**: 1 plan - -**Test coverage:** -1. Dynamic inputs from provider schemas render correctly as parameters and input handles on node creation -2. Validation of dynamic inputs (required fields, type checking, constraints) -3. Standard inputs (image, text) validate properly -4. On submission, all parameters and inputs are included in API call payload correctly (covered by Phase 18) -5. Coverage across all providers (Gemini, Replicate, fal.ai) - -Plans: -- [x] 22-01: ModelParameters tests, GenerateImageNode/GenerateVideoNode dynamic handle tests - -#### Phase 23: Model Browser Improvements - -**Goal**: Improve model browser UX with recently used models section, icon-based provider dropdown with Gemini, and include Gemini models in browse list -**Depends on**: Phase 22 -**Research**: Unlikely (existing UI patterns) -**Plans**: TBD - -**Features:** -1. Recently used models section at top of browse dialog showing last 4 models used -2. Provider dropdown uses icons instead of text, add Gemini to the provider list -3. Include Gemini models (nano-banana, nano-banana-pro) in the browsable model list alongside Replicate/fal.ai models - -Plans: -- [x] 23-01: Recently used models, icon-based provider filter, Gemini models in browse - -#### Phase 24: Improved Cost Summary - -**Goal**: Expand cost tracking to include fal.ai models (via their pricing API), video generation nodes, and graceful handling for Replicate (no pricing API available) -**Depends on**: Phase 23 -**Research**: Complete (see research notes below) -**Plans**: 3 plans - -**Research Notes:** -- fal.ai has a pricing API: `GET /v1/models/pricing?endpoint_id=model1,model2` returns `{ prices: [{ endpoint_id, unit_price, unit, currency }] }` -- Replicate does NOT expose pricing via API (open GitHub issue, unresolved). Prediction response has `metrics.predict_time` but no hardware/cost info. -- `ProviderModel` type already has `pricing?: { type, amount, currency }` field at `src/lib/providers/types.ts:69-74` - -**UX Decision: Provider-Grouped with Uncertainty Section** -- Two sections: "Known Costs" (Gemini + fal.ai) and "Pricing Unavailable" (Replicate) -- Known costs show total at section header, breakdown by provider with icons -- fal.ai items show billing unit from API (e.g., "per image", "per 5s video") -- Replicate section includes help text: "Pricing varies by hardware and runtime. Check replicate.com" -- Incurred cost only tracks Gemini & fal.ai (Replicate excluded with note) -- Empty provider sections hidden - -**Features:** -1. Fetch fal.ai model pricing via their pricing API and populate `ProviderModel.pricing` -2. Track generateVideo nodes in cost predictions (currently ignored) -3. For Replicate models: display "pricing unavailable" gracefully in dedicated section -4. Update CostDialog with two-section layout: Known Costs (Gemini, fal.ai) and Pricing Unavailable (Replicate) -5. Show billing units for fal.ai models (per image, per second of video, etc.) -6. Incurred cost excludes Replicate with explanatory note - -Plans: -- [x] 24-01: fal.ai pricing API integration and ProviderModel.pricing population -- [x] 24-02: Expand costCalculator to handle video nodes and external providers -- [x] 24-03: Update CostDialog UI for multi-provider breakdown - -
-✅ v1.3 Improvements (Phases 25-30) - SHIPPED 2026-01-24 - -**Milestone Goal:** Add rich template exploration with search/filters, node default preferences in settings, and canvas performance optimizations - -#### Phase 25: Template Explorer UI - -**Goal**: Full template exploration interface with cards layout, left-hand filter/search panel, supporting many templates -**Depends on**: Phase 24 -**Research**: Unlikely (internal UI patterns) -**Plans**: 2 plans - -**Features:** -- Cards layout for template browsing (scalable for many templates) -- Left-hand panel with search and category filters -- Template metadata display (name, description, node count, etc.) -- Replace current quickstart template selection - -Plans: -- [x] 25-01: Template types, TemplateCard component, TemplateExplorerView grid layout -- [x] 25-02: Sidebar filters (search, category, tags), WelcomeModal integration - -#### Phase 26: Template Preview Rendering ✅ - -**Goal**: Visual preview of workflow templates showing node layout and connections before loading -**Depends on**: Phase 25 -**Research**: Unlikely (React Flow rendering, existing patterns) -**Plans**: 1 plan - -**Final Implementation:** -- Horizontal card layout with thumbnail image, details, and "Use workflow" button -- Two-column grid layout for better scanning -- Conditional dialog width (6xl for explorer, 2xl for other views) -- Direct workflow loading without intermediate modal - -Plans: -- [x] 26-01: Horizontal cards with direct workflow loading, conditional dialog sizing - -#### Phase 27: Node Defaults Infrastructure ✅ - -**Goal**: Store default model and settings preferences per node type in localStorage -**Depends on**: Phase 26 -**Research**: Unlikely (localStorage, existing patterns) -**Plans**: 1 plan - -**Features:** -- Schema for node type defaults (model, parameters, settings) -- Apply defaults when creating nodes via keyboard shortcuts (Shift+G, etc.) -- Per-provider model defaults (e.g., default fal.ai model for GenerateImage) - -Plans: -- [x] 27-01: NodeDefaultsConfig types, localStorage helpers, createDefaultNodeData integration - -#### Phase 28: Node Defaults UI ✅ - -**Goal**: Settings panel for configuring default node preferences per node type -**Depends on**: Phase 27 -**Research**: Unlikely (internal UI patterns) -**Plans**: 1 plan - -**Features:** -- Section in settings for node defaults -- Select default model per node type (GenerateImage, GenerateVideo, LLM) -- Configure default parameters (e.g., seedream 4.5 with specific settings) -- Clear/reset to system defaults -- Shift+V keyboard shortcut for video nodes - -Plans: -- [x] 28-01: Node Defaults tab in ProjectSetupModal with model selection and LLM controls - -#### Phase 29: Canvas Performance - -**Goal**: Optimize canvas rendering for large workflows using virtualization and memoization -**Depends on**: Phase 28 -**Research**: Likely (React Flow virtualization, performance optimization) -**Research topics**: React Flow performance best practices, node virtualization, selective rendering -**Plans**: TBD - -Plans: -- [ ] 29-01: TBD (run /gsd:plan-phase 29 to break down) - -#### Phase 30: Small Fixes - -**Goal**: [To be planned] -**Depends on**: Phase 29 -**Research**: Unlikely -**Plans**: TBD - -Plans: -- [ ] 30-01: TBD (run /gsd:plan-phase 30 to break down) - -
- -### ✅ v1.4 Features (Shipped 2026-02-01) - -**Milestone Goal:** Transform the Prompt-to-Workflow feature into a full agentic workflow builder with proposal dialogs, chat-based editing, and safe file handling for large workflows - -#### Phase 31: Workflow Proposal System - -**Goal**: Agent proposes workflow details before building with user feedback/approve flow -**Depends on**: Phase 30 -**Research**: Likely (LLM prompting for structured proposals) -**Research topics**: Structured output formats for workflow proposals, dialog UX patterns for agent communication -**Plans**: TBD - -Plans: -- [x] 31-01: Workflow proposal types, prompt, and API endpoint -- [x] 31-02: Chat infrastructure (Vercel AI SDK, streaming endpoint, types) - -#### Phase 32: Chat UI Foundation - -**Goal**: Floating chat window above minimap using Vercel AI SDK patterns -**Depends on**: Phase 31 -**Research**: Likely (Vercel AI SDK, chat UI components) -**Research topics**: Vercel AI SDK (ai package) integration, useChat hook, streaming responses, chat UI component patterns -**Plans**: 2 plans - -Plans: -- [x] 32-01: ChatPanel component with useChat hook, WorkflowCanvas integration -- [x] 32-02: Chat context and workflow generation integration - -#### Phase 33: Workflow Edit Safety - -**Goal**: Original file preservation before edits with accept/reject change flow -**Depends on**: Phase 32 -**Research**: Likely (versioning strategies, JSON diffing) -**Research topics**: Shadow copy vs diff-based vs undo stack approaches, JSON diff visualization -**Plans**: TBD - -**Plans**: 2 plans - -Plans: -- [x] 33-01-PLAN.md — Store snapshot state, capture/revert/clear actions, manual change tracking -- [x] 33-02-PLAN.md — Wire snapshot capture in AI flow, Revert AI Changes button in Header - -#### Phase 34: Context-Aware Agentic Workflow Editing - -**Goal**: Make the chat agent context-aware and multi-modal — routing user messages by intent to different behaviors: answering app usage questions, creating new workflows from scratch, or making targeted edits to the current workflow with full project context awareness and change narration -**Depends on**: Phase 33 -**Research**: Likely (intent classification, context-aware prompting, structured edits) -**Research topics**: Intent classification approaches (keyword vs LLM-based), system prompt design for context-aware editing, workflow JSON mutation strategies, project context injection, change explanation patterns -**Plans**: 3 plans - -**Features:** -1. Intent detection: classify user messages as help/question, new workflow creation, or workflow editing -2. Help mode: answer questions about how to use the app without modifying anything -3. Create mode: build new workflows from scratch (existing behavior) -4. Edit mode: interpret edit requests against the current workflow, make targeted JSON modifications -5. Project context awareness: agent understands current nodes, connections, models, and parameters -6. Change narration: explain what was modified and why - -Plans: -- [x] 34-01-PLAN.md — Chat agent library: tool definitions, edit operations, context builder -- [x] 34-02-PLAN.md — Enhanced /api/chat route with tool calling for intent routing -- [x] 34-03-PLAN.md — Store applyEditOperations, ChatPanel tool result handling, end-to-end wiring - -#### Phase 35: Large Workflow Handling - -**Goal**: Handle workflows with base64 images/videos efficiently, token management -**Depends on**: Phase 34 -**Research**: Likely (chunking, context window strategies) -**Research topics**: Token optimization for large payloads, base64 extraction/reinsertion, workflow summarization techniques -**Plans**: 3 plans - -Plans: -- [ ] 35-01-PLAN.md — Binary stripping utility and rich workflow context builder (TDD) -- [ ] 35-02-PLAN.md — Selection-aware subgraph extraction (TDD) -- [ ] 35-03-PLAN.md — Client-side wiring, ChatPanel selection chip, API subgraph integration - -#### Phase 40: Node Enhancements - -**Goal**: Add new node types and UI improvements — output gallery, image compare node, prompt constructor node, and image numbering on connections -**Depends on**: Phase 35 -**Research**: Likely (gallery UI patterns, image comparison approaches) -**Research topics**: Gallery/carousel component patterns, image diff/compare slider UX, prompt builder UX patterns, edge label rendering in React Flow -**Plans**: 4 plans - -**Features:** -1. Output gallery node — scrollable thumbnail grid with full-screen lightbox -2. Image compare node — slider overlay comparison of two images -3. Prompt constructor node — template-based prompts with @variable interpolation -4. Connection numbering — "Image N" labels on image edges when selected - -Plans: -- [x] 40-01-PLAN.md — OutputGallery node (thumbnail grid + lightbox) -- [x] 40-02-PLAN.md — Connection numbering (image edge sequence labels) -- [x] 40-03-PLAN.md — ImageCompare node (react-compare-slider) -- [x] 40-04-PLAN.md — PromptConstructor node (variable system + template interpolation) - -### 📋 v1.5 Store Refactoring (Planned) - -**Milestone Goal:** Major refactoring of workflowStore.ts (2,900+ lines) into modular, testable components. Extract execution engine, create Zustand slices, and improve code maintainability while maintaining full backward compatibility. - -**Problem Statement:** -- `workflowStore.ts` is 2,907 lines with 15+ mixed concerns -- `executeWorkflow` (~800 lines) and `regenerateNode` (~600 lines) contain node-specific logic for 8 node types inline -- Duplicated patterns (provider header building repeated 6+ times) -- Difficult to test individual concerns -- High cognitive load for any modifications - -**Target Architecture:** -``` -src/store/ -├── workflowStore.ts # ~100 lines - combines slices -├── slices/ # Zustand slice pattern -│ ├── coreSlice.ts # nodes, edges, clipboard -│ ├── groupsSlice.ts # group operations -│ ├── executionSlice.ts # execution state & actions -│ ├── persistenceSlice.ts # save/load, auto-save -│ ├── uiSlice.ts # modal state, quickstart -│ ├── costSlice.ts # cost tracking -│ ├── providersSlice.ts # provider settings -│ ├── historySlice.ts # global history, recent models -│ └── commentsSlice.ts # comment navigation -├── execution/ # Modular execution engine -│ ├── engine.ts # Core orchestration -│ ├── helpers.ts # Shared utilities -│ └── executors/ # Per-node-type executors -└── utils/ # Pure helper functions -``` - -#### Phase 36: Execution Engine Extraction - -**Goal**: Extract the massive executeWorkflow and regenerateNode functions into a modular execution engine with per-node-type executors -**Depends on**: Phase 35 -**Research**: Unlikely (internal Zustand patterns, existing codebase) -**Plans**: 3 plans - -**Scope:** -- Create `src/store/execution/` directory structure -- Extract execution types and interfaces -- Create node executor interface and registry -- Extract all 8 node-type executors (imageInput, annotation, prompt, nanoBanana, generateVideo, llmGenerate, splitGrid, output) -- Extract shared helpers (buildProviderHeaders, trackSaveGeneration, waitForPendingImageSyncs) -- Integrate engine back into workflowStore with same public API - -Plans: -- [ ] 36-01: Execution types, helpers, and engine orchestration -- [ ] 36-02: Node executors (all 8 types) with tests -- [ ] 36-03: Integrate engine with store, backward compatibility verification - -#### Phase 37: Pure Helpers Extraction - -**Goal**: Extract pure functions from the store into testable utility modules -**Depends on**: Phase 36 -**Research**: Unlikely (internal refactoring) -**Plans**: 2 plans - -**Scope:** -- Extract `getConnectedInputs` as pure function -- Extract `validateWorkflow` as pure function -- Extract `topologicalSort` logic -- Extract workflow migration logic (legacy nanoBanana nodes) -- Extract `clearNodeImageRefs` helper - -Plans: -- [ ] 37-01: getConnectedInputs, validateWorkflow, topologicalSort with tests -- [ ] 37-02: workflowMigration, clearNodeImageRefs with tests - -#### Phase 38: Zustand Slice Pattern - -**Goal**: Split monolithic store into composable Zustand slices for better separation of concerns -**Depends on**: Phase 37 -**Research**: Unlikely (Zustand slice patterns) -**Plans**: 3 plans - -**Scope:** -- Create slice pattern infrastructure -- Extract core slice (nodes, edges, edgeStyle, clipboard operations) -- Extract groups slice (all group operations) -- Extract execution slice (isRunning, currentNodeId, execute/regenerate actions) -- Extract persistence slice (save/load, auto-save, workflow metadata) -- Extract UI slice (modal count, quickstart, model search) -- Extract cost slice (incurredCost, cost tracking actions) -- Extract providers slice (providerSettings, API key management) -- Extract history slice (globalImageHistory, recentModels) -- Extract comments slice (comment navigation state) - -Plans: -- [ ] 38-01: Core slices (core, groups, execution) with tests -- [ ] 38-02: Feature slices (persistence, UI, cost, providers) with tests -- [ ] 38-03: Remaining slices (history, comments), final store composition - -#### Phase 39: Type Extraction & Final Integration - -**Goal**: Move store types to proper locations and verify full backward compatibility -**Depends on**: Phase 38 -**Research**: Unlikely (internal refactoring) -**Plans**: 1 plan - -**Scope:** -- Extract `WorkflowStore`, `WorkflowFile`, `ClipboardData`, `EdgeStyle` to `src/types/store.ts` -- Update all imports across codebase -- Verify all existing tests pass -- Verify all re-exports maintain backward compatibility -- Final cleanup and documentation - -Plans: -- [ ] 39-01: Extract store types, verify backward compatibility, final cleanup - -## Progress - -**Execution Order:** -Phases execute in numeric order: 1 → 2 → ... → 35 → 40 → 36 → 37 → 38 → 39 - -| Phase | Milestone | Plans Complete | Status | Completed | -|-------|-----------|----------------|--------|-----------| -| 1. Provider Infrastructure | v1.0 | 2/2 | Complete | 2026-01-09 | -| 2. Model Discovery | v1.0 | 3/3 | Complete | 2026-01-09 | -| 3. Generate Node Refactor | v1.0 | 3/3 | Complete | 2026-01-09 | -| 4. Model Search Dialog | v1.0 | 2/2 | Complete | 2026-01-09 | -| 5. Image URL Server | v1.0 | 2/2 | Complete | 2026-01-09 | -| 6. Video & Polish | v1.0 | 4/4 | Complete | 2026-01-11 | -| 7. Video Connections | v1.1 | 1/1 | Complete | 2026-01-12 | -| 8. Error Display | v1.1 | 1/1 | Complete | 2026-01-12 | -| 9. Video History | v1.1 | 1/1 | Complete | 2026-01-12 | -| 10. Node Autosizing | v1.1 | 1/1 | Complete | 2026-01-12 | -| 11. UI Polish | v1.1 | 1/1 | Complete | 2026-01-12 | -| 12. Model Improvements | v1.1 | 1/1 | Complete | 2026-01-12 | -| 13. Fix Duplicate Generations | v1.1 | 1/1 | Complete | 2026-01-12 | -| 14. Fix Drag-Connect Node Creation Bugs | v1.1 | 1/1 | Complete | 2026-01-12 | -| 15. Test Infrastructure | v1.2 | 1/1 | Complete | 2026-01-12 | -| 16. Store Modularization | v1.2 | 1/1 | Complete | 2026-01-12 | -| 17. Component Tests | v1.2 | 11/11 | Complete | 2026-01-13 | -| 18. API Route Tests | v1.2 | 5/5 | Complete | 2026-01-13 | -| 19. Type Refactoring | v1.2 | 2/2 | Complete | 2026-01-13 | -| 20. Integration Tests | v1.2 | 2/2 | Complete | 2026-01-13 | -| 21. Fix Image Input & Deduplication | v1.2 | 1/1 | Complete | 2026-01-13 | -| 22. Generate Node Dynamic Input Tests | v1.2 | 1/1 | Complete | 2026-01-13 | -| 23. Model Browser Improvements | v1.2 | 1/1 | Complete | 2026-01-13 | -| 24. Improved Cost Summary | v1.2 | 3/3 | Complete | 2026-01-17 | -| 25. Template Explorer UI | v1.3 | 2/2 | Complete | 2026-01-16 | -| 26. Template Preview Rendering | v1.3 | 1/1 | Complete | 2026-01-17 | -| 27. Node Defaults Infrastructure | v1.3 | 1/1 | Complete | 2026-01-17 | -| 28. Node Defaults UI | v1.3 | 1/1 | Complete | 2026-01-17 | -| 29. Canvas Performance | v1.3 | 0/0 | Deferred | - | -| 30. Small Fixes | v1.3 | 0/0 | Deferred | - | -| 31. Workflow Proposal System | v1.4 | 2/2 | Complete | 2026-01-26 | -| 32. Chat UI Foundation | v1.4 | 2/2 | Complete | 2026-01-27 | -| 33. Workflow Edit Safety | v1.4 | 2/2 | Complete | 2026-01-30 | -| 34. Context-Aware Agentic Editing | v1.4 | 3/3 | Complete | 2026-01-31 | -| 35. Large Workflow Handling | v1.4 | 3/3 | Complete | 2026-01-31 | -| 40. Node Enhancements | v1.4 | 4/4 | Complete | 2026-02-01 | -| 36. Execution Engine Extraction | v1.5 | 0/3 | Not started | - | -| 37. Pure Helpers Extraction | v1.5 | 0/2 | Not started | - | -| 38. Zustand Slice Pattern | v1.5 | 0/3 | Not started | - | -| 39. Type Extraction & Final Integration | v1.5 | 0/1 | Not started | - | diff --git a/.planning/STATE.md b/.planning/STATE.md deleted file mode 100644 index 8483d9d7..00000000 --- a/.planning/STATE.md +++ /dev/null @@ -1,240 +0,0 @@ -# Project State - -## Project Reference - -See: .planning/PROJECT.md (updated 2026-01-09) - -**Core value:** Provider infrastructure that dynamically discovers models from external APIs, enabling users to access hundreds of image/video generation models without hardcoding schemas. -**Current focus:** Store Refactoring (v1.5) - -## Current Position - -Phase: 40 of 40 (Node Enhancements) -Plan: 4 of 4 in current phase -Status: Complete -Last activity: 2026-02-01 - Phase 40 complete (all 4 node enhancements shipped) - -Progress: ████████████ 100% - -## Performance Metrics - -**Velocity:** -- Total plans completed: 40 -- Average duration: 6.6 min -- Total execution time: 4.50 hours - -**By Phase:** - -| Phase | Plans | Total | Avg/Plan | -|-------|-------|-------|----------| -| 1. Provider Infrastructure | 2/2 | 14 min | 7 min | -| 2. Model Discovery | 3/3 | 14 min | 4.7 min | -| 3. Generate Node Refactor | 3/3 | 13 min | 4.3 min | -| 4. Model Search Dialog | 2/2 | 17 min | 8.5 min | -| 5. Image URL Server | 2/2 | 5 min | 2.5 min | -| 6. Video & Polish | 4/4 | 43 min | 14.3 min | -| 7. Video Connections | 1/1 | 4 min | 4 min | -| 8. Error Display | 1/1 | 14 min | 14 min | -| 9. Video History | 1/1 | 12 min | 12 min | -| 10. Node Autosizing | 1/1 | 2 min | 2 min | -| 11. UI Polish | 1/1 | 8 min | 8 min | -| 12. Model Improvements | 1/1 | - | - | -| 13. Fix Duplicate Generations | 1/1 | 1 min | 1 min | -| 14. Fix Drag-Connect Bugs | 1/1 | 7 min | 7 min | -| 15. Test Infrastructure | 1/1 | 3 min | 3 min | -| 16. Store Modularization | 1/1 | 22 min | 22 min | -| 19. Type Refactoring | 2/2 | 16 min | 8 min | -| 22. Generate Node Dynamic Input Tests | 1/1 | 20 min | 20 min | -| 23. Model Browser Improvements | 1/1 | 5 min | 5 min | -| 20. Integration Tests | 2/2 | 11 min | 5.5 min | -| 24. Improved Cost Summary | 3/3 | 25 min | 8 min | -| 25. Template Explorer UI | 2/2 | 23 min | 11.5 min | -| 26. Template Preview Rendering | 1/1 | 25 min | 25 min | -| 27. Node Defaults Infrastructure | 1/1 | 15 min | 15 min | -| 28. Node Defaults UI | 1/1 | 32 min | 32 min | -| 31. Workflow Proposal System | 2/2 | 6 min | 3 min | -| 32. Chat UI Foundation | 2/2 | 9 min | 4.5 min | -| 33. Workflow Edit Safety | 2/2 | 5 min | 5 min | -| 34. Agentic Workflow Editing | 3/3 | 13 min | 4.3 min | -| 35. Large Workflow Handling | 3/3 | 18 min | 6 min | -| 40. Node Enhancements | 3/4 | 13 min | 4.3 min | - -**Recent Trend:** -- Last 5 plans: 5 min, 2 min, 5 min, 8 min, 5 min -- Trend: Phase 40 in progress - 3/4 plans complete (OutputGallery, Connection numbering, ImageCompare) - -## Accumulated Context - -### Decisions - -Decisions are logged in PROJECT.md Key Decisions table. -Recent decisions affecting current work: - -- Gemini always enabled via env var (GEMINI_API_KEY), Replicate/fal.ai optional -- API keys stored in localStorage under node-banana-provider-settings key -- Local state in modal to avoid saving on every keystroke -- Provider config pattern: {id, name, enabled, apiKey, apiKeyEnvVar?} -- Provider registry uses self-registration pattern via registerProvider() -- Gemini remains special-cased in /api/generate for now, not yet migrated -- Capability inference from model name/description keywords -- fal.ai API key optional (works without but rate limited) -- fal.ai auth header: "Key {apiKey}" format (not Bearer) -- fal.ai category maps directly to ModelCapability (no inference) -- 1-hour cache TTL for model lists (extended from 10 min) -- Unified API at /api/models with header-based auth -- Provider dropdown shows Gemini always, others only if API key configured -- Aspect ratio/resolution controls shown only for Gemini provider -- Backward compatibility via aliases: NanoBananaNode, saveNanoBananaDefaults -- Server-side provider execution in API route (not client-side) -- Header-based API key passing: X-Replicate-API-Key, X-Fal-API-Key -- fal.ai sync API (fal.run) instead of queue-based async -- Dual migration approach: loadWorkflow migrates + UI effect for runtime -- fal.ai icon always visible in action bar (works without key but rate limited) -- Replicate icon only visible when API key is configured -- Client-side search filtering for Replicate (their search API unreliable) -- Show all capability badges to differentiate similar models -- Extract variant suffix from fal.ai model IDs for display name -- No TTL for image store - explicit cleanup pattern (callers delete after use) -- 256KB threshold for shouldUseImageUrl (Replicate recommendation) -- Gemini excluded from video node (doesn't support video generation) -- Large videos (>20MB) return URL instead of base64 to avoid memory issues -- Fetch schema from provider API at model selection time with 10-min cache -- Filter internal params, prioritize user-relevant ones (seed, steps, guidance) -- Collapsible parameters section to keep node UI compact -- Node autosizing constraints: 200-500px width, 200-600px height, ~100px chrome -- Provider badges prepend node title (left side) with w-4 h-4 icons -- Node titles show only model name (no "Generate Image/Video" prefix) -- BaseNode supports titlePrefix prop for icon prepending -- Header aligned for saved/unsaved states with same icon layout -- isImageInput() uses word-boundary checks (not substring) to avoid matching num_images -- Workflow execution gets fresh node data from store (not stale captured array) -- regenerateNode includes parameters in request body -- MD5 content hashing for generation deduplication (fast, collision resistance not critical) -- Hash prefix in filename for O(1) duplicate lookup -- Normalized handle IDs (image, text, image-0) for connection stability across model changes -- Handle-to-schema mapping built at execution time from inputSchema -- Placeholder handles (dimmed 30%) for input types not used by model, preserving connections -- ReactFlowProvider wrapper for component tests using @xyflow/react hooks -- Zustand store mocking with vi.mock pattern returning mocked functions -- Vitest jsdom environment for React component tests -- Store utilities extracted to src/store/utils/ with re-exports for backward compatibility -- Consolidated defaultNodeDimensions (was duplicated in addNode and createGroup) -- localStorage mocking pattern for testing utility modules -- Type domain files live in src/types/*.ts with re-exports from index.ts -- BaseNodeData in annotation.ts to avoid circular imports (nodes.ts imports annotation.ts) -- 7 type domain files: annotation, nodes, providers, models, workflow, api, quickstart -- index.ts is pure re-export hub with no type definitions -- Max 8 recent models stored in localStorage, 4 displayed in UI -- Gemini models hardcoded in /api/models (not fetched from external API) -- Green color theme for Gemini provider (bg-green-500/20 text-green-300) -- Cost dialog: Gemini Cost section (with prices) + External Providers section (with model links) -- External provider pricing removed (fal.ai 429 errors, Replicate no API) - show model links instead -- Incurred cost only tracks Gemini generations -- Template category colors: blue=product, purple=style, green=composition, amber=community -- Template grid layout: 2 cols mobile, 3 cols lg+ -- Template node count calculated at runtime from workflow.nodes.length -- onWheelCapture pattern for isolating modal scroll from React Flow canvas -- min-h-0 on flexbox containers enables overflow scrolling -- overflow-clip instead of overflow-hidden when child needs scroll -- Node defaults UI uses local state pattern (load on open, save on button click) -- ModelSearchDialog reused with onModelSelected callback for defaults UI -- LLM_PROVIDERS/LLM_MODELS duplicated from LLMGenerateNode (not exported) -- Shift+V keyboard shortcut for video node creation -- Proposal types focus on purpose/description, not internal state or positions -- Reuse parseJSONFromResponse from validation.ts for consistent JSON parsing -- Validate proposal shape before returning to catch LLM errors early -- Use createGoogleGenerativeAI factory for custom API key injection (AI SDK) -- toTextStreamResponse for AI SDK v6 streaming (not toDataStreamResponse) -- DefaultChatTransport with api option for useChat hook in AI SDK v6 -- Manage input state locally (AI SDK v6 removed built-in input state from useChat) -- Use toUIMessageStreamResponse() for useChat hook compatibility -- Build Workflow button extracts user messages only (not assistant responses) -- Workflow generation uses contentLevel "full" for complete workflows -- Chat panel closes automatically after successful workflow generation -- AI workflow snapshot state: previousWorkflowSnapshot + manualChangeCount -- Deep copy workflow state using JSON.parse(JSON.stringify()) for snapshots -- Snapshot auto-clears after 3 manual structural changes -- Manual changes: add/remove nodes, add/remove edges (not position/selection) -- clearSnapshot called in clearWorkflow and loadWorkflow -- dynamicInputs type is Record to support multi-image aggregation -- Single image stays as string; only multiple images to same schema key become array -- Array.isArray guard on dynamicInputs.prompt access (takes first element) -- AI SDK v6 tool pattern uses inputSchema (not parameters) for zod schemas -- Chat agent tools use "generate" pattern (no execute function) - LLM provides structured output -- Edit operations use batched immutable updates with skip tracking for invalid operations -- Node IDs for AI-generated nodes: ${nodeType}-ai-${Date.now()}-${index} pattern -- Workflow context builder strips base64 data, history arrays, and internal state for LLM consumption -- /api/chat accepts optional workflowState (nodes/edges) from client for context-aware routing -- AI SDK v6 uses stopWhen: stepCountIs(N) for multi-step tool execution (not maxSteps) -- Chat API uses toolChoice: 'auto' to let LLM decide which tool to call based on intent -- System prompt dynamically built per request with workflow context via buildEditSystemPrompt -- Custom fetch wrapper pattern for injecting extra body data with useChat (AI SDK v6) -- Tool invocation parts use part.type === "tool-{name}" with input property (AI SDK v6) -- state === "output-available" for tool completion detection (not "result") -- AI edits do NOT increment manualChangeCount (only manual edits clear snapshot) -- Snapshot captured before AI edits via captureSnapshot() in handleApplyEdits -- chatWorkflowState strips base64 from nodes to reduce API request size -- stripBinaryData() utility strips all base64, history arrays, and ref fields with metadata placeholders -- Binary field metadata format: [image: context, sizeKB] or [video: sizeKB] or [N image(s)] -- Base64 size estimation: (dataUrl.length * 3) / 4 / 1024 for KB (no library needed) -- History arrays (imageHistory, videoHistory) completely removed from LLM context -- Ref fields (imageRef, outputImageRef, etc.) completely removed from LLM context -- All node parameters, positions, and model settings preserved in stripped context -- Enhanced WorkflowContext includes full StrippedNode[] with all non-binary data -- createdAt timestamp on edge data for stable connection ordering -- Image edge sequence numbers shown only when 2+ connections to same target -- EdgeToolbar displays "Image N" labels for multi-image connections -- OutputGallery node uses createPortal for full-screen lightbox (avoids z-index conflicts) -- Real-time image collection pattern: useMemo watching edges/nodes for live updates -- nowheel class for scroll isolation inside React Flow nodes -- Pink minimap color (#ec4899) for OutputGallery nodes -- PromptNode variable naming with optional variableName field (backward compatible) -- @ icon in PromptNode header indicates variable status (blue when set, dimmed when not) -- PromptConstructor template resolution uses @variable syntax with alphanumeric + underscore -- Autocomplete triggers on @ character with keyboard navigation (arrows, enter, tab, escape) -- Unresolved @variables show warning badge but execution proceeds with literal text -- Template resolution happens during workflow execution, not on template change -- ImageCompare node with react-compare-slider for side-by-side visual comparison -- Horizontal-only slider (portrait=false) for image comparison - no vertical toggle -- Multi-input nodes use distinct handle positions (35% and 65% vertical spacing) -- Handle IDs for multi-input: 'image' (first) and 'image-1' (second) for stable connections -- Teal minimap color (#14b8a6) for ImageCompare nodes - -### Deferred Issues - -- UAT-001: Resolved - Provider icons now use real Replicate/fal.ai logos -- ISS-001: Resolved - Generate nodes now adapt to model requirements via dynamic parameters - -### Blockers/Concerns - -- Pre-existing lint configuration issue (ESLint not configured). Not blocking development. - -### Quick Tasks Completed - -| # | Description | Date | Commit | Directory | -|---|-------------|------|--------|-----------| -| 001 | Fix multiple image inputs lost in dynamicInputs | 2026-01-30 | 1564a1b | [001-fix-multiple-image-inputs-lost-in-dynamicinputs](./quick/001-fix-multiple-image-inputs-lost-in-dynamicinputs/) | - -### Roadmap Evolution - -- v1.0 Multi-Provider Support shipped: 6 phases (Phase 1-6), 15 plans -- Milestone v1.1 Improvements created: 6 phases (Phase 7-12), improvements and polish -- Phase 13 added: Fix duplicate generations (hashing failure investigation) -- Phase 14 added: Fix drag-connect node creation bugs (consolidated from two phases) -- Milestone v1.2 Improvements created: 6 phases (Phase 15-20), testing and modularization -- Phase 21 added: Fix Gemini Pro image input handling and image deduplication issues -- Phase 22 added: Generate node dynamic input validation and API call tests -- Phase 23 added: Model browser improvements (recently used, icon dropdown, Gemini models in list) -- Phase 24 added: Improved cost summary (fal.ai pricing API, video node tracking, multi-provider support) -- Milestone v1.3 Improvements created: 5 phases (Phase 25-29), template explorer, node defaults, canvas performance -- Phase 30 added: Small fixes -- Milestone v1.4 Features created: 5 phases (Phase 31-35), agentic workflow builder with proposal dialogs and chat-based editing -- Milestone v1.5 Store Refactoring created: 4 phases (Phase 36-39), major refactor of workflowStore.ts into modular execution engine and Zustand slices -- Phase 40 added to v1.4: Node enhancements — output gallery, image compare node, prompt constructor node, image numbering on connections - -## Session Continuity - -Last session: 2026-02-01 -Stopped at: Phase 40 complete - all 4 node enhancements shipped. v1.4 milestone complete. -Resume file: None -Next action: v1.4 milestone complete. Ready for v1.5 (Store Refactoring) or new priorities. diff --git a/.planning/phases/31-workflow-proposal-system/31-01-SUMMARY.md b/.planning/phases/31-workflow-proposal-system/31-01-SUMMARY.md deleted file mode 100644 index 912b4d7f..00000000 --- a/.planning/phases/31-workflow-proposal-system/31-01-SUMMARY.md +++ /dev/null @@ -1,97 +0,0 @@ ---- -phase: 31-workflow-proposal-system -plan: 01 -subsystem: api -tags: [quickstart, workflow-proposal, llm, gemini, typescript] - -# Dependency graph -requires: - - phase: 30-quickstart-experience - provides: quickstart API patterns, validation utilities, prompts.ts structure -provides: - - WorkflowProposal types for reviewable workflow structure - - buildProposalPrompt function for proposal generation - - /api/quickstart/propose endpoint for generating proposals -affects: [31-02, quickstart-ui, workflow-generation] - -# Tech tracking -tech-stack: - added: [] - patterns: [proposal-before-build, reviewable-workflow-structure] - -key-files: - created: - - src/lib/quickstart/proposalPrompt.ts - - src/app/api/quickstart/propose/route.ts - modified: - - src/types/quickstart.ts - -key-decisions: - - "Proposal types focus on purpose/description, not internal state or positions" - - "Reuse parseJSONFromResponse from validation.ts for consistent JSON parsing" - - "Validate proposal shape before returning to catch LLM errors early" - -patterns-established: - - "ProposedNode.purpose: Human-readable explanation of node's role in workflow" - - "ProposedConnection.description: Data flow explanation for user understanding" - - "WorkflowProposal.warnings: Surface limitations before user commits to workflow" - -issues-created: [] - -# Metrics -duration: 12min -completed: 2026-01-26 ---- - -# Phase 31-01: Workflow Proposal Infrastructure Summary - -**WorkflowProposal types, buildProposalPrompt function, and /api/quickstart/propose endpoint for reviewable workflow structure before JSON generation** - -## Performance - -- **Duration:** 12 min -- **Started:** 2026-01-26T15:00:00Z -- **Completed:** 2026-01-26T15:12:00Z -- **Tasks:** 3 -- **Files modified:** 3 - -## Accomplishments -- WorkflowProposal types enable reviewable structure with purpose descriptions -- buildProposalPrompt generates LLM prompt focused on human-readable descriptions -- /api/quickstart/propose endpoint returns structured proposals for user review - -## Task Commits - -Each task was committed atomically: - -1. **Task 1: Define WorkflowProposal types** - `602130a` (feat) -2. **Task 2: Create proposal generation prompt** - `0e42a7d` (feat) -3. **Task 3: Create /api/quickstart/propose endpoint** - `c9b887b` (feat) - -**Plan metadata:** TBD (docs: complete plan) - -## Files Created/Modified -- `src/types/quickstart.ts` - Added ProposedNode, ProposedConnection, ProposedGroup, WorkflowProposal types -- `src/lib/quickstart/proposalPrompt.ts` - buildProposalPrompt function for proposal generation -- `src/app/api/quickstart/propose/route.ts` - POST endpoint for generating workflow proposals - -## Decisions Made -- **Proposal types focus on purpose**: Unlike full workflow JSON, proposals emphasize human-readable descriptions for each node and connection. No positions, no internal state. -- **Reuse validation utilities**: parseJSONFromResponse already handles markdown extraction and error recovery. -- **Shape validation before return**: The endpoint validates that LLM output matches expected structure to fail fast on malformed responses. - -## Deviations from Plan - -None - plan executed exactly as written - -## Issues Encountered -None - -## Next Phase Readiness -- Proposal endpoint is ready for ProposalReviewView UI integration (Plan 02) -- Types are exported and can be imported by components -- Endpoint tested via build verification - ---- -*Phase: 31-workflow-proposal-system* -*Completed: 2026-01-26* diff --git a/.planning/phases/31-workflow-proposal-system/31-02-SUMMARY.md b/.planning/phases/31-workflow-proposal-system/31-02-SUMMARY.md deleted file mode 100644 index 8c63ec50..00000000 --- a/.planning/phases/31-workflow-proposal-system/31-02-SUMMARY.md +++ /dev/null @@ -1,118 +0,0 @@ ---- -phase: 31-workflow-proposal-system -plan: 02 -subsystem: api -tags: [vercel-ai-sdk, streaming, gemini, chat, ai-sdk-google] - -# Dependency graph -requires: - - phase: 31-01 - provides: WorkflowProposal types, domain prompts -provides: - - Streaming /api/chat endpoint - - Chat types (ChatMessage, ChatRole, ConversationState) - - Vercel AI SDK integration with Gemini -affects: [31-03-chat-ui, workflow-planning] - -# Tech tracking -tech-stack: - added: [ai@6.0.49, @ai-sdk/google@3.0.13] - patterns: [createGoogleGenerativeAI, streamText, toTextStreamResponse] - -key-files: - created: - - src/types/chat.ts - - src/app/api/chat/route.ts - modified: - - src/types/index.ts - - package.json - -key-decisions: - - "Use createGoogleGenerativeAI factory for custom API key injection" - - "toTextStreamResponse for AI SDK v6 streaming (not toDataStreamResponse)" - - "System prompt contains full Node Banana domain expertise for conversational planning" - -patterns-established: - - "Vercel AI SDK streaming pattern: createGoogleGenerativeAI → streamText → toTextStreamResponse" - -issues-created: [] - -# Metrics -duration: 2min -completed: 2026-01-26 ---- - -# Phase 31 Plan 02: Chat Infrastructure Summary - -**Streaming chat endpoint with Vercel AI SDK, Gemini integration, and conversational workflow planning assistant** - -## Performance - -- **Duration:** 2 min -- **Started:** 2026-01-26T09:37:57Z -- **Completed:** 2026-01-26T09:40:07Z -- **Tasks:** 4 -- **Files modified:** 4 - -## Accomplishments -- Installed Vercel AI SDK packages (ai, @ai-sdk/google) -- Created chat types for conversation state management -- Built streaming /api/chat endpoint with domain expertise -- Exported chat types from central type index - -## Task Commits - -Each task was committed atomically: - -1. **Task 1: Install Vercel AI SDK packages** - `744e0a7` (chore) -2. **Task 2: Create chat types** - `b263c03` (feat) -3. **Task 3: Create streaming chat endpoint** - `727e54d` (feat) -4. **Task 4: Update type exports** - `e558777` (feat) - -## Files Created/Modified -- `src/types/chat.ts` - ChatMessage, ChatRole, ConversationState, ChatRequest types -- `src/app/api/chat/route.ts` - Streaming POST endpoint with Node Banana system prompt -- `src/types/index.ts` - Added chat types re-export -- `package.json` - Added ai and @ai-sdk/google dependencies - -## Decisions Made -- Used `createGoogleGenerativeAI` factory instead of default `google` export to inject API key from GEMINI_API_KEY env var -- Used `toTextStreamResponse()` which is the correct method in AI SDK v6 (not toDataStreamResponse) -- System prompt includes comprehensive Node Banana domain knowledge for conversational workflow planning - -## Deviations from Plan - -### Auto-fixed Issues - -**1. [Rule 3 - Blocking] Fixed google() API usage** -- **Found during:** Task 3 (Create streaming chat endpoint) -- **Issue:** Plan specified `google('gemini-2.5-flash', { apiKey })` but @ai-sdk/google doesn't accept second argument -- **Fix:** Used `createGoogleGenerativeAI({ apiKey })` factory to create provider with custom API key -- **Files modified:** src/app/api/chat/route.ts -- **Verification:** Build succeeds -- **Committed in:** 727e54d (Task 3 commit) - -**2. [Rule 3 - Blocking] Fixed streaming response method** -- **Found during:** Task 3 (Create streaming chat endpoint) -- **Issue:** `toDataStreamResponse()` doesn't exist in AI SDK v6 -- **Fix:** Changed to `toTextStreamResponse()` which is the correct method -- **Files modified:** src/app/api/chat/route.ts -- **Verification:** Build succeeds -- **Committed in:** 727e54d (Task 3 commit) - ---- - -**Total deviations:** 2 auto-fixed (2 blocking) -**Impact on plan:** Both fixes required for build to succeed. API signature differences between plan and actual AI SDK v6. - -## Issues Encountered -None beyond the API fixes above. - -## Next Phase Readiness -- Chat streaming infrastructure complete -- Ready for Chat UI integration in WelcomeModal (31-03) -- Vercel AI SDK useChat hook can connect to /api/chat endpoint - ---- -*Phase: 31-workflow-proposal-system* -*Completed: 2026-01-26* diff --git a/.planning/phases/32-chat-ui-foundation/32-02-PLAN.md b/.planning/phases/32-chat-ui-foundation/32-02-PLAN.md deleted file mode 100644 index 63e1bef4..00000000 --- a/.planning/phases/32-chat-ui-foundation/32-02-PLAN.md +++ /dev/null @@ -1,168 +0,0 @@ ---- -phase: 32-chat-ui-foundation -plan: 02 -type: execute ---- - - -Connect chat conversation to workflow generation, enabling users to build workflows from chat discussions. - -Purpose: Complete the chat-to-workflow flow where users can discuss workflow ideas with the assistant, see a proposal, and generate the workflow with one click. -Output: ChatPanel with "Build Workflow" button, proposal display in chat, and workflow loading integration. - - - -~/.claude/get-shit-done/workflows/execute-phase.md -./summary.md - - - -@.planning/PROJECT.md -@.planning/ROADMAP.md -@.planning/STATE.md -@.planning/phases/32-chat-ui-foundation/32-01-SUMMARY.md -@.planning/phases/31-workflow-proposal-system/31-01-SUMMARY.md -@.planning/phases/31-workflow-proposal-system/31-02-SUMMARY.md - -# Key files from Phase 31 and 32-01: -@src/components/ChatPanel.tsx -@src/app/api/chat/route.ts -@src/types/quickstart.ts -@src/app/api/quickstart/propose/route.ts -@src/app/api/quickstart/route.ts - -# Store for workflow loading: -@src/store/workflowStore.ts - -**Tech stack available:** ai@6.0.49, @ai-sdk/google@3.0.13, @ai-sdk/react -**Established patterns:** -- useChat hook with DefaultChatTransport -- WorkflowProposal types for reviewable structure -- /api/quickstart/propose for proposal generation -- /api/quickstart for workflow JSON generation -- loadWorkflow() to load WorkflowFile into store - -**Constraining decisions:** -- Phase 31: Proposal types focus on purpose/description, not internal state -- Phase 32-01: Manage input state locally (AI SDK v6) -- Phase 32-01: Use toUIMessageStreamResponse() for useChat compatibility - - - - - - Task 1: Add "Build Workflow" button and proposal state to ChatPanel - src/components/ChatPanel.tsx - -Extend ChatPanel to support workflow building: - -1. Add props for workflow building: - - onBuildWorkflow: (description: string) => Promise - callback to parent - -2. Add internal state: - - buildingWorkflow: boolean - loading state for the build button - -3. Add "Build Workflow" button below the input area: - - Only visible when there are assistant messages (conversation has started) - - Disabled while buildingWorkflow is true - - On click: extract conversation context and call onBuildWorkflow prop - - Style: w-full bg-green-600 hover:bg-green-500, similar to send button style - - Text: "Build Workflow" or "Building..." when loading - -4. Extract conversation description: - - Combine user messages to form the description for the proposal - - Simple approach: join all user message texts with newlines - - Pass this to onBuildWorkflow callback - -Keep the existing chat functionality unchanged. The parent component (WorkflowCanvas) will handle the actual API calls. - - TypeScript compiles, button appears when conversation exists - ChatPanel has "Build Workflow" button that passes conversation context to parent callback - - - - Task 2: Integrate workflow generation in WorkflowCanvas - src/components/WorkflowCanvas.tsx - -Connect ChatPanel to workflow generation flow: - -1. Add state for workflow building: - - isBuildingWorkflow: boolean - -2. Create handleBuildWorkflow async function: - - Set isBuildingWorkflow to true - - Call /api/quickstart with description and contentLevel "full" - - On success: call loadWorkflow() from store with the generated workflow - - On success: close the chat panel (setIsChatOpen(false)) - - On error: show toast error message - - Always set isBuildingWorkflow to false in finally block - -3. Pass handleBuildWorkflow to ChatPanel: - - setIsChatOpen(false)} - onBuildWorkflow={handleBuildWorkflow} - isBuildingWorkflow={isBuildingWorkflow} - /> - -4. Update ChatPanel props interface to accept: - - onBuildWorkflow: (description: string) => Promise - - isBuildingWorkflow?: boolean (optional, defaults to false) - -Use existing toast system for error display. The workflow loading will clear the canvas and load the new workflow. - - npm run build succeeds, clicking "Build Workflow" calls API and loads workflow - Clicking "Build Workflow" generates workflow via API and loads it onto canvas - - - - Task 3: Update ChatPanel to use external loading state - src/components/ChatPanel.tsx - -Update ChatPanel to use the loading state from props instead of internal state: - -1. Update ChatPanelProps interface: - - Add isBuildingWorkflow?: boolean prop - -2. Remove internal buildingWorkflow state (if added in Task 1), use prop instead - -3. Update "Build Workflow" button: - - Disabled when isBuildingWorkflow is true OR when isLoading (chat streaming) - - Show "Building..." text when isBuildingWorkflow is true - - Show spinner or loading dots when building - -4. Ensure button doesn't interfere with chat streaming: - - Chat input remains functional during workflow building - - But "Build Workflow" button is disabled to prevent double submissions - -This task ensures the loading state is properly synchronized between ChatPanel and WorkflowCanvas. - - npm run dev, start conversation, click Build Workflow, see loading state, workflow loads - Build workflow button shows proper loading state synced with parent component - - - - - -Before declaring phase complete: -- [ ] `npm run build` succeeds without errors -- [ ] Chat button opens ChatPanel -- [ ] After starting a conversation, "Build Workflow" button appears -- [ ] Clicking "Build Workflow" shows loading state -- [ ] Workflow is generated and loaded onto canvas -- [ ] Chat panel closes after successful workflow load -- [ ] Error shows toast if generation fails -- [ ] No TypeScript errors - - - -- All tasks completed -- All verification checks pass -- No errors or warnings introduced -- Chat conversation can be converted to a workflow -- End-to-end flow works: chat -> discuss -> build -> workflow on canvas - - - -After completion, create `.planning/phases/32-chat-ui-foundation/32-02-SUMMARY.md` - diff --git a/.planning/phases/34-agentic-workflow-editing/34-02-SUMMARY.md b/.planning/phases/34-agentic-workflow-editing/34-02-SUMMARY.md deleted file mode 100644 index 5ed63d80..00000000 --- a/.planning/phases/34-agentic-workflow-editing/34-02-SUMMARY.md +++ /dev/null @@ -1,96 +0,0 @@ -# Plan 34-02 Execution Summary - -**Phase:** 34 - Context-Aware Agentic Workflow Editing -**Plan:** 02 - Enhanced Chat API Route with Tool Calling -**Status:** Complete -**Date:** 2026-01-30 - -## Objective - -Enhance the `/api/chat` route to use AI SDK tool calling for intent-based routing between help, create, and edit modes. The chat API becomes context-aware by accepting workflow state from the client, building a context-enriched system prompt, and using tool calling to let the LLM choose the right action. - -## Tasks Completed - -### Task 1: Rewrite /api/chat route with tool calling ✅ -**Commit:** `7602dac` - feat(34-02): enhance chat API route with tool-based intent routing - -**Implementation:** -- Rewrote `src/app/api/chat/route.ts` with tool-based intent routing -- Added imports: `stepCountIs` from 'ai', `createChatTools`, `buildEditSystemPrompt` from tools, `buildWorkflowContext` from contextBuilder, `WorkflowNode`, `WorkflowEdge` types -- Request body now accepts `{ messages: UIMessage[], workflowState?: { nodes: WorkflowNode[], edges: WorkflowEdge[] } }` -- Build workflow context: `buildWorkflowContext(workflowState?.nodes || [], workflowState?.edges || [])` -- Generate context-aware system prompt: `buildEditSystemPrompt(context)` -- Extract node IDs for tool validation: `nodeIds = workflowState?.nodes.map(n => n.id)` -- Create tools: `createChatTools(nodeIds)` returns answerQuestion, createWorkflow, editWorkflow -- Configure `streamText` with: - - `model: google('gemini-2.5-flash')` - - `system: systemPrompt` (context-enriched) - - `messages: modelMessages` - - `tools: tools` - - `toolChoice: 'auto'` (LLM decides) - - `stopWhen: stepCountIs(3)` (multi-step reasoning) -- Return `result.toUIMessageStreamResponse()` for useChat hook compatibility -- Removed old static SYSTEM_PROMPT constant (replaced by dynamic prompt from tools.ts) - -**Verification:** -- `npx tsc --noEmit` passes for chat route (no errors in src/app/api/chat/route.ts) -- Route compiles correctly with all new imports and tool configuration -- All type errors are pre-existing in test files, unrelated to this change - -## Deviations from Plan - -**Deviation 1: Changed `maxSteps` to `stopWhen: stepCountIs(3)` (Rule 1 - Auto-fix bug)** -- **Issue:** Plan specified `maxSteps: 3`, but AI SDK v6 doesn't support that parameter -- **Resolution:** Used `stopWhen: stepCountIs(3)` instead, which is the correct API for controlling multi-step tool execution in AI SDK v6 -- **Impact:** Same behavior (allow up to 3 steps), correct API usage -- **Type:** Auto-fix bug (incorrect API parameter name) - -## Key Technical Decisions - -1. **Tool pattern:** Used "generate" pattern (no execute function) - LLM generates structured output matching zod schema, results stream back to client for application -2. **Context injection:** Workflow state is optional for backward compatibility when no workflow is loaded -3. **System prompt:** Dynamic prompt built per request with current workflow context (replaces static prompt) -4. **Multi-step reasoning:** Allow up to 3 steps via `stopWhen: stepCountIs(3)` for handling complex edit requests -5. **Streaming:** Results stream via `toUIMessageStreamResponse()` for seamless useChat hook integration - -## Verification - -- ✅ `npx tsc --noEmit` passes for chat route -- ✅ Route accepts workflowState in request body -- ✅ System prompt includes current workflow context -- ✅ Tools are properly passed to streamText -- ✅ Response streams correctly via toUIMessageStreamResponse - -## Success Criteria Met - -- ✅ /api/chat route uses tool calling with answerQuestion, createWorkflow, editWorkflow -- ✅ Workflow state is accepted from client and injected into system prompt -- ✅ LLM chooses appropriate tool based on user message intent -- ✅ Streaming response compatible with useChat hook on client - -## Files Modified - -- `src/app/api/chat/route.ts` - Enhanced with tool calling and workflow context awareness - -## Dependencies - -- **Imports from:** `src/lib/chat/tools.ts` (createChatTools, buildEditSystemPrompt) -- **Imports from:** `src/lib/chat/contextBuilder.ts` (buildWorkflowContext) -- **Uses:** AI SDK v6 tool calling with streamText -- **Blocks:** Plan 34-03 (ChatPanel client integration) - needs this API to be ready - -## Next Steps - -Plan 34-03 will enhance the ChatPanel component to: -- Send workflow state to /api/chat -- Read tool call results from streaming messages -- Apply edit operations returned by editWorkflow tool -- Invoke quickstart API for createWorkflow tool -- Display answers from answerQuestion tool - -## Notes - -- All test file errors are pre-existing and unrelated to this change -- The route maintains backward compatibility (workflowState is optional) -- Tool results are streamed to client without server-side execution (client applies changes) -- Context-aware prompt gives LLM full visibility into current workflow for intelligent routing diff --git a/.planning/phases/34-agentic-workflow-editing/34-03-SUMMARY.md b/.planning/phases/34-agentic-workflow-editing/34-03-SUMMARY.md deleted file mode 100644 index 5259e5cc..00000000 --- a/.planning/phases/34-agentic-workflow-editing/34-03-SUMMARY.md +++ /dev/null @@ -1,130 +0,0 @@ -# Phase 34 Plan 03 - Store Integration and ChatPanel Wiring - Summary - -**Status**: ✅ Complete -**Date**: 2026-01-30 -**Execution**: Human-verified checkpoint (approved) - -## Objective - -Wire `applyEditOperations` into workflowStore and integrate ChatPanel with tool invocation handling. Enable the chat UI to process LLM tool calls (createWorkflow, editWorkflow) and apply workflow mutations, completing the full agentic workflow editing pipeline. - -## Tasks Completed - -### Task 1: Add applyEditOperations to workflowStore and wire ChatPanel ✅ -**Commit**: f5c4830 - -Modified three files: - -**workflowStore.ts**: -- Added `EditOperation` import from `lib/chat/editOperations` -- Added `applyEditOperations` action method that wraps the library function -- Applies edit operations with access to store state (nodes, edges, updateNodeData, etc.) -- Returns `ApplyEditResult` with applied count and skipped operations - -**ChatPanel.tsx**: -- Added `onApplyEdits` prop: `(operations: EditOperation[], explanation: string) => void` -- Added `workflowState` prop: `{ nodes: Node[], edges: Edge[] }` for context-aware requests -- Created custom `fetch` wrapper to inject `workflowState` into POST body - - Merges `workflowState` with request body from AI SDK - - Preserves AI SDK's default fetch behavior for all other requests -- Added `useEffect` to process tool invocations when `state === "output-available"` - - Extracts tool parts using `part.type === "tool-{name}"` pattern (AI SDK v6) - - Handles `createWorkflow` tool: calls `/api/quickstart` with description - - Handles `editWorkflow` tool: calls `onApplyEdits(operations, explanation)` -- Enhanced message rendering: - - Tool invocation parts display with green/blue text indicators - - Shows tool name and formatted arguments inline - -**WorkflowCanvas.tsx**: -- Added `chatWorkflowState` memo that strips base64 data from nodes/edges - - Uses `buildWorkflowContext` logic to remove image/outputImage fields - - Keeps workflow structure for LLM context without payload bloat -- Added `handleApplyEdits` callback: - - Captures workflow snapshot via `captureSnapshot()` before applying AI edits - - Calls `workflowStore.applyEditOperations(operations)` - - Logs applied/skipped operations to console -- Passes `onApplyEdits={handleApplyEdits}` and `workflowState={chatWorkflowState}` to ChatPanel - -### Task 2: Checkpoint human-verify ✅ -**Status**: APPROVED by user - -Human verification confirmed: -- Tool invocations display correctly in chat UI -- createWorkflow tool successfully generates workflows via quickstart API -- editWorkflow tool successfully applies edit operations to store -- Workflow context stripped of base64 data in API requests -- Snapshot captured before AI edits (manual changes can still clear it) - -## Verification - -- ✅ Tool invocations render inline with clear indicators -- ✅ createWorkflow tool calls /api/quickstart and loads result -- ✅ editWorkflow tool applies operations via store -- ✅ workflowState passed to API for context-aware routing -- ✅ Snapshot captured before AI edits (not after - AI edits don't count as manual) -- ✅ Custom fetch wrapper preserves AI SDK behavior while injecting extra data - -## Files Modified - -1. `src/store/workflowStore.ts` - - Added `EditOperation` import - - Added `applyEditOperations` action method - -2. `src/components/ChatPanel.tsx` - - Added `onApplyEdits` and `workflowState` props - - Custom fetch wrapper for workflowState injection - - Tool invocation processing via useEffect - - Enhanced message rendering for tool display - -3. `src/components/WorkflowCanvas.tsx` - - Added `chatWorkflowState` memo (strips base64) - - Added `handleApplyEdits` callback (captures snapshot) - - Passes new props to ChatPanel - -## Deviations - -**Custom fetch wrapper instead of `body` option**: -- Plan specified using `body` option with useChat hook -- AI SDK v6 `DefaultChatTransport` doesn't support `body` directly -- Solution: Custom fetch wrapper that merges workflowState into POST body -- Preserves all AI SDK default behavior while injecting extra data - -**Tool invocation part structure**: -- Plan referenced `toolName` and `args` properties -- AI SDK v6 uses `part.type === "tool-{name}"` pattern with `input` property -- Adapted code to match actual AI SDK v6 message part structure - -**Tool completion detection**: -- Plan referenced checking `state === "result"` -- AI SDK v6 uses `state === "output-available"` for tool completion -- Updated useEffect dependency to correct state value - -## Key Implementation Decisions - -1. **AI edits do NOT increment manualChangeCount**: Only manual user edits (add/remove nodes/edges) clear the snapshot. AI edits preserve the snapshot for undo/comparison. - -2. **Snapshot timing**: Captured BEFORE AI edits via `captureSnapshot()`, not after. This preserves pre-AI state for potential rollback. - -3. **Custom fetch pattern**: Wrapper function checks for POST requests with body, merges workflowState, passes through everything else unchanged. - -4. **Tool invocation rendering**: Inline display with color-coded indicators (green for createWorkflow, blue for editWorkflow) using `JSON.stringify` for args. - -5. **Base64 stripping**: `chatWorkflowState` memo removes image/outputImage fields from all nodes to keep API request size reasonable while preserving structure. - -6. **Tool state detection**: Uses `state === "output-available"` and checks for tool parts in final message to trigger processing. - -## Integration Points - -This completes the full agentic workflow editing pipeline: -- ✅ Chat agent library (phase 34 plan 01) - provides tools and edit operations -- ✅ Chat API route (phase 34 plan 02) - handles tool calling via AI SDK -- ✅ Chat UI integration (phase 34 plan 03) - processes tool results and applies edits - -Users can now: -- Ask questions about workflow building → `answerQuestion` tool -- Request new workflows → `createWorkflow` tool → quickstart API -- Request workflow edits → `editWorkflow` tool → `applyEditOperations` - -## Next Steps - -Phase 34 (Context-Aware Agentic Workflow Editing) is complete. Proceed to next phase in milestone v1.4 or other priorities. diff --git a/.planning/phases/35-large-workflow-handling/35-01-SUMMARY.md b/.planning/phases/35-large-workflow-handling/35-01-SUMMARY.md deleted file mode 100644 index d7ebbfd2..00000000 --- a/.planning/phases/35-large-workflow-handling/35-01-SUMMARY.md +++ /dev/null @@ -1,207 +0,0 @@ ---- -phase: 35-large-workflow-handling -plan: 01 -subsystem: chat -tags: [context-builder, binary-stripping, metadata-placeholders, tdd] -requires: [phase-34-complete, contextBuilder.ts] -provides: [stripBinaryData, enhanced-buildWorkflowContext, rich-node-parameters] -affects: [src/lib/chat/contextBuilder.ts, src/lib/chat/contextBuilder.test.ts] -tech-stack: [vitest, typescript] -key-files: - - src/lib/chat/contextBuilder.ts - - src/lib/chat/contextBuilder.test.ts -key-decisions: - - Binary stripping uses metadata placeholders with size/dimensions - - History arrays completely removed (imageHistory, videoHistory, etc.) - - Ref fields completely removed (imageRef, outputImageRef, etc.) - - Format: [image: 1024x768, 245KB] or [image: model-name, 245KB] - - All node parameters, positions, and settings preserved -duration: 3 minutes -completed: 2026-01-31 ---- - -# Plan 35-01 Summary: Binary Stripping and Rich Workflow Context - -## One-liner -Implemented comprehensive base64 data stripping with metadata placeholders and enhanced workflow context to send full node parameters without bloating HTTP payloads. - -## Performance -- **Duration**: 3 minutes -- **Test Coverage**: 22 tests, 100% pass rate -- **Files Modified**: 2 -- **Lines Added**: 742 (569 test + 173 implementation) -- **Binary Field Coverage**: 8 node types, 15 total binary fields - -## Accomplishments - -### Core Features Delivered -1. **stripBinaryData() utility** - - Strips base64 data from all 8 node types - - Replaces with rich metadata placeholders - - Preserves all non-binary parameters - - Removes history and ref fields - -2. **Enhanced buildWorkflowContext()** - - Returns full StrippedNode[] with parameters - - Includes node positions for layout reasoning - - Includes source/target handles on connections - - Maintains backward-compatible interface - -3. **Metadata Placeholder System** - - Single images: `[image: 1024x768, 245KB]` with dimensions - - Generated images: `[image: Gemini 3 Pro, 245KB]` with model - - Image arrays: `[2 image(s)]` for multiple inputs - - Videos: `[video: 1024KB]` - - Empty arrays: `[no images]` - -### Binary Field Inventory -Comprehensive coverage across all node types: -- **imageInput**: `image` -- **annotation**: `sourceImage`, `outputImage` -- **nanoBanana**: `inputImages`, `outputImage` -- **generateVideo**: `inputImages`, `outputVideo` -- **llmGenerate**: `inputImages` -- **splitGrid**: `sourceImage` -- **output**: `image`, `video` -- **prompt**: (none) - -### Stripped Fields -- **History**: `imageHistory`, `videoHistory`, `selectedHistoryIndex`, `selectedVideoHistoryIndex` -- **Refs**: `imageRef`, `outputImageRef`, `sourceImageRef`, `inputImageRefs`, `outputVideoRef` - -### Preserved Fields -All node parameters remain intact: -- Model settings (aspectRatio, resolution, model, selectedModel) -- Generation parameters (temperature, maxTokens, useGoogleSearch, parameters) -- Status fields (status, error) -- User fields (customTitle, comment, prompt) -- Node positions (x, y) - -## Task Commits - -### RED (Test) -- **a344aea**: test(35-01): add comprehensive tests for binary stripping and rich context - - 22 tests covering all node types - - Binary field stripping validation - - Parameter preservation checks - - Exhaustive base64 detection test - -### GREEN (Implementation) -- **e9abe1c**: feat(35-01): implement binary stripping and rich workflow context - - stripBinaryData() with metadata generation - - Enhanced WorkflowContext interface - - Updated buildWorkflowContext() and formatContextForPrompt() - - All tests passing - -### REFACTOR -- None needed - implementation is clean and well-structured - -## Files Created/Modified - -### Created -- `src/lib/chat/contextBuilder.test.ts` (569 lines) - - Comprehensive test suite for all node types - - Binary stripping validation - - Context building tests - -### Modified -- `src/lib/chat/contextBuilder.ts` (+173 lines) - - Added stripBinaryData() utility - - Added helper functions (estimateBase64Size, formatBinaryPlaceholder) - - Enhanced buildWorkflowContext() to use stripBinaryData - - Updated WorkflowContext interface with StrippedNode[] - - Added constants: BINARY_FIELDS_BY_TYPE, HISTORY_FIELDS, REF_FIELDS - -## Decisions Made - -1. **Metadata Format: Context First, Then Size** - - Format: `[type: context, sizeKB]` - - Examples: `[image: 1024x768, 245KB]`, `[image: Gemini 3 Pro, 245KB]` - - Rationale: Context is more important than exact size for LLM reasoning - -2. **Base64 Size Estimation** - - Formula: `(dataUrl.length * 3) / 4 / 1024` for KB - - Rationale: Standard base64 decoding math, no library needed - - Accurate enough for metadata purposes - -3. **Null/Undefined Binary Fields** - - Keep as-is (don't add placeholder) - - Rationale: LLM should know when a field is empty vs. stripped - -4. **History Array Removal** - - Complete removal (not even metadata) - - Rationale: Carousel history is irrelevant for workflow editing context - -5. **Export stripBinaryData()** - - Made public for client-side use in WorkflowCanvas - - Rationale: Plan 35-02 needs it for chatWorkflowState construction - -## Deviations from Plan - -None. Implementation follows plan exactly: -- All binary fields stripped per RESEARCH.md inventory -- Metadata placeholders match spec -- History/ref fields removed -- Parameters preserved -- TDD workflow followed (RED → GREEN) - -## Issues Encountered - -### Issue 1: Test Expectation Order -- **Problem**: Initial test expected `.*KB.*context` but implementation generated `context.*KB` -- **Solution**: Adjusted test expectation to match more logical format (context first) -- **Impact**: None - format is internal to system, LLM doesn't care about order - -### Issue 2: Fake Base64 Size Estimation -- **Problem**: Test base64 strings were minimal, resulting in "0KB" estimates -- **Solution**: Tests validate format pattern, not exact size (size testing not critical) -- **Impact**: None - real base64 data will have accurate sizes - -## Next Phase Readiness - -### Completed Deliverables -- ✅ stripBinaryData() utility exported and tested -- ✅ Enhanced buildWorkflowContext() with full node parameters -- ✅ Metadata placeholder system working -- ✅ 100% test coverage for all node types - -### Ready for Plan 35-02 -Plan 35-02 (Client-Side Integration) can now: -- Import and use stripBinaryData() in WorkflowCanvas -- Construct chatWorkflowState with stripped nodes -- Pass rich context to /api/chat without binary bloat - -### Blocked Items -None. All dependencies satisfied. - -### Integration Points -- WorkflowCanvas.tsx will use stripBinaryData() for chatWorkflowState -- ChatPanel.tsx will receive enhanced context with parameters -- /api/chat/route.ts will use buildWorkflowContext() for system prompt - -## Test Results - -```bash -npx vitest run src/lib/chat/contextBuilder.test.ts - -✓ src/lib/chat/contextBuilder.test.ts (22 tests) 4ms - ✓ stripBinaryData (14 tests) - ✓ imageInput nodes (2 tests) - ✓ annotation nodes (1 test) - ✓ nanoBanana nodes (3 tests) - ✓ generateVideo nodes (1 test) - ✓ llmGenerate nodes (1 test) - ✓ splitGrid nodes (1 test) - ✓ output nodes (1 test) - ✓ prompt nodes (1 test) - ✓ common fields (2 tests) - ✓ exhaustive binary stripping (1 test) - ✓ buildWorkflowContext (5 tests) - ✓ formatContextForPrompt (3 tests) - -Test Files 1 passed (1) -Tests 22 passed (22) -Duration 656ms -``` - -All tests passing with comprehensive coverage. diff --git a/.planning/phases/35-large-workflow-handling/35-02-SUMMARY.md b/.planning/phases/35-large-workflow-handling/35-02-SUMMARY.md deleted file mode 100644 index 651910dd..00000000 --- a/.planning/phases/35-large-workflow-handling/35-02-SUMMARY.md +++ /dev/null @@ -1,102 +0,0 @@ ---- -phase: 35-large-workflow-handling -plan: 02 -subsystem: chat-context -tags: [tdd, selection, subgraph, context-scoping] -requires: - - src/types/nodes.ts (WorkflowNode type) - - src/types/workflow.ts (WorkflowEdge type) -provides: - - src/lib/chat/subgraphExtractor.ts (extractSubgraph function) - - SubgraphResult interface for selection-aware context -affects: - - Future chat context builder integration (plan 35-03) -tech-stack: - - TypeScript - - Vitest (testing) -key-files: - - src/lib/chat/subgraphExtractor.ts (89 lines) - - src/lib/chat/subgraphExtractor.test.ts (289 lines) -key-decisions: - - Use Set for O(1) node lookup during edge classification - - Boundary connections track both direction and handle type - - Empty selection returns all nodes/edges with isScoped=false - - Type breakdown uses node.type directly (not node data) - - Handle type from sourceHandle/targetHandle with fallback to "unknown" -duration: 10 minutes -completed: 2026-01-31 ---- - -# Selection-aware subgraph extraction for focused LLM context - -## Performance - -**Velocity:** 10 minutes (plan 35-02) -**Code output:** 378 lines (89 implementation + 289 tests) -**Test coverage:** 8 comprehensive test cases, all passing -**Efficiency:** Clean TDD flow (RED -> GREEN, no refactor needed) - -## Accomplishments - -Implemented `extractSubgraph()` function that splits a workflow into detailed selected nodes and a lightweight summary of the rest. This enables context-aware chat where users can select specific nodes to focus the LLM's attention, crucial for large workflows with 50+ nodes. - -Key features: -- O(1) node lookup using Set for efficient edge classification -- Three-way edge classification: fully within selection, boundary, outside -- Boundary connections identify direction (incoming/outgoing) and handle type -- Type breakdown counts unselected nodes by type for LLM context -- Empty selection returns full workflow (no scoping) - -## Task Commits - -| Phase | Commit | Description | -|-------|--------|-------------| -| RED | d102922 | test(35-02): add failing tests for subgraph extraction | -| GREEN | e44bdfa | feat(35-02): implement extractSubgraph for selection-aware context | - -## Files Created/Modified - -**Created:** -- `src/lib/chat/subgraphExtractor.ts` - Core extraction function with SubgraphResult interface -- `src/lib/chat/subgraphExtractor.test.ts` - Comprehensive test suite (8 test cases) - -**Modified:** -- None (net new functionality) - -## Decisions Made - -1. **Set-based lookup** - Use `new Set(selectedNodeIds)` for O(1) membership testing during edge classification (scales well for large workflows) - -2. **Boundary direction semantics** - "incoming" when target is selected (data flows INTO selection), "outgoing" when source is selected (data flows OUT of selection) - -3. **Handle type tracking** - Store sourceHandle/targetHandle on boundary connections so LLM knows what type of data is crossing the boundary (image vs text) - -4. **Empty selection behavior** - When `selectedNodeIds` is empty, return all nodes/edges with `isScoped=false` (no scoping applied) - -5. **Type breakdown source** - Use `node.type` directly rather than inspecting node data (simpler, faster) - -## Deviations from Plan - -None. Implementation followed plan exactly: -- SubgraphResult interface as specified -- All required test cases implemented -- Edge classification logic matches spec -- Boundary connection structure with direction/handleType fields - -## Issues Encountered - -None. Clean TDD execution with no blockers. - -## Next Phase Readiness - -**Ready for plan 35-03 (Chat Context Builder Integration):** -- extractSubgraph function exported and tested -- SubgraphResult interface provides all needed data -- Boundary connections enable LLM to understand data flow across selection -- Type breakdown gives high-level overview of unselected nodes - -**Integration points for 35-03:** -- Call extractSubgraph() when user has selected nodes -- Pass selectedNodes to detailed context builder -- Format restSummary into concise text summary for LLM -- Include boundary connections in context (e.g., "Node B receives image from Node A outside selection") diff --git a/.planning/phases/35-large-workflow-handling/35-03-SUMMARY.md b/.planning/phases/35-large-workflow-handling/35-03-SUMMARY.md deleted file mode 100644 index d6ad8333..00000000 --- a/.planning/phases/35-large-workflow-handling/35-03-SUMMARY.md +++ /dev/null @@ -1,125 +0,0 @@ ---- -phase: 35-large-workflow-handling -plan: 03 -subsystem: chat-integration -tags: [client-side, selection-ui, api-integration, subgraph] -requires: - - src/lib/chat/subgraphExtractor.ts (extractSubgraph) - - src/lib/chat/contextBuilder.ts (stripBinaryData) -provides: - - Selection-aware chat workflow state - - Selection chip UI in ChatPanel - - API-side subgraph context building -affects: - - src/components/WorkflowCanvas.tsx (chatWorkflowState computation) - - src/components/ChatPanel.tsx (selection UI, error handling) - - src/app/api/chat/route.ts (subgraph-aware context) - - src/lib/chat/tools.ts (system prompt) -tech-stack: - - React - - TypeScript - - Next.js API Routes - - AI SDK v6 -key-files: - - src/components/WorkflowCanvas.tsx (chatWorkflowState with stripped data) - - src/components/ChatPanel.tsx (selection chip UI) - - src/app/api/chat/route.ts (subgraph extraction) - - src/lib/chat/tools.ts (system prompt updates) -key-decisions: - - Client-side binary stripping (not API-side) for smaller HTTP payloads - - selectedNodeIds derived from node.selected state - - Selection chip dismissible per session (resets when selection changes) - - Server-side subgraph extraction in API route - - System prompt includes metadata placeholder notes - - 413 error handling for oversized payloads -duration: 5 minutes -completed: 2026-01-31 ---- - -# Client-side wiring, ChatPanel selection chip, API subgraph integration - -## Performance - -**Started:** 2026-01-31 -**Completed:** 2026-01-31 -**Duration:** ~5 minutes -**Code output:** 4 files modified -**Efficiency:** Clean execution phase, all tasks completed successfully - -## Accomplishments - -Completed the full integration of selection-aware chat context for large workflows: - -1. **Client-side binary stripping** - `chatWorkflowState` uses `stripBinaryData()` to create rich but compact node data, removing base64 images/videos and large arrays while preserving all parameters and settings - -2. **Selection-aware subgraph scoping** - `selectedNodeIds` computed from `nodes.filter(n => n.selected)` and passed to chat API for focused context - -3. **Selection chip UI** - Dismissible "Focused on N selected nodes" chip appears above chat input when nodes are selected, providing clear visual feedback - -4. **API-side subgraph extraction** - `extractSubgraph()` called in chat API route before context building, enabling detailed focus on selected nodes with lightweight summaries of the rest - -5. **Error handling** - Oversized payload errors return friendly 413 message to user - -6. **System prompt updates** - Enhanced with metadata placeholder notes and subgraph summary section to guide LLM understanding of stripped context - -## Task Commits - -| Task | Commit | Description | -|------|--------|-------------| -| 1 | aaafef4 | feat(35-03): rewrite chatWorkflowState with selection context | -| 2 | 89fca5f | feat(35-03): update API route and system prompt for subgraph-aware context | -| 3 | (checkpoint) | Human verification - approved by user | - -## Files Created/Modified - -**Created:** -- None (all modifications to existing files) - -**Modified:** -- `src/components/WorkflowCanvas.tsx` - Rich stripped `chatWorkflowState` with `selectedNodeIds` -- `src/components/ChatPanel.tsx` - Selection chip UI, `selectedNodeIds` forwarding, error handling -- `src/app/api/chat/route.ts` - Subgraph-aware context building, 413 error handling -- `src/lib/chat/tools.ts` - System prompt with metadata awareness and subgraph summary - -## Decisions Made - -1. **Client-side stripping** - Call `stripBinaryData()` in WorkflowCanvas (not API-side) to reduce HTTP payload size before network transmission - -2. **Selection derivation** - Compute `selectedNodeIds` from `nodes.filter(n => n.selected).map(n => n.id)` rather than maintaining separate state - -3. **Chip dismissibility** - Selection chip dismissible per session using local `useState`, resets when selection changes (chip reappears) - -4. **Server-side extraction** - Subgraph extraction happens in API route (not client-side) to keep client bundle smaller and logic centralized - -5. **Metadata placeholders** - System prompt explicitly notes that `[image: ...]`, `[video: ...]`, `[N image(s)]` are non-editable metadata placeholders representing stripped binary data - -6. **Error handling** - 413 status for token/size issues with friendly message: "Workflow too large to send. Try selecting fewer nodes." - -## Deviations from Plan - -None. Implementation followed plan exactly: -- Binary stripping integrated client-side -- Selection chip UI implemented as specified -- API route modified for subgraph-aware context -- System prompt enhanced with metadata notes -- All checkpoint requirements met - -## Issues Encountered - -None. Clean execution with no blockers. All tasks completed successfully and checkpoint approved. - -## Next Phase Readiness - -**Phase 35 complete** - All 3 plans finished: -- Plan 01: Binary stripping and rich workflow context (TDD) -- Plan 02: Selection-aware subgraph extraction (TDD) -- Plan 03: Client-side wiring and integration (execution) - -**System capabilities:** -- Chat agent receives rich stripped node data with all parameters -- Users can select nodes to focus LLM attention on specific workflow areas -- Selection chip provides clear visual feedback -- Subgraph extraction enables scalable context for large workflows (50+ nodes) -- Error handling prevents silent failures from oversized payloads - -**Ready for next milestone** - Phase 35 completes Milestone v1.4 (Agentic Workflow Builder). Ready to proceed to Milestone v1.5 (Store Refactoring) or address any new priorities.