Use MediaBunny as the primary decoder following the documented pattern:
getPrimaryAudioTrack() + canDecode() check + AudioBufferSink.buffers().
Falls back to Web Audio API's decodeAudioData if MediaBunny cannot parse
the format (e.g. unusual codecs). This ensures MediaBunny handles
container formats it supports while Web Audio API catches the rest.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
MediaBunny's Input/BlobSource is designed for video containers and cannot
parse standalone audio files (MP3, WAV, etc.), causing "unsupported format"
errors at getAudioTracks(). Replace with Web Audio API's decodeAudioData
which natively handles all browser-supported audio formats.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add "video" to Output node's getNodeHandles inputs list
- Consistent with isValidConnection and findCompatibleHandle video->output handling
- Output node now properly accepts VideoStitch connections via all code paths
- Add video handle mapping for videoStitch, generateVideo, and output
- Add audio handle mapping for audioInput and videoStitch
- Connection drop now auto-connects for all handle types
- Extract MIME type from data URL header when fetch() Blob loses it
- Create new Blob with correct type for MediaBunny format detection
- Fix both executeWorkflow and regenerateNode videoStitch paths
- Detect audio file types in handleDragOver (audio/*)
- Handle audio files in handleDrop creating AudioInput nodes
- Show "Drop audio to create node" overlay for audio drag
- Add "audio" to dropType state union
Ensure handles render in all node states (disabled/checking/normal) by
extracting a shared renderHandles function. Move clipOrder syncing from
useMemo side-effect into a proper useEffect. Add dynamic video handle
resolution in WorkflowCanvas for indexed handles. Add VideoStitch
execution case in regenerateNode.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
WorkflowCanvas.tsx:
- Import VideoStitchNode
- Add videoStitch to nodeTypes
- Add getNodeHandles case: inputs=["video", "audio"], outputs=["video"]
- Update isValidConnection to allow video→videoStitch connections
- Add minimap color: orange-500 (#f97316)
ConnectionDropMenu.tsx:
- Add videoStitch to VIDEO_TARGET_OPTIONS (first position)
- Add videoStitch to VIDEO_SOURCE_OPTIONS (produces video output)
- Add videoStitch to AUDIO_TARGET_OPTIONS (accepts audio input)
workflowStore.ts:
- Import VideoStitchNodeData type
- Add videoStitch to getSourceOutput (returns video type)
- Add executeWorkflow case for videoStitch with full stitching logic:
* Check encoder support
* Get connected inputs (videos, audio)
* Convert video data URLs to Blobs
* Prepare audio if connected (using prepareAudioAsync)
* Call stitchVideosAsync with progress callback
* Store output (blob URL for >20MB, data URL otherwise)
* Handle errors
Note: nodeDefaults.ts already had videoStitch dimensions and
createDefaultNodeData from Plan 02, so no changes needed there.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Component with encoder detection on mount (checkEncoderSupport)
- Dynamic video input handles (video-0, video-1, etc.) based on connections
- Audio input handle at bottom (violet color, matching AudioInput)
- Video output handle
- Filmstrip UI with thumbnail extraction (seek to 25% duration)
- Clip ordering via clipOrder array with drag-to-reorder support
- X button per clip to disconnect
- Processing trigger button (disabled until 2+ clips connected)
- Progress overlay during stitching (shows percentage)
- Output video preview on completion
- Disabled state when encoder not supported
- Export added to nodes/index.ts
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Added AudioInputNode export to nodes/index.ts
- Added audioInput to defaultNodeDimensions (300x200) and createDefaultNodeData
- Registered audioInput in WorkflowCanvas nodeTypes map
- Updated getHandleType to recognize audio handles
- Added audioInput to getNodeHandles (no inputs, audio output)
- Updated isValidConnection for audio-to-audio validation
- Added audio to ConnectionDropState handleType union
- Updated findCompatibleHandle to accept audio type
- Added AUDIO_SOURCE_OPTIONS and AUDIO_TARGET_OPTIONS to ConnectionDropMenu
- Added audio array to getConnectedInputs return type
- Updated getSourceOutput to handle audioInput nodes
- Added audioInput execution case (no-op, data source)
- Added AudioInputNodeData import to workflowStore
- Updated DEFAULT_DIMENSIONS in WorkflowCanvas and quickstart/validation.ts
- Updated createDefaultNodeData in quickstart/validation.ts
- Added violet minimap color (#a78bfa) for audioInput nodes
- File upload via drag-and-drop or click
- Waveform visualization using useAudioVisualization hook
- Play/pause controls with scrub bar
- Canvas-based rendering with ResizeObserver for responsive sizing
- Audio handle (purple/violet) on right side
- Min dimensions: 250px width, 150px height
- Remove button to clear audio file
- Port useAudioVisualization.ts with waveform peak extraction
- Port useAudioMixing.ts with prepareAudioAsync standalone function
- Port useStitchVideos.ts with stitchVideosAsync and checkEncoderSupport
- Export applyFades from useAudioMixing for shared use
- Remove preview mode logic from useStitchVideos (always full quality)
- Inline constants (DEFAULT_BITRATE, MAX_OUTPUT_FPS) in useStitchVideos
- All hooks use MediaBunny for audio/video processing
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Install mediabunny@^1.28.0 for video processing
- Add audioInput and videoStitch to NodeType union
- Add AudioInputNodeData interface (audioFile, filename, duration, format)
- Add VideoStitchClip interface for filmstrip items
- Add VideoStitchNodeData interface (clips, clipOrder, outputVideo, progress, encoderSupported)
- Update WorkflowNodeData union to include new node types
- Extend HandleType to include audio and video handles
- Create src/lib/video-encoding.ts with AVC encoding config helpers
These files are already in .gitignore but were committed
before the rule was added. Kept locally, removed from repo.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add onKeyDown handler to PromptNode variable name input so Enter saves.
Add outputGallery, imageCompare, and promptConstructor to ConnectionDropMenu
handle mappings for auto-connect on drag-drop. For imageCompare, redirect
connections to the second handle (image-1) when the first is occupied.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Extract autocomplete logic to usePromptAutocomplete hook
- Add PromptConstructorEditorModal with portal rendering
- Sync PromptConstructor outputText reactively for downstream nodes
- Add headerButtons prop to BaseNode for custom header actions
- Rename Blank Canvas to New Project with ProjectSetupModal integration
- Auto-push generated images to connected OutputGallery nodes
- Deduplicate images in OutputGallery during workflow execution
- Move PromptNode variable button to header with hover animation
- Extract AvailableVariable interface to types/nodes.ts
- Update tests for onNewProject rename
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Test buildEditSystemPrompt basic structure (all major sections present)
- Test with workflow context including node listing and connection formatting
- Test scoped selection with restSummary (node count, type breakdown, boundary connections)
- Test empty restSummary does not show SELECTED SUBSET section
- Test createChatTools returns 3 properly structured tools
- Test execute behavior for answerQuestion, createWorkflow, and editWorkflow
- 21 test cases covering prompt building and tool creation
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Test all 5 operation types: addNode, removeNode, updateNode, addEdge, removeEdge
- Test skip scenarios for missing nodes/edges with descriptive reasons
- Test batch operations with mixed applied/skipped counts
- Test narrateOperations formatting for all operation types
- 28 test cases covering happy paths and edge cases
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Two labeled image input handles: 'A' (top 35%) and 'B' (bottom 65%)
- Real-time image collection from connected nodes (same pattern as OutputGalleryNode)
- ReactCompareSlider with horizontal-only slider (portrait=false)
- Corner labels A and B on comparison view
- Placeholder states for 0 or 1 images with connection status feedback
- nodrag nopan nowheel classes for interaction isolation
- BaseNode wrapper with customizable title and comments
- Build verified: zero TypeScript errors
- Install react-compare-slider dependency
- Add imageCompare to NodeType union
- Add ImageCompareNodeData interface (imageA, imageB fields)
- Add to WorkflowNodeData union type
- Register in nodeDefaults with 400x360 dimensions
- Add to WorkflowCanvas nodeTypes and minimap (teal color)
- Add to ConnectionDropMenu IMAGE_TARGET_OPTIONS
- Add executeWorkflow case to assign images[0] to imageA, images[1] to imageB
- Export ImageCompareNode from components/nodes/index
- Add to validation.ts DEFAULT_DIMENSIONS and createDefaultNodeData
- Template textarea with @ autocomplete dropdown
- Detects @ typing and shows matching connected variable names
- Autocomplete dropdown shows @name and prompt value preview
- Arrow keys navigate, Enter/Tab select, Escape closes
- Client-side computation of unresolvedVars from @(\w+) patterns
- Warning badge displays unresolved @tags in amber
- Live preview tooltip on hover showing resolved template
- availableVariables computed from connected prompt nodes with variableName
- Text input handle (target, left) and text output handle (source, right)
- BaseNode wrapper with title 'Prompt Constructor'
- Sync template to store on blur (same pattern as PromptNode)
- Local state prevents cursor jumping during typing
- Filtered autocomplete based on typed characters after @
- Position autocomplete relative to textarea with line height calculation
- Add variableName field to PromptNodeData (optional, backward compatible)
- Add promptConstructor to NodeType union
- Add PromptConstructorNodeData interface (template, outputText, unresolvedVars)
- Add to WorkflowNodeData union type
- Add promptConstructor to defaultNodeDimensions (340x280)
- Add createDefaultNodeData case for promptConstructor
- Modified PromptNode with @ variable naming dialog using createPortal
- Variable name validation: alphanumeric + underscore, max 30 chars
- Blue @ icon when variableName set, dimmed when not set
- Dialog shows variable name input, preview, Save/Clear buttons
- Register PromptConstructorNode in WorkflowCanvas nodeTypes
- Add promptConstructor minimap color (#f472b6 pink)
- Add to TEXT_TARGET_OPTIONS and TEXT_SOURCE_OPTIONS in ConnectionDropMenu
- Add getConnectedInputs support for promptConstructor outputText
- Add executeWorkflow case for promptConstructor:
- Finds connected prompt nodes via text edges
- Builds variable map from variableName + prompt values
- Resolves template by replacing @variableName with values
- Tracks unresolved variables in unresolvedVars array
- Stores outputText and unresolvedVars on node data
- Export PromptConstructorNode from components/nodes/index.ts
- Update duplicate defaultDimensions in WorkflowCanvas and quickstart validation
- Create OutputGalleryNode.tsx with BaseNode wrapper
- Single image input handle (type target, id image)
- Real-time image collection from connected nodes (imageInput, annotation, nanoBanana)
- 3-column scrollable thumbnail grid with nowheel for scroll isolation
- Full-screen lightbox via createPortal with:
- Left/right arrow navigation
- Download button (saves as gallery-image-N.png)
- Close button (X icon, top-right)
- Click backdrop to close
- Keyboard support (Escape, ArrowLeft, ArrowRight)
- Image counter display (N / Total)
- Empty state placeholder when no images connected
- Fix validation.ts: add outputGallery dimensions and createDefaultNodeData case
- Build passes with zero errors
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add outputGallery to NodeType union in types/nodes.ts
- Add OutputGalleryNodeData interface with images array field
- Add outputGallery to WorkflowNodeData union
- Register in nodeDefaults: dimensions 320x360, default data with empty images array
- Export OutputGalleryNode from nodes/index.ts
- Register outputGallery in WorkflowCanvas nodeTypes with pink minimap color (#ec4899)
- Add outputGallery to ConnectionDropMenu IMAGE_TARGET_OPTIONS with grid icon
- Add outputGallery case to executeWorkflow in workflowStore
- Add outputGallery to getNodeHandles (image input, no outputs)
- Add outputGallery dimensions to keyboard shortcut default dimensions
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add createdAt timestamp to WorkflowEdgeData interface
- Store createdAt on edge creation in onConnect and addEdgeWithType
- Add getImageSequenceNumber helper to EdgeToolbar
- Display "Image N" label for image connections with 2+ edges to same target
- Sequence numbers based on createdAt for stable ordering
- Single connections and text connections show no numbering
Resolve package.json conflict: take master's react-dom ^19.2.4,
keep develop's AI SDK and react-markdown additions.
Regenerate package-lock.json.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Import extractSubgraph from subgraphExtractor
- Parse selectedNodeIds from request body
- Call extractSubgraph before building workflow context
- Pass subgraph.selectedNodes and selectedEdges to buildWorkflowContext
- Update buildEditSystemPrompt to accept optional restSummary parameter
- Add WORKFLOW CONTEXT (SELECTED SUBSET) section when scoped
- Include rest-of-workflow summary with type breakdown and boundary connections
- Add metadata placeholder note to system prompt
- Enhance error handling to return 413 for oversized payloads
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Import stripBinaryData from contextBuilder
- Rewrite chatWorkflowState memo to use stripBinaryData for rich but compact node data
- Compute selectedNodeIds from nodes.filter(n => n.selected)
- Pass selectedNodeIds as new prop to ChatPanel
- Update ChatPanel to accept and forward selectedNodeIds in API requests
- Add dismissible "Focused on N selected nodes" chip above input when nodes selected
- Add error handling for oversized payloads with user-friendly message
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add 22 tests covering:
- Binary field stripping across all 8 node types
- Metadata placeholder generation with size/dimensions
- History and ref field removal
- Parameter preservation
- Position inclusion
- Enhanced buildWorkflowContext with full node data
- formatContextForPrompt rendering
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Implement extractSubgraph function with full selection logic
- Early return for no selection (return all nodes/edges, isScoped=false)
- Filter nodes into selected vs unselected with O(1) Set lookup
- Classify edges: fully within selection, boundary, or outside
- Build type breakdown from unselected nodes
- Identify boundary connections with direction (incoming/outgoing)
- All 8 tests passing (GREEN phase)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Create comprehensive test suite for extractSubgraph function
- Test cases cover: no selection, select all, single node, connected pairs,
subset selection, type breakdown, disconnected nodes, multi-handle scenarios
- All 8 tests failing with "Not implemented" error (RED phase)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add execute functions to all three chat tools (pass-through pattern)
so AI SDK produces proper tool-call/tool-result pairs in conversation
history, fixing AI_MissingToolResultsError on subsequent messages
- Fix stale closure in ChatPanel customFetch by using a ref for
workflowState, ensuring current canvas state is sent with each request
- Add EDITABLE NODE PROPERTIES section to system prompt so LLM uses
correct data property names (e.g. "prompt" not "text" for prompt nodes)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add applyEditOperations to workflowStore for AI-driven workflow modifications.
Wire ChatPanel to process editWorkflow tool calls and apply operations via store.
Pass lightweight workflow state to chat API for context-aware assistance.
Changes:
- workflowStore: add applyEditOperations method and EditOperation import
- ChatPanel: add onApplyEdits prop, workflowState prop, custom fetch wrapper
- ChatPanel: process tool-invocation parts for editWorkflow and createWorkflow
- ChatPanel: render tool explanations in chat UI
- WorkflowCanvas: extract chatWorkflowState, create handleApplyEdits callback
- WorkflowCanvas: pass onApplyEdits and workflowState to ChatPanel
AI edits trigger captureSnapshot before applying (revert button support).
AI edits do NOT increment manualChangeCount (only manual edits clear snapshot).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Create SUMMARY.md documenting plan execution and update STATE.md
with current position and accumulated decisions.
Summary includes:
- Task completion: enhanced chat API route with tool calling
- Deviation: changed maxSteps to stopWhen: stepCountIs(3) (AI SDK v6 API)
- Key decisions: tool pattern, context injection, multi-step reasoning
- Verification: TypeScript passes, streaming works correctly
STATE.md updates:
- Current position: Plan 2 of phase 34
- Performance metrics: 2/? plans, 13 min total, 6.5 min avg
- Recent trend: 5-8 min execution
- Accumulated context: 4 new decisions about chat API and tool calling
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Broaden the trailing-slash exemption to recognize both C:\ and C:/ as
drive roots. Tighten the client-side absolute path check to require a
separator after the drive letter and accept UNC paths (\\server\share).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Rewrite /api/chat route to accept workflow context from client and use
AI SDK tool calling for multi-modal agent routing.
Changes:
- Accept optional workflowState (nodes/edges) in request body
- Build workflow context using buildWorkflowContext from contextBuilder
- Generate context-aware system prompt using buildEditSystemPrompt
- Create chat tools (answerQuestion, createWorkflow, editWorkflow) with
current node IDs for validation
- Configure streamText with tools, toolChoice: 'auto', and stopWhen for
multi-step reasoning (up to 3 steps)
- Stream results back via toUIMessageStreamResponse for useChat hook
compatibility
LLM now automatically routes to the correct tool based on user intent.
Tool results stream back to client for application in Plan 03.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Create tools.ts with buildEditSystemPrompt function that constructs enhanced system prompt with workflow context and tool usage rules. Implement createChatTools function that returns three tool definitions using AI SDK v6 tool() pattern:
- answerQuestion: informational queries without workflow modification
- createWorkflow: build new workflows from scratch via quickstart
- editWorkflow: targeted edits with operation list and explanation
All tools use zod schemas for type-safe input validation. Tools follow "generate" pattern (no execute function) - LLM provides structured output that client/route handles.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Create editOperations.ts with EditOperation type union covering all 5 operation types (addNode, removeNode, updateNode, addEdge, removeEdge). Implement applyEditOperations function with batched immutable updates, proper ID generation, and validation with skip tracking. Add narrateOperations function for human-readable summaries.
Create contextBuilder.ts with WorkflowContext interface and buildWorkflowContext function that strips base64 data, history arrays, and internal state. Add formatContextForPrompt function for system prompt injection.
Both modules are pure TypeScript with no UI or API dependencies.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
On macOS, osascript's folder picker can return hostname-prefixed paths
(e.g. "HOSTNAME/Users/...") for network-mounted volumes instead of
absolute POSIX paths, causing directory validation to fail.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Use neutral-300 text, neutral-700/50 bg, and neutral-600 border
to make the button noticeable without clashing with the app style.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
shrimbly