Local reference images were reaching the gateway as localhost URLs, and selected image/video nodes still exposed duplicate inline settings while the composer is the active control surface. This keeps gateway media inputs self-contained, maps GPT Image 2 sizes from the actual aspect ratio controls, and keeps node bodies focused on preview/output state.
Constraint: NewApiWG upstream cannot fetch browser-local /api/images URLs.
Constraint: Selected-node composer owns image and video generation controls.
Rejected: Keep localhost reference URLs | remote gateway requests cannot dereference local browser routes.
Rejected: Keep duplicate inline settings | it creates conflicting controls with the composer.
Confidence: high
Scope-risk: moderate
Directive: Do not send /api/images temporary URLs to NewApiWG upstream; resolve them to data URLs before provider calls.
Tested: npm run test:run -- src/app/api/generate/providers/__tests__/newapiwg.test.ts src/app/api/generate/__tests__/route.test.ts src/components/__tests__/GenerateImageNode.test.tsx src/components/__tests__/GenerateVideoNode.test.tsx src/components/__tests__/PromptNode.test.tsx src/lib/__tests__/llmModels.test.ts src/store/__tests__/workflowStore.integration.test.ts
Tested: npm run build
Tested: git diff --check
Not-tested: npm run lint still fails because next lint resolves to a nonexistent lint directory under Next 16.
Popi/NewAPIWG is now the primary gateway, so the UI and request paths need to avoid legacy provider assumptions while keeping local reference media usable for upstream calls. This change aligns model defaults, point display, inline node controls, and temporary reference image handling around that gateway boundary.
Constraint: Upstream providers cannot fetch localhost reference URLs, but large base64 payloads previously caused request/body reliability issues.
Constraint: Seedance point estimates depend on authenticated NewAPIWG /api/points/estimate responses rather than static catalog prices.
Rejected: Send all reference images directly as large data URLs | would reintroduce oversized JSON/request failures.
Rejected: Restore legacy per-provider key controls by default | current product direction is a single Popi/NewAPI gateway.
Confidence: high
Scope-risk: moderate
Directive: Do not pass /api/images localhost URLs to upstream generation APIs; resolve them server-side or fail before the upstream call.
Tested: npm run test:run -- src/app/api/generate/providers/__tests__/newapiwg.test.ts src/app/api/points/estimate/__tests__/route.test.ts src/lib/images/store.test.ts src/components/__tests__/GenerationComposer.test.tsx src/components/__tests__/ImageInputNode.test.tsx src/components/__tests__/PromptNode.test.tsx src/components/__tests__/ProjectSetupModal.test.tsx src/store/execution/__tests__/generateVideoExecutor.test.ts src/store/execution/__tests__/nanoBananaExecutor.test.ts src/store/utils/__tests__/nodeDefaults.test.ts
Tested: npm run build
Tested: git diff --check
Not-tested: npm run lint fails because next lint is parsed as a missing /lint project directory under Next 16.
Multi-image NewAPIWG generations were sending base64 reference images
inside the /api/generate JSON body. Two ordinary canvas references can
push that payload near the dev/server body boundary, leaving the route
to parse an incomplete JSON string and surface a raw syntax error.
Move large NewAPIWG references through the existing in-memory image
serving path: the browser uploads large data URLs as multipart image
files, sends short /api/images/{id} URLs in the generation request, and
the route resolves those temporary URLs back to server-local image data
before calling providers. The route also turns malformed/truncated JSON
into a readable validation error.
Constraint: Keep provider payload semantics unchanged after /api/generate receives the request
Constraint: Do not rely on request origin matching because Next dev can report 0.0.0.0 while the browser uses localhost
Rejected: Increase body size limits only | still sends multi-MB JSON and does not protect hosted/proxied environments
Rejected: Compress reference images client-side | changes model input fidelity and still leaves large JSON possible
Confidence: high
Scope-risk: moderate
Directive: Do not forward /api/images/{id} temporary URLs to external providers; resolve them server-side first
Tested: npm run test:run -- src/store/execution/__tests__/nanoBananaExecutor.test.ts src/app/api/generate/__tests__/route.test.ts
Tested: npm run build
Not-tested: Live NewAPIWG generation with the user's exact two images
This keeps the testing branch aligned with the current Popi.TV interaction direction: generation is controlled from the selected node context, model/provider selection is simplified around the NewAPI gateway path, Popiai embedding can hydrate workflow state, and media nodes expose names plus dimensions in the canvas header.
Constraint: Testing branch needs the full current source state, not runtime logs or local browser artifacts
Rejected: Commit logs, browserfs state, and handoff bundles | they are local runtime artifacts rather than application source
Confidence: high
Scope-risk: moderate
Reversibility: clean
Directive: Keep model selection consolidated around the NewAPI gateway unless supplier-specific behavior is intentionally reintroduced
Tested: npm run test:run -- src/components/__tests__/FloatingNodeHeader.test.tsx src/components/__tests__/nodeHeaderMedia.test.ts src/components/__tests__/WorkflowCanvas.test.tsx
Tested: npm run test:run -- src/components/__tests__/ImageInputNode.test.tsx src/components/__tests__/GenerateVideoNode.test.tsx src/components/__tests__/GenerateImageNode.test.tsx
Tested: npm run build
Tested: git diff --check -- src
Not-tested: npm run lint | existing script uses next lint and resolves to nonexistent ./lint directory
The accumulated fixes move the test build toward the current product shape: Popi models are the default provider surface, remote community workflow APIs are removed from the local template flow, media nodes expose accurate preview/preview-modal behavior, and Generate Video owns first-class duration, resolution, and sound defaults instead of burying those values in provider schemas.
The test suite was brought back in line with the product mode split: Popi provider mode is the default runtime, while multi-provider model search remains covered explicitly under multi mode.
Constraint: Product direction is canvas-owned workflow structure with Popi Models as the tested default provider path.
Rejected: Keep legacy community workflow API surface | user requested removing remote data paths and retaining local templates.
Rejected: Treat video duration/resolution/sound as hidden provider params | these are user-facing Generate Video controls.
Confidence: high
Scope-risk: broad
Directive: Do not reintroduce remote community template loading without a product decision on hosted templates and i18n ownership.
Tested: npm test -- --run src/store/utils/__tests__/nodeDefaults.test.ts src/store/execution/__tests__/generateVideoExecutor.test.ts src/components/__tests__/GenerateVideoNode.test.tsx src/components/__tests__/GenerationComposer.test.tsx
Tested: npm run build
Tested: npm test -- --run src/components/__tests__/ModelSearchDialog.test.tsx src/store/utils/__tests__/localStorage.test.ts
Tested: npm run test:run
Not-tested: Live provider generation calls against paid external APIs.
New users now land on Popi gateway defaults that match the tested production-style path: APIYI Nano Banana 2 for images, Doubao Seedance for video, and Doubao Seed 2.0 Lite for LLM chat/tool use. Settings and FTUX screens render these effective defaults instead of empty placeholders, while existing local node defaults keep taking precedence.
The same pass keeps provider-overload errors user-facing by formatting the model name that the node currently displays, so fallback and switched-model failures are easier to understand across media nodes.
Constraint: Popi mode should hide internal NewApiWG naming and default to models available through the Popi gateway.
Rejected: Persist default values into localStorage for every new user | would make future system default changes harder and could overwrite explicit user preferences.
Confidence: high
Scope-risk: moderate
Directive: Do not change these defaults without checking the Popi gateway model ids and www.popi.art login/model-routing assumptions.
Tested: npm run test:run -- src/lib/__tests__/llmModels.test.ts src/store/utils/__tests__/nodeDefaults.test.ts src/store/execution/__tests__/generateVideoExecutor.test.ts src/components/__tests__/ProjectSetupModal.test.tsx src/components/__tests__/GenerationComposer.test.tsx src/app/api/chat/__tests__/route.test.ts src/components/__tests__/LLMGenerateNode.test.tsx src/utils/__tests__/userFacingErrors.test.ts
Tested: PROVIDER_MODE=popi NEXT_PUBLIC_PROVIDER_MODE=popi npm run build
Tested: git diff --check
Not-tested: npm run lint currently fails because next lint is no longer a valid project command in this Next.js setup
The right-side assistant now uses the existing chat/tooling path as a docked canvas surface instead of a hidden floating experiment. It can prepare workflow edits, prepare run actions, and keep all mutations behind an explicit preview/apply step.
A shared chat node schema keeps the model prompt and local edit validator aligned with real node types, while the context builder strips broader media payloads before chat requests.
Constraint: Keep the deployed master/TEST-s version stable while this work proceeds on a feature branch
Constraint: No new dependencies and no broad canvas rewrite
Rejected: Directly applying model-suggested edits | local validation and preview remain the safety boundary
Rejected: Full multimodal image upload in the first pass | current implementation only exposes media metadata and failure state
Confidence: high
Scope-risk: moderate
Reversibility: clean
Directive: Add new AI-editable node fields through src/lib/chat/nodeSchema.ts before exposing them in prompts
Tested: npm run test:run -- src/lib/chat/contextBuilder.test.ts src/lib/chat/tools.test.ts src/lib/chat/editOperations.test.ts src/components/__tests__/ChatPanel.test.tsx src/components/__tests__/WorkflowCanvas.test.tsx src/app/api/chat/__tests__/route.test.ts
Tested: PROVIDER_MODE=popi NEXT_PUBLIC_PROVIDER_MODE=popi npm run build
Tested: Browser smoke on http://localhost:3000 verified docked assistant, close/reopen, selected-node focus, and run controls
Not-tested: Live LLM tool-call execution against the gateway
Default provider mode now exposes only Popi Models while preserving the existing multi-provider implementation behind an explicit multi mode. The model browser, project settings, onboarding, node badges, and model API all share the same provider-mode decision so the test deployment matches the www.popi.art login-aligned direction.
This also includes the pending UI and i18n node-label changes already present in the tracked worktree so the committed state matches the verified build.
Constraint: Product entry must align with www.popi.art by exposing Popi Models instead of internal NewApiWG naming
Constraint: Test deployment should match the full tracked working tree that passed build
Rejected: Delete legacy provider code | multi-provider internals remain useful behind PROVIDER_MODE=multi
Rejected: UI-only hiding | API and store layers now enforce Popi mode too
Confidence: high
Scope-risk: broad
Reversibility: clean
Directive: Keep NewApiWG as the internal provider id unless the gateway API/env contract is migrated
Tested: npm run test:run -- src/components/__tests__/ProjectSetupModal.test.tsx src/app/api/models/__tests__/route.test.ts src/lib/__tests__/llmModels.test.ts
Tested: npm run test:run -- src/components/__tests__/GenerateImageNode.test.tsx src/components/__tests__/GenerateVideoNode.test.tsx src/components/__tests__/GenerateAudioNode.test.tsx src/components/__tests__/Generate3DNode.test.tsx src/components/__tests__/LLMGenerateNode.test.tsx
Tested: npm run test:run -- src/lib/chat/tools.test.ts
Tested: PROVIDER_MODE=popi NEXT_PUBLIC_PROVIDER_MODE=popi npm run build
Tested: Browser smoke on http://localhost:3000 verified Project Settings and Model Search only expose Popi Models
Not-tested: Direct npx tsc --noEmit remains blocked by pre-existing test typing errors outside this change
The LLM selector exposed display-style Doubao model names that the NewApiWG test gateway cannot route. The UI now stores the gateway model ids returned by /models, while the API route maps legacy saved selector values to those ids so existing workflows do not keep failing.
Constraint: Preserve GPT-4.1 Mini as the NewApiWG default and keep old workflow JSON compatible.
Rejected: Remove Doubao from the LLM selector | the gateway exposes working Doubao chat model ids, so the issue is stale ids rather than product support.
Confidence: high
Scope-risk: narrow
Directive: LLM selector values must be callable provider ids; use labels only for display text.
Tested: Live NewApiWG chat completion with doubao-seed-2-0-lite-260428 and doubao-seed-2-0-pro-260215.
Tested: npm run test:run -- src/lib/__tests__/llmModels.test.ts src/app/api/llm/__tests__/route.test.ts src/components/__tests__/LLMGenerateNode.test.tsx src/store/utils/__tests__/nodeDefaults.test.ts
Tested: npm run build
Tested: git diff --check
P0 regressions showed several workflow paths needed to stay visible and deterministic: language switching had a hardcoded composer label, Video Trim hid controls before media loaded, and media handoffs to LLM needed real canvas edges rather than implicit conversion. P1 fixes keep canvas-owned structure by adding explicit image/video LLM handles, audio composer validation, 3D output/file routing, and Chat proposal cards that require user apply before mutating nodes.
Constraint: User asked to stop new feature work and only repair P0/P1 product-fit gaps.
Rejected: Auto-convert image or video directly into text for LLM | hides workflow structure and violates the visible-node product model.
Rejected: Let Chat tool calls silently add or edit nodes | canvas structure must be previewed and explicitly applied.
Confidence: high
Scope-risk: moderate
Directive: Keep media transformations represented by visible handles, nodes, and edges; do not add silent Chat or media mutation paths without product review.
Tested: Browser automation for language switch, video input to Generate Video, prompt to Generate Video, LLM default model, Video Trim controls, and video media to LLM input.
Tested: npm run test:run -- src/components/__tests__/ConnectionDropMenu.test.tsx src/components/__tests__/GenerationComposer.test.tsx src/components/__tests__/VideoTrimNode.test.tsx src/components/__tests__/LLMGenerateNode.test.tsx src/store/execution/__tests__/llmGenerateExecutor.test.ts src/app/api/llm/__tests__/route.test.ts src/store/execution/__tests__/generateAudioExecutor.test.ts src/store/execution/__tests__/generate3dExecutor.test.ts src/store/execution/__tests__/simpleNodeExecutors.test.ts src/components/__tests__/OutputNode.test.tsx src/components/__tests__/Generate3DNode.test.tsx src/components/__tests__/ChatPanel.test.tsx src/lib/chat/tools.test.ts src/lib/chat/editOperations.test.ts src/lib/__tests__/llmModels.test.ts src/store/utils/__tests__/nodeDefaults.test.ts
Tested: npm run build
Tested: git diff --cached --check
Not-tested: npm run lint, because the existing script calls removed Next.js lint command and exits with invalid project directory /lint.
Not-tested: Live paid provider calls for LLM, audio, and 3D generation.
P0/P1 reports clustered around unavailable LLM defaults, video media connections, and missing controls on the generation surface. This routes the default NewApiWG LLM away from the unavailable Doubao channel, makes video a first-class schema and canvas connection type, carries request ids into node errors, and fills the composer, trim, audio, and 3D gaps without changing the canvas workflow model.
Constraint: NewApiWG Doubao channels are not reliably routed in the default gateway
Constraint: Video model schemas expose video fields that must not flow through image handles
Rejected: Continue accepting video through image handles | hides incorrect graph types and loses schema-specific dynamic inputs
Confidence: high
Scope-risk: moderate
Directive: Keep media handle types explicit; add schema mapping when adding a new connectable media type
Tested: npm run test:run -- src/store/execution/__tests__/generateVideoExecutor.test.ts src/store/utils/__tests__/connectedInputs.test.ts src/store/execution/__tests__/generateAudioExecutor.test.ts src/store/execution/__tests__/llmGenerateExecutor.test.ts src/store/__tests__/workflowStore.integration.test.ts src/lib/__tests__/llmModels.test.ts src/store/utils/__tests__/nodeDefaults.test.ts src/components/__tests__/GenerationComposer.test.tsx src/app/api/chat/__tests__/route.test.ts src/app/api/llm/__tests__/route.test.ts src/app/api/models/[modelId]/__tests__/route.test.ts src/app/api/generate/providers/__tests__/newapiwg.test.ts
Tested: npm run build
Tested: git diff --check
Not-tested: npm run lint | package script calls removed Next 16 next lint command
Not-tested: npx tsc --noEmit | existing test type errors remain; changed runtime files filtered clean
This aligns image, video, and text defaults with the PopiArt NewApi gateway while keeping provider-specific escape hatches available. Chat now uses the OpenAI-compatible NewApiWG route so workflow editing and LLM generation share the same Doubao-first text default. The canvas composer and quick-add controls are tightened so visible actions map to real clickable behavior, while temporarily unused header actions stay hidden.
Constraint: Text/chat requests must prefer Doubao-Seed-2.0-lite through NewApiWG.
Constraint: Image and video defaults should prefer PopiArt NewApi gateway models when configured.
Rejected: Keep /api/chat on Gemini 3 Flash | conflicts with the requested NewApiWG text/chat default.
Rejected: Hand-roll AI SDK chat streaming | would risk tool-call and UI stream compatibility.
Confidence: high
Scope-risk: moderate
Directive: Do not add new text/chat defaults without updating src/lib/llmModels.ts and the chat route together.
Tested: npm test -- --run src/app/api/chat/__tests__/route.test.ts src/components/__tests__/LLMGenerateNode.test.tsx src/lib/__tests__/llmModels.test.ts src/store/utils/__tests__/nodeDefaults.test.ts src/components/__tests__/GenerationComposer.test.tsx src/lib/chat/tools.test.ts
Tested: npm test -- --run src/lib/__tests__/llmModels.test.ts src/store/utils/__tests__/nodeDefaults.test.ts src/components/__tests__/GenerationComposer.test.tsx src/lib/quickstart/__tests__/templates.test.ts src/lib/quickstart/__tests__/validation.test.ts src/lib/quickstart/__tests__/prompts.test.ts src/components/__tests__/ProjectSetupModal.test.tsx
Tested: npm test -- --run src/components/__tests__/WorkflowCanvas.test.tsx
Not-tested: Full suite; repository has pre-existing unrelated type/test warnings outside this change.
Add "audio-to-video" to ModelCapability union, extend ModelInput.type
and ModelInputDef.type with "audio", and include "audio-to-video" in
fal.ai RELEVANT_CATEGORIES for model discovery.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds a new videoInput node type that allows users to drag-and-drop or
browse for video files (MP4, WebM, QuickTime), preview them inline with
native video controls, and feed them into downstream video processing
nodes. Includes full media externalization/hydration support, Shift+Y
keyboard shortcut, and ConnectionDropMenu integration.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The customTitle and comment fields are typed as string | undefined, not
string | null. Remove them from the quickstart default data to fix the
TypeScript build error.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The createDefaultNodeData function in validation.ts had cases for router
and switch but was missing conditionalSwitch, despite it being listed in
VALID_NODE_TYPES and DEFAULT_DIMENSIONS.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Nano Banana 2 (gemini-3.1-flash-image) supports a new searchTypes API
that allows enabling web search and image search independently. This adds
a useImageSearch toggle visible only for Nano Banana 2, while keeping the
existing Google Search toggle using the legacy format for Nano Banana Pro.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove router/switch/conditionalSwitch entries from WorkflowCanvas
defaultDimensions and validation.ts (these types only exist on develop)
- Add ModelType import and cast in nodeDefaults.ts for type safety
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Pre-existing type error — conditionalSwitch was missing from the
createDefaultNodeData switch in validation.ts.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add MatchMode type (exact, contains, starts-with, ends-with)
- Add ConditionalSwitchRule interface with id, value, mode, label, isMatched
- Add ConditionalSwitchNodeData interface with incomingText and rules array
- Add conditionalSwitch to NodeType union (after switch)
- Add ConditionalSwitchNodeData to WorkflowNodeData union
- Add default data with single rule (mode: contains, empty value)
- Add default dimensions (260x180, wider than Switch for mode dropdown)
- Add executeConditionalSwitch passthrough executor
- Add conditionalSwitch to VALID_NODE_TYPES in validation
- Export ConditionalSwitchNode from components index (component created in next task)
- Add "switch" to NodeType union in types/nodes.ts
- Create SwitchNodeData interface with inputType and switches array
- Add SwitchNodeData to WorkflowNodeData union
- Register switch dimensions (220x120) in nodeDefaults.ts and validation.ts
- Add createDefaultNodeData case with single default switch
- Implement executeSwitch passthrough executor in simpleNodeExecutors.ts
- Add SwitchNode export to components/nodes/index.ts
- Add switch dimensions to WorkflowCanvas.tsx default dimensions map
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add 'router' to NodeType union in types/nodes.ts
- Add RouterNodeData interface (extends BaseNodeData, no internal state)
- Add RouterNodeData to WorkflowNodeData union
- Add router dimensions (200x80) to nodeDefaults.ts and validation.ts
- Add router default data (empty object) to both files
- Add executeRouter pure passthrough function to simpleNodeExecutors.ts
- Add RouterNode export to components/nodes/index.ts
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
VALID_NODE_TYPES was missing audioInput, promptConstructor, outputGallery,
imageCompare, videoStitch, easeCurve, videoTrim, videoFrameGrab, and glbViewer —
all of which are defined in NodeType and have entries in DEFAULT_DIMENSIONS and
createDefaultNodeData. VALID_HANDLE_TYPES was missing video, easeCurve, and 3d
handle types used by EaseCurveNode, GenerateVideoNode, Generate3DNode, GLBViewerNode,
VideoStitchNode, VideoTrimNode, and VideoFrameGrabNode.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Extract shared video-probing module (constants, helpers, probeVideoMetadata)
from useTrimVideo and useStitchVideos into src/lib/video-probing.ts
- Fix audio interleaving in useTrimVideo to feed audio before video frames,
matching useStitchVideos fix for Discord playback compatibility
- Fix blob URL leaks in VideoTrimNode and videoProcessingExecutors
- Fix nanoBananaExecutor: null coercion for non-string prompts, redact PII
in console.warn, remove duplicate dynamicInputs.prompt
- Fix generateAudioExecutor double updateNodeData on error paths
- Fix fal.ts media fetch ordering for 3D model detection
- Fix kie.ts and replicate.ts content-type detection for audio models
- Fix WorkflowCanvas connection drop menu wiring for generateAudio nodes
- Fix open-file route Windows path quoting
- Add path traversal protection to workflow-images route
- Add AudioInputNode onerror handler for metadata loading failures
- Fix useAudioPlayback uncaught play() promise rejection
- Add generateAudio case to regenerateNode in workflowStore
- Update placeholder Discord links to real invite URL
- Extract ProviderBadge into shared component in Generate3DNode
- Fix VideoTrimNode duration detection with proper cleanup and cancellation
- Fix GenerateAudioNode test fetch mock cleanup
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Switch AVC_LEVEL_4_0 from Baseline profile (avc1.420028) to High profile
(avc1.640028) for broader platform compatibility. Feed audio to the muxer
before video frames so packets are properly interleaved — previously all
audio was dumped after video, causing Discord to skip the audio stream.
Probe total video duration upfront to trim audio to match video length.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Register VideoFrameGrabNode in node exports, nodeTypes map, and minimap colors
- Add to ConnectionDropMenu as video target (Frame Grab) and image source
- Add executeVideoFrameGrab executor with canvas-based frame extraction
- Wire into all three execution paths: executeWorkflow, regenerateNode, executeSelectedNodes
- Add video connection validation for videoFrameGrab target type
- Add handle mapping for video->videoFrameGrab and videoFrameGrab->image connections
- Add videoFrameGrab case to quickstart validation defaults
- Add 'videoFrameGrab' to NodeType union and VideoFrameGrabNodeData interface
- Register default dimensions (320x320) and default data in nodeDefaults
- Wire getSourceOutput to return image type from videoFrameGrab.outputImage
- Add videoFrameGrab dimensions to local Record types in WorkflowCanvas and validation
- Add 'videoTrim' to NodeType union and VideoTrimNodeData interface in nodes.ts
- Add VideoTrimNodeData to WorkflowNodeData union
- Add default dimensions (360x360) and default data to nodeDefaults.ts
- Create useTrimVideo.ts hook with MediaBunny frame-by-frame trim + audio preservation
- Add getSourceOutput case for videoTrim in connectedInputs.ts
- Add executeVideoTrim executor to videoProcessingExecutors.ts
- Wire 'videoTrim' case into executeNode.ts dispatcher
- Fix duplicate Record<NodeType> dimensions in WorkflowCanvas.tsx and validation.ts
Add comprehensive text-to-speech (TTS) support with a new generateAudio node
that integrates with multiple AI providers (Kie.ai, Replicate, fal.ai, WaveSpeed).
Changes:
- Add 'text-to-audio' capability to ModelCapability type
- Add GenerateAudioNodeData type and CarouselAudioItem for audio history
- Create GenerateAudioNode component with audio playback and waveform visualization
- Add TTS models from Kie.ai (ElevenLabs Turbo v2.5, Multilingual v2, Eleven V3)
- Update fal.ai category mapping to support 'text-to-speech' -> 'text-to-audio'
- Add audio filter option to ModelSearchDialog
- Register generateAudio node in WorkflowCanvas with fuchsia minimap color
- Add generateAudio to ConnectionDropMenu for both audio source and text target
- Update connectedInputs to handle audio output from generateAudio nodes
- Add default node data and dimensions for generateAudio type
- Update quickstart validation to include generateAudio node type
The node supports:
- Text-to-audio generation with multiple TTS models
- Real-time audio playback with waveform visualization
- Audio history carousel for navigating generated audio
- Dynamic model parameters based on selected provider
- Integration with existing audio handle system
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add optional fields to track the auto-saved 3D model filename and
file path, with null defaults in nodeDefaults and quickstart validation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: extract schema utilities from generate route
Extract INPUT_PATTERNS, InputMapping, ParameterTypeInfo, getParameterTypesFromSchema,
coerceParameterTypes, and getInputMappingFromSchema into schemaUtils.ts with 23 new tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: extract Gemini provider from generate route
Move MODEL_MAP and generateWithGemini() into providers/gemini.ts.
Remove unused imports (GoogleGenAI, uploadImageForUrl, shouldUseImageUrl,
deleteImages, ProviderModel) from route.ts.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: extract Replicate provider from generate route
Move generateWithReplicate() into providers/replicate.ts.
It imports schema utilities from schemaUtils.ts.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: extract fal.ai, Kie.ai, WaveSpeed providers from generate route
Move generateWithFalQueue, generateWithKie, generateWithWaveSpeed and all
their helpers into providers/fal.ts, providers/kie.ts, providers/wavespeed.ts.
Add barrel export at providers/index.ts.
route.ts reduced from 2,838 to 518 lines (POST handler + routing only).
Re-export clearFalInputMappingCache from route.ts for test backward compat.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: extract API header builder from workflow store
Replace 6 duplicated header-building if/else chains (across executeWorkflow
and regenerateNode) with buildGenerateHeaders() and buildLlmHeaders() utility
functions. Add 13 new tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: extract execution utilities from workflow store
Move groupNodesByLevel, chunk, revokeBlobUrl, clearNodeImageRefs,
and concurrency settings into executionUtils.ts with 21 new tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: extract getConnectedInputs and validateWorkflow as pure functions
Move getConnectedInputs (136 lines) and validateWorkflow (59 lines) into
connectedInputs.ts as pure functions taking (nodeId, nodes, edges).
Store methods become thin wrappers. Add 22 new tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: define node executor interface and context types
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: extract simple node executors
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: unify nanoBanana execution into shared executor
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: unify generateVideo execution into shared executor
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: unify llmGenerate execution into shared executor
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: unify splitGrid execution into shared executor
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: unify videoStitch and easeCurve executors
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: replace switch/if-else with node executor registry
Wire extracted executors into workflowStore, replacing the ~1000-line
switch statement in executeWorkflow and ~750-line if-else chain in
regenerateNode with executor registry calls. Store reduced from 3,402
to 1,782 lines.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: final cleanup of workflowStore module structure
Remove dead trackSaveGeneration function (no longer called after executor
extraction) and unused type imports. Store: 3,838 -> 1,721 lines (55% reduction).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: video regeneration spinner stuck + cancel error message
Fix video element not reloading on regeneration by adding key prop tied
to video history ID. Fix cancel showing timeout message by passing
"user-cancelled" reason through AbortController and checking it in all
three executor catch blocks.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address PR #59 review — 18 bug, security, and robustness fixes
Providers:
- fal: don't cache failed schema fetches, remove duplicate param spread, SSRF-validate mediaUrl
- gemini: remove redundant header overrides, fix nano-banana model ID
- kie: module-level MAX_MEDIA_SIZE, post-download size check, upload size guard
- replicate: remove duplicate param spread, validate modelId format
- wavespeed: post-download size check, NaN-safe parseInt, prevent prompt overwrite
Executors:
- generateVideo: validate stored fallback inputs same as regular path
- nanoBanana: cap imageHistory at 50 entries
- splitGrid: handle img.onerror with warning + null data
- videoProcessing: add FileReader onerror/onabort handlers
Utilities:
- connectedInputs: use ?? instead of || for outputText/template
- executionUtils: guard orphan children in topological sort, validate chunk() size
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: additional PR review fixes — SSRF, data semantics, sync tracking, error propagation
Providers:
- replicate: add SSRF validation on mediaUrl before fetch
- fal/kie/replicate/wavespeed: large-video returns data="" with url set
instead of putting raw URL in data field (violates GenerationOutput.data semantic)
- kie: remove stale WaveSpeed JSDoc block from kie provider file
- route.ts: update all output handling to support data="" + url for large videos
Executors:
- generateVideoExecutor/nanoBananaExecutor: track save-generation fetch in
pendingImageSyncs via new trackSaveGeneration context method so auto-save
waits for server ID resolution
- videoProcessingExecutors: throw errors instead of returning in validation
branches (encoderSupported, insufficient videos, no video connected) so
Promise.allSettled sees rejections
- types.ts: add trackSaveGeneration to NodeExecutionContext interface
- workflowStore: wire trackSaveGeneration into both execution context sites
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: content-type fallback, replicate polling/validation, atomic gallery append
- fal.ts: Use model capabilities to determine fallback content-type
(video/mp4 vs image/png) instead of defaulting to video for all models
- replicate.ts: Increase polling timeout to 10 min for video models;
filter output array to valid strings before URL validation
- nanoBananaExecutor.ts: Replace read-modify-write on outputGallery with
atomic appendOutputGalleryImage using Zustand set() callback
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: extract _buildExecutionContext to deduplicate NodeExecutionContext construction
Replace identical inline context creation in executeWorkflow and regenerateNode
with a single shared helper, ensuring consistent behavior across both code paths.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add keyboard shortcuts help dialog
- New KeyboardShortcutsDialog component showing all available shortcuts
- Accessible via ? button in header and ? key on canvas
- Groups shortcuts by category: General, Add Nodes, Layout, Canvas
- Auto-detects Mac vs Windows/Linux for modifier key display
- Follows existing modal patterns (Escape to close, backdrop click)
* feat: add keyboard shortcuts dialog and 3D GLB viewer node
- Keyboard shortcuts: press ? or click keyboard icon to view all shortcuts
- 3D GLB viewer: upload .GLB files, orbit/rotate, capture as image output
- Fix: restore trailing newline in gemini.ts provider
* fix: make Prompt node editable when connected to LLM node
When a Prompt node receives text from an LLM Generate node, the text
should be editable by the user. Previously, the textarea was marked as
readOnly when there was an incoming text connection, preventing edits
in both the main view and the expanded modal.
Changes:
- Track last received LLM output to detect when it changes
- Only update prompt when LLM node runs again (output changes)
- Allow user edits in between LLM runs
- Remove readOnly restriction on textarea
- Update placeholder text to indicate editability
This allows users to edit the LLM-generated text, which will be
replaced only when the LLM node runs again, not on every render.
Fixes#1470061054246518927
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix: PR review fixes — lazy-load three.js, blob URL cleanup, missing integrations
- Remove unused `useLoader` import from GLBViewerNode
- Add useEffect cleanup to revoke blob URL on node unmount (memory leak)
- Replace alert() with useToast for file validation errors
- Lazy-load GLBViewerNode via next/dynamic to avoid bundling three.js for all users
- Add glbViewer to ConnectionDropMenu IMAGE_SOURCE_OPTIONS
- Add glbViewer case to quickstart validation createDefaultNodeData (build fix)
- Restore trailing newline in gemini.ts
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address PR #64 review — blob leak, error handling, tests, docs
Move @types/three to devDependencies, fix blob URL cleanup to track
changes (not just unmount), add try/catch to capture and scene disposal,
remove GLBViewerNode from barrel export to preserve lazy-loading, fix
JSDoc, update CLAUDE.md with glbViewer node and ? shortcut, add
connectedInputs and nodeDefaults tests for glbViewer.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: improve GLB viewer — fix viewport sizing, add spot light, handle labels, scroll isolation
- Fix 3D viewport not filling node by adding resize={{ offsetSize: true }} for correct measurement in CSS-transformed React Flow nodes
- Replace grid/axis indicators with spot light for cleaner scene
- Add labeled input (3D) and output (Image) handles matching generate node style
- Add native wheel event listener to prevent graph zoom/pan while scrolling to zoom 3D model
- Remove redundant CSS workarounds (absolute positioning style, extra relative wrapper)
- Add edge-to-edge viewport via contentClassName (removes default padding when GLB loaded)
- Overlay controls bar on viewport with gradient background
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add customizable canvas navigation settings (#65)
* feat: add customizable canvas navigation settings
Add a new settings modal that allows users to configure canvas navigation preferences:
- Pan Mode: Space + Drag (default), Middle Mouse, or Always On (ComfyUI-style)
- Zoom Mode: Alt + Scroll (default), Ctrl + Scroll, or Scroll (no modifier)
- Selection Mode: Click (default) or Alt + Drag
Features:
- Settings persist in localStorage
- New canvas settings button in header (monitor icon)
- Modal UI with radio button groups for each setting
- React Flow props dynamically configured based on user preferences
- Updated wheel event handler to respect zoom mode settings
This addresses user requests for ComfyUI/TouchDesigner-style navigation where
panning and zooming work without holding modifier keys.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix: PR #65 review — altDrag selection bug, icon, dedup, tests
- Fix altDrag selection mode: use selectionKeyCode="Alt" instead of
selectionOnDrag=true which made drag-select work without any key
- Replace misleading video camera icon with arrows-pointing-out for
canvas navigation settings button
- Extract duplicated settings buttons JSX into settingsButtons variable
shared by both configured/unconfigured project branches
- Add defensive useEffect to sync CanvasSettingsModal state on open
- Add missing semicolon in types/index.ts canvas re-export
- Add localStorage tests for getCanvasNavigationSettings and
saveCanvasNavigationSettings
- Add integration tests for updateCanvasNavigationSettings
- Fix WorkflowCanvas test mocks to include canvasNavigationSettings
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: move canvas navigation settings into Project Settings modal
Consolidate canvas settings from standalone CanvasSettingsModal into a
new "Canvas" tab in ProjectSetupModal. Remove "like ComfyUI" text, add
Shift+Drag selection mode, and fix save button to bottom of dialog.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: remove double semicolon in types/index.ts
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
* feat: add executeSelectedNodes with topo sort, abort, and downstream propagation
Implements the "Run selected nodes" feature (PR #61) with fixes for all 8
review issues: topological ordering via groupNodesByLevel, AbortController
cancellation, glbViewer case coverage, AbortError filtering, downstream
consumer propagation with dedup guard, and dynamic button label. Also
extracts saveLogSession helper to replace 4 inline duplicates across
executeWorkflow and regenerateNode.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: 3D model saving, error handling, and node improvements
- Save 3D models (GLB) to generations folder via save-generation API
- Add try-catch to WaveSpeed polling loop to handle network errors
- Skip downloading 3D model bodies in WaveSpeed (return URL directly)
- Fix schemaUtils false-positive matching for short property names
- Stabilize GenerateImageNode output handle ID (always "image")
- Fix GLBViewerNode: block-body forEach lint, store-based resize
- Use useRef instead of useState for PromptNode lastReceivedText
- Add executeGlbViewer tests and abort signal support
- Fix annotation stale pass-through when upstream image changes
- Skip error status on AbortError in all executor catch blocks
- Revoke glbUrl blob URLs in clearNodeImageRefs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: require per-task commits in multi-task plans
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add Generate3DNodeData type definitions
Add "generate3d" to NodeType union, Generate3DNodeData interface,
Generate3DNodeDefaults, and update NodeDefaultsConfig.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add generate3d node defaults and dimensions
Add default dimensions (300x300) and createDefaultNodeData case for
the generate3d node type.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: create generate3d executor
Extracted from nanoBananaExecutor's 3D handling code. Handles 3D model
generation via /api/generate with mediaType: "3d", cost tracking, and
auto-save to generations folder.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: create Generate3DNode component
Dedicated node component for 3D model generation, modeled after
GenerateVideoNode. Features: dynamic input handles from schema,
3D output handle, model browsing, parameter controls, status overlays.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: register generate3d node type across codebase
Register in execution index, workflowStore (executeWorkflow,
regenerateNode, executeSelectedNodes), nodes index, WorkflowCanvas
(nodeTypes, getNodeHandles, minimap color, findCompatibleHandle).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add generate3d source output in connectedInputs
Add generate3d case in getSourceOutput() to return 3D URL data,
enabling downstream nodes to receive 3D model output.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: route 3D models to generate3d in ModelSearchDialog
Add "3d" capability filter, update handleSelectModel to detect 3D
models and create generate3d nodes with capabilities passed through,
update recent models filter for 3D.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: update ConnectionDropMenu 3D source to use generate3d
Change THREE_D_SOURCE_OPTIONS from nanoBanana to generate3d with
cube icon and "Generate 3D" label.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: remove 3D code from GenerateImageNode and nanoBananaExecutor
- Remove text-to-3d/image-to-3d from IMAGE_CAPABILITIES
- Remove is3D memo and conditional handle/label styling
- Remove 3D output preview block
- Remove output3dUrl from model change handlers
- Remove is3DModel detection and mediaType from nanoBanana executor
- Remove 3D response handling from nanoBanana executor
- Remove output3dUrl from NanoBananaNodeData and its defaults
- Remove 3D fallback from nanoBanana in connectedInputs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: update tests and remaining registrations for generate3d
- Add Generate3DNode.test.tsx with render, handle, and output tests
- Add 3D handle tests to ConnectionDropMenu.test.tsx
- Add 3D model routing test to ModelSearchDialog.test.tsx
- Update ModelSearchDialog test expectations for capabilities passthrough
- Register generate3d in executeNode.ts dispatcher
- Add generate3d to WorkflowCanvas keyboard shortcut dimensions
- Add generate3d to quickstart validation (VALID_NODE_TYPES, dimensions, defaults)
- Add generate3d to chat tools (VALID_NODE_TYPES, description)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: unwrap array inputs for non-array API parameters
When multiple images are connected to a generation node, dynamicInputs
aggregates them into arrays. Providers wrapped single values into arrays
for array-typed params, but never unwrapped arrays for string-typed params,
causing API failures (e.g. fal's image_url receiving an array instead of
a string). Add symmetric unwrap logic across all four providers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: move 3D from floating action bar to Generate submenu
Replace top-level glbViewer button with generate3d option in the
Generate combo dropdown, between Video and Text (LLM).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: hide CostIndicator when non-Gemini providers are in workflow
The pricing calculator only has reliable data for Gemini models. When
non-Gemini providers (fal, replicate, kie, wavespeed) are used, the
displayed cost is incomplete/misleading. Add hasNonGeminiProviders()
utility and use it to hide the CostIndicator entirely in that case.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add 3D capability badges to ModelSearchDialog
Add text-to-3d and image-to-3d capability badge styling.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add play/run icon to LLM Generate node title bar
Wire onRun and isExecuting props to BaseNode, matching the pattern
used in GenerateImageNode and GenerateVideoNode.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: clear imageRef when user provides new image to prevent stale saves
When a user pastes/uploads a new image into a node that already has a
saved imageRef, the externalization logic incorrectly assumes the base64
is just hydrated data and discards it. Clear *Ref fields whenever new
image data is set so the save logic correctly persists the new image.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: guard against undefined from empty array unwrap in fal provider
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add 3x2 grid layout option to split grid node
Change layout selector from count-based to layout-based, allowing both
2x3 and 3x2 (portrait) layouts that produce 6 images. Now shows 7
layout options (2x2, 1x5, 2x3, 3x2, 2x4, 3x3, 2x5) with RxC labels.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: charlieshoelace <charlieshoelace@gmail.com>
- Add Generate3DNode.test.tsx with render, handle, and output tests
- Add 3D handle tests to ConnectionDropMenu.test.tsx
- Add 3D model routing test to ModelSearchDialog.test.tsx
- Update ModelSearchDialog test expectations for capabilities passthrough
- Register generate3d in executeNode.ts dispatcher
- Add generate3d to WorkflowCanvas keyboard shortcut dimensions
- Add generate3d to quickstart validation (VALID_NODE_TYPES, dimensions, defaults)
- Add generate3d to chat tools (VALID_NODE_TYPES, description)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix 3D viewport not filling node by adding resize={{ offsetSize: true }} for correct measurement in CSS-transformed React Flow nodes
- Replace grid/axis indicators with spot light for cleaner scene
- Add labeled input (3D) and output (Image) handles matching generate node style
- Add native wheel event listener to prevent graph zoom/pan while scrolling to zoom 3D model
- Remove redundant CSS workarounds (absolute positioning style, extra relative wrapper)
- Add edge-to-edge viewport via contentClassName (removes default padding when GLB loaded)
- Overlay controls bar on viewport with gradient background
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
huangmin
TianYun