The LLM selector exposed display-style Doubao model names that the NewApiWG test gateway cannot route. The UI now stores the gateway model ids returned by /models, while the API route maps legacy saved selector values to those ids so existing workflows do not keep failing.
Constraint: Preserve GPT-4.1 Mini as the NewApiWG default and keep old workflow JSON compatible.
Rejected: Remove Doubao from the LLM selector | the gateway exposes working Doubao chat model ids, so the issue is stale ids rather than product support.
Confidence: high
Scope-risk: narrow
Directive: LLM selector values must be callable provider ids; use labels only for display text.
Tested: Live NewApiWG chat completion with doubao-seed-2-0-lite-260428 and doubao-seed-2-0-pro-260215.
Tested: npm run test:run -- src/lib/__tests__/llmModels.test.ts src/app/api/llm/__tests__/route.test.ts src/components/__tests__/LLMGenerateNode.test.tsx src/store/utils/__tests__/nodeDefaults.test.ts
Tested: npm run build
Tested: git diff --check
Seedream 5.0 rejects the previous default 1024x1024 size when the user selects high resolution output. The NewApiWG image endpoint now derives a provider pixel size from the visible resolution and aspect-ratio controls unless the model schema supplied an explicit size.
Constraint: Keep the fix server-side and preserve explicit model size parameters when provided.
Rejected: Change the composer resolution UI for one model | the UI already sends the intended 4K and 16:9 values; the loss happened in provider payload mapping.
Confidence: high
Scope-risk: narrow
Directive: Do not default OpenAI-compatible image endpoints to 1024x1024 when resolution/imageSize is present.
Tested: npm run test:run -- src/app/api/generate/providers/__tests__/newapiwg.test.ts
Tested: npm run test:run -- src/app/api/generate/__tests__/route.test.ts
Tested: npm run build
Tested: git diff --check
Not-tested: Live Seedream 5.0 paid generation call.
P0 regressions showed several workflow paths needed to stay visible and deterministic: language switching had a hardcoded composer label, Video Trim hid controls before media loaded, and media handoffs to LLM needed real canvas edges rather than implicit conversion. P1 fixes keep canvas-owned structure by adding explicit image/video LLM handles, audio composer validation, 3D output/file routing, and Chat proposal cards that require user apply before mutating nodes.
Constraint: User asked to stop new feature work and only repair P0/P1 product-fit gaps.
Rejected: Auto-convert image or video directly into text for LLM | hides workflow structure and violates the visible-node product model.
Rejected: Let Chat tool calls silently add or edit nodes | canvas structure must be previewed and explicitly applied.
Confidence: high
Scope-risk: moderate
Directive: Keep media transformations represented by visible handles, nodes, and edges; do not add silent Chat or media mutation paths without product review.
Tested: Browser automation for language switch, video input to Generate Video, prompt to Generate Video, LLM default model, Video Trim controls, and video media to LLM input.
Tested: npm run test:run -- src/components/__tests__/ConnectionDropMenu.test.tsx src/components/__tests__/GenerationComposer.test.tsx src/components/__tests__/VideoTrimNode.test.tsx src/components/__tests__/LLMGenerateNode.test.tsx src/store/execution/__tests__/llmGenerateExecutor.test.ts src/app/api/llm/__tests__/route.test.ts src/store/execution/__tests__/generateAudioExecutor.test.ts src/store/execution/__tests__/generate3dExecutor.test.ts src/store/execution/__tests__/simpleNodeExecutors.test.ts src/components/__tests__/OutputNode.test.tsx src/components/__tests__/Generate3DNode.test.tsx src/components/__tests__/ChatPanel.test.tsx src/lib/chat/tools.test.ts src/lib/chat/editOperations.test.ts src/lib/__tests__/llmModels.test.ts src/store/utils/__tests__/nodeDefaults.test.ts
Tested: npm run build
Tested: git diff --cached --check
Not-tested: npm run lint, because the existing script calls removed Next.js lint command and exits with invalid project directory /lint.
Not-tested: Live paid provider calls for LLM, audio, and 3D generation.
P0/P1 reports clustered around unavailable LLM defaults, video media connections, and missing controls on the generation surface. This routes the default NewApiWG LLM away from the unavailable Doubao channel, makes video a first-class schema and canvas connection type, carries request ids into node errors, and fills the composer, trim, audio, and 3D gaps without changing the canvas workflow model.
Constraint: NewApiWG Doubao channels are not reliably routed in the default gateway
Constraint: Video model schemas expose video fields that must not flow through image handles
Rejected: Continue accepting video through image handles | hides incorrect graph types and loses schema-specific dynamic inputs
Confidence: high
Scope-risk: moderate
Directive: Keep media handle types explicit; add schema mapping when adding a new connectable media type
Tested: npm run test:run -- src/store/execution/__tests__/generateVideoExecutor.test.ts src/store/utils/__tests__/connectedInputs.test.ts src/store/execution/__tests__/generateAudioExecutor.test.ts src/store/execution/__tests__/llmGenerateExecutor.test.ts src/store/__tests__/workflowStore.integration.test.ts src/lib/__tests__/llmModels.test.ts src/store/utils/__tests__/nodeDefaults.test.ts src/components/__tests__/GenerationComposer.test.tsx src/app/api/chat/__tests__/route.test.ts src/app/api/llm/__tests__/route.test.ts src/app/api/models/[modelId]/__tests__/route.test.ts src/app/api/generate/providers/__tests__/newapiwg.test.ts
Tested: npm run build
Tested: git diff --check
Not-tested: npm run lint | package script calls removed Next 16 next lint command
Not-tested: npx tsc --noEmit | existing test type errors remain; changed runtime files filtered clean
NewApiWG model discovery was allowing Doubao text models to appear in media-generation contexts, which made image nodes call a model with no generation channel. The change filters Doubao chat/vision model names out of generation capability inference, bumps model caches, and guards execution for stale saved workflows.
Constraint: NewApiWG /models may return chat models without explicit capability metadata
Rejected: Keep all unknown models visible | it lets text-only models fail inside media nodes
Confidence: high
Scope-risk: narrow
Directive: Do not classify chat/VLM models as media generators without explicit generation capability metadata
Tested: npm test -- src/app/api/generate/providers/__tests__/newapiwg.test.ts
Tested: npm test -- src/components/__tests__/ModelSearchDialog.test.tsx
Not-tested: Full tsc remains blocked by existing unrelated test type errors
Image generation settings arrive at /api/generate as top-level aspectRatio and resolution fields, but the NewApiWG provider path only forwarded model-specific parameters. Native NewApiWG Gemini image generation therefore fell back to default imageConfig values and ignored composer/control-panel ratio choices. The route now merges top-level image settings into provider parameters while preserving explicit model parameter overrides.
Constraint: NewApiWG provider implementations read aspect ratio and image size from GenerationInput.parameters.
Rejected: Parse aspect ratio from prompt text | UI settings should be authoritative and prompts are ambiguous.
Confidence: high
Scope-risk: narrow
Directive: Keep top-level GenerateRequest image settings and provider parameters in sync for image providers.
Tested: npm test -- --run src/app/api/generate/__tests__/route.test.ts src/store/execution/__tests__/nanoBananaExecutor.test.ts src/components/__tests__/GenerationComposer.test.tsx
Tested: npx tsc --noEmit --pretty false filtered to generate route files
Not-tested: Full tsc due existing test mock overload errors in route.test.ts
This aligns image, video, and text defaults with the PopiArt NewApi gateway while keeping provider-specific escape hatches available. Chat now uses the OpenAI-compatible NewApiWG route so workflow editing and LLM generation share the same Doubao-first text default. The canvas composer and quick-add controls are tightened so visible actions map to real clickable behavior, while temporarily unused header actions stay hidden.
Constraint: Text/chat requests must prefer Doubao-Seed-2.0-lite through NewApiWG.
Constraint: Image and video defaults should prefer PopiArt NewApi gateway models when configured.
Rejected: Keep /api/chat on Gemini 3 Flash | conflicts with the requested NewApiWG text/chat default.
Rejected: Hand-roll AI SDK chat streaming | would risk tool-call and UI stream compatibility.
Confidence: high
Scope-risk: moderate
Directive: Do not add new text/chat defaults without updating src/lib/llmModels.ts and the chat route together.
Tested: npm test -- --run src/app/api/chat/__tests__/route.test.ts src/components/__tests__/LLMGenerateNode.test.tsx src/lib/__tests__/llmModels.test.ts src/store/utils/__tests__/nodeDefaults.test.ts src/components/__tests__/GenerationComposer.test.tsx src/lib/chat/tools.test.ts
Tested: npm test -- --run src/lib/__tests__/llmModels.test.ts src/store/utils/__tests__/nodeDefaults.test.ts src/components/__tests__/GenerationComposer.test.tsx src/lib/quickstart/__tests__/templates.test.ts src/lib/quickstart/__tests__/validation.test.ts src/lib/quickstart/__tests__/prompts.test.ts src/components/__tests__/ProjectSetupModal.test.tsx
Tested: npm test -- --run src/components/__tests__/WorkflowCanvas.test.tsx
Not-tested: Full suite; repository has pre-existing unrelated type/test warnings outside this change.
The bottom composer now edits and reruns the current generation node while canvas-visible actions own downstream node creation. NewApiWG media normalization, video frame extraction, and node quick-add keep generated media flows explicit and connected rather than hidden behind the prompt box.
Constraint: Composer must not call providers directly or create hidden downstream branches when a node is selected
Constraint: NewApiWG video families use inconsistent media schemas, so common media aliases are normalized in the provider adapter
Rejected: Let the bottom composer create next-step nodes from selected outputs | hides graph mutation and conflicts with the PRD source-of-truth rules
Rejected: Put NewApiWG model-family payload rules in the composer | provider-specific schemas would leak into UI state
Confidence: high
Scope-risk: moderate
Directive: Keep provider payload normalization in provider/executor layers; do not move it into GenerationComposer
Tested: npm run test:run -- src/components/__tests__/GenerationComposer.test.tsx
Tested: npm run test:run -- src/components/__tests__/WorkflowCanvas.test.tsx
Tested: npm run test:run -- src/components/__tests__/GenerateVideoNode.test.tsx
Tested: npm run test:run -- src/app/api/generate/providers/__tests__/newapiwg.test.ts src/app/api/generate/__tests__/route.test.ts
Tested: npm run test:run -- src/store/execution/__tests__/videoProcessingExecutors.test.ts
Tested: npm run build
Not-tested: npm run lint fails because next lint is incompatible with the current Next 16 script configuration
Remote and server-hosted deployments cannot rely on the browser user's desktop tools being available inside the Node server process. Directory browsing now checks for native picker commands first and falls back to a configurable server-side workflows directory instead of surfacing command-not-found errors.
Constraint: The browse endpoint runs on the server, not on the Windows client browser
Constraint: Test servers may not have osascript, zenity, kdialog, PowerShell, or an interactive desktop session
Rejected: Keep propagating native picker failures to the modal | successful server deployments without desktop tools showed red command errors
Confidence: high
Scope-risk: narrow
Directive: For true client-local folder selection, use a separate File System Access API flow instead of this server filesystem endpoint
Tested: npx vitest run src/app/api/browse-directory/__tests__/route.test.ts src/components/__tests__/ProjectSetupModal.test.tsx
Tested: npm run build
Not-tested: Interactive Windows Server desktop picker session
NewApiWG image and video models need to work against the PopiArt test gateway, with generated remote media visible in the canvas and UI text localized for Chinese users. This wires the gateway-specific image/video paths, model fallback display, remote media proxying, and related i18n cleanup into one releaseable change.
Constraint: Test gateway exposes image and video models through mixed OpenAI-compatible and Gemini-native response shapes
Constraint: Browser video playback can fail when loading signed third-party media URLs directly
Rejected: Keep remote video URLs as direct <video> sources | CORS and range handling made successful generations appear unloaded
Confidence: high
Scope-risk: moderate
Directive: Do not remove the media proxy without validating remote video playback, trimming, and stitching against signed CDN URLs
Tested: npm run build
Tested: npx vitest run src/components/__tests__/OutputNode.test.tsx src/components/__tests__/GenerateVideoNode.test.tsx src/components/__tests__/VideoStitchNode.test.tsx
Tested: npx vitest run src/components/__tests__/WorkflowCanvas.test.tsx src/store/execution/__tests__/generateVideoExecutor.test.ts
Tested: npx vitest run src/app/api/generate/providers/__tests__/newapiwg.test.ts
Not-tested: Live browser replay after pushing to remote deployment
Kie rejects the call when both reference_image_urls and first_frame_url
(or last_frame_url) are populated — the API treats them as mutually
exclusive scene modes. Two changes:
1. kie.ts submit: skip the legacy image fallback whenever any
schema-driven image key was already handled via dynamicInputs. The
executor passes both `images` and `dynamicInputs`, so the fallback
was duplicating user-provided references into first_frame_url.
2. seedance2 I2V schema: mark first_frame_url optional and note the
mutual exclusivity in the handle descriptions so users see it in
the node UI.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add last_frame_url, reference_image_urls, reference_video_urls, and
reference_audio_urls handles to Seedance 2.0 and Seedance 2.0 Fast I2V
schemas so users can seed an end frame, supplementary reference images,
reference videos, and reference audio in addition to the first frame.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Resolve conflicts in 4 executor files by combining develop's
runWithFallback structure with feature branch's client-side polling.
Both capabilities are now integrated: executors use runWithFallback
for model redundancy, and pollGenerateTask for long-running Kie tasks.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
New endpoint accepts POST with taskId/provider/modelId and returns
either a polling-continue response or the final media result. Each
request is short-lived (~1s) instead of holding a connection open.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The /api/generate route now submits Kie tasks and returns immediately
with polling metadata (taskId, pollProvider, etc.) instead of holding
the HTTP connection open for 2-10 minutes. Export buildMediaResponse
for reuse by the poll endpoint.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Split the monolithic generateWithKie into three reusable lifecycle functions
for client-side polling support. generateWithKie now composes them internally
for backward compatibility. Added polling fields to GenerateResponse type.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wrap fetch and JSON parsing in try-catch to handle network errors,
add 429/5xx to retryable status codes, and implement exponential
backoff (2s → 10s) to reduce API load during long-running generations.
Caps consecutive errors at 5 before failing permanently.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The Kie API can return 404/422 or HTTP 200 with non-200 code when
polling starts before the task is fully registered. Previously these
were treated as fatal errors, causing the client to see a network error
even though the generation was still processing on Kie's side. Now
these transient responses are retried instead of aborting.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
maxDuration was 5 minutes which caused Next.js to kill the route handler
before Seedance 2.0 video generation polling could complete, resulting
in a network error on the client despite successful generation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The Kie API expects "bytedance/seedance-2" as the model value, but we
were sending our internal ID "bytedance/seedance-2/text-to-video" which
includes the capability suffix. Strip the suffix for Seedance models
before sending to the API.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Gemini image models (nano-banana, nano-banana-pro, nano-banana-2) were
not showing parameters when set as fallback models. Three root causes:
1. Schema API returned empty for Gemini image models — added
getGeminiImageSchema() with aspect ratio, resolution, and search
grounding parameters per model
2. ModelParameters had an early-return guard skipping all non-Veo Gemini
models — removed to make the component provider-agnostic
3. nanoBananaExecutor ignored parametersOverride for Gemini-specific
fields (aspectRatio, resolution, useGoogleSearch, useImageSearch) —
now reads from override when available
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
The Kie docs have contradictory info — the parameter description says
"std"/"pro" but the example request body uses "720p"/"1080p". The API
actually accepts "720p"/"1080p" (same as Kling 2.6 Motion Control),
rejecting "std"/"pro" with "mode is not within the range of allowed
options."
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Per Kie docs, background_source accepts "input_video"/"input_image",
not "image"/"video". The previous wrong values were being sent to the
API after schema defaults pre-population, causing request rejection.
Also added background_source to getKieModelDefaults() for consistency.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
The dynamic input upload logic only matched data:image URLs, so video
data (data:video/mp4) was passed as raw base64 instead of being
uploaded to get a hosted URL. This caused "File type not supported"
errors for models like Kling 3.0 Motion Control that accept video.
- Rename uploadImageToKie → uploadMediaToKie (keep alias for compat)
- Rename detectImageType → detectMediaType with video magic bytes and
fallback to declared MIME type for non-image content
- Broaden data URL checks from data:image to data: in generateWithKie
- Set uploadPath to videos/audio/images based on detected MIME type
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add Seedance 2.0 and 2.0 Fast with text-to-video and image-to-video
variants. Both support audio generation, web search, and 4-15 second
duration. Image input uses first_frame_url (singular string).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add first_frame_url, last_frame_url, and first_clip_url to the
singular-key checks in generateWithKie() so these params receive a
single string URL instead of being wrapped in an array.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
isImageInput() has permissive description heuristics (e.g. "data uri")
that false-positive on audio properties like audio_url whose description
mentions "base64 data URI". Reorder the checks so isAudioInput() — which
uses specific name patterns — runs first.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add AUDIO_INPUT_PATTERNS, isAudioInput() function modeled after
isImageInput(), and integrate into extractParametersFromSchema() to
detect audio inputs from model schemas. Audio inputs are sorted
between image and text inputs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add "audio-to-video" to ModelCapability union, extend ModelInput.type
and ModelInputDef.type with "audio", and include "audio-to-video" in
fal.ai RELEVANT_CATEGORIES for model discovery.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- GroupsOverlay: reset color picker when menu closes, guard locked groups
from drag/resize/rename
- AudioInputNode: add keyboard accessibility to dropzone (role, tabIndex,
onKeyDown)
- list-workflows: fix name regex to handle escaped quotes
- GroupsOverlay.test: add missing NBP Input menu item assertion
- WorkflowBrowserView: make relativePath required to match API contract
- WorkflowBrowserModal: add dialog role, aria-modal, aria-labelledby
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- cleanup-workflow.js: Filter only embedded data: images from outputGallery
instead of wiping entire array; preserve imageRef/video on output nodes
- workflow/route.ts: Validate parsed.version is number, nodes/edges are
arrays; sort candidates by mtime desc for deterministic file selection
- mediaStorage.ts: Guard against undefined mediaData in loadMediaById
before caching/returning
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add recursive directory scanning (up to 3 levels deep) to the
list-workflows API so projects organized into subdirectories are
visible in the workflow browser. Skips hidden dirs and node_modules.
Returns relativePath field displayed as the subtitle in the UI.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Instead of parsing entire workflow JSON files (which can be many MB with
embedded base64 data), read only the first 1KB to extract the name via
regex. Drop nodeCount from the listing. Probe all directories in parallel.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Scans subdirectories of a given parent path for valid workflow JSON files,
returning name, path, node count, and last modified time.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace browser file picker with the existing /api/browse-directory native
OS folder picker so loadWorkflow receives the actual filesystem path,
enabling media hydration from sibling inputs/ and generations/ folders.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use API response imageId instead of locally generated ID in
saveVideoAndGetRef/saveAudioAndGetRef (filename mismatch)
- Read result.video/result.audio instead of result.data in
loadMediaById (response field mismatch)
- Add audio extensions and MIME types to load-generation route
so audio files can be found and served
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Critical bug fix - the mediaStorage.ts functions were using incorrect
parameter names when calling the save-generation and load-generation APIs.
Fixed parameter mappings:
- workflowPath → directoryPath (API expects this name)
- id → imageId (API expects this name)
- data + type → video/audio (API expects media in named fields)
- Changed load from GET to POST with correct body structure
Also increased serverActions bodySizeLimit from 50mb to 100mb to handle
large video files during externalization.
This fixes the save failures when externalizing workflows with videos/audio.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add video, audio, text, 3d, easeCurve entries to EDGE_COLORS in SharedEdgeGradients
- Normalize handle type suffixes (e.g. image-0 -> image) in EditableEdge color lookup
- Gate Kie provider filter behind API key check (return 400 if missing)
- Fix getMediaDimensions to check URL pathname for image extensions before defaulting to video
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fal was unconditionally listed as available, which could surface
fal models to users without a fal API key configured.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The ModelSearchDialog only checked client-side localStorage keys to
determine available providers, missing providers configured via .env
server-side vars. Now the /api/models endpoint returns an
availableProviders array and the dialog merges both sources.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Instead of an inline text field, negative prompt is now a proper text
handle on the node that can receive connections from prompt/LLM nodes.
Updates the API route to merge dynamicInputs.negative_prompt into the
Veo parameters, and aligns the model schema definition accordingly.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add early validation for image-to-video models when no image is provided
- Wrap Veo generation/polling in try/catch with specific error messages
- Add 60s AbortController timeout to video download fetch
- Fix SwitchNode toggle knob not sliding (peer-checked on non-sibling)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The MAX_BASE64_SIZE check was returning the video URL with the API key
appended as a query parameter to the browser client for videos over 20MB.
Always encode to base64 instead since the buffer is already downloaded.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
jiajia