popiart-node-canvas/prd-bottom-composer-node-console.md

PRD: Bottom Composer as Current Node Console

Version: 1.2 Last Updated: 2026-05-03 Status: Draft synced with NewApiWG gateway notes Scope: popiart-node-canvas

1. Summary

The bottom composer should become the primary control surface for the currently focused node. For generation nodes it edits prompt/model/parameters; for supported processing nodes it acts as a node inspector with execution controls. It should not act as an implicit "next step" generator when an existing node is selected.

Core product rule:

The canvas owns workflow structure. The bottom composer owns the selected node's parameters and execution.

This means:

• Selecting a supported generation node opens its prompt, model, and parameters in the bottom composer.
• Selecting a supported processing node opens a node-specific inspector, not a prompt editor.
• Clicking the main generate button applies the current composer values to that selected node and reruns that node.
• Creating a next-step node must be expressed through canvas actions: dragging from an output handle, using a node plus action, or using the connection drop menu.
• When there is no selected node, the composer may create a new root generation node in the canvas center. This is the only creation mode owned by the composer.

2. Problem

The current prototype bottom composer visually matches the desired direction, but its behavior is still ambiguous:

• It currently supports only image generation nodes (nanoBanana).
• It can create a new image node or update a selected image node from the same button.
• It always opens the model browser with an image capability filter.
• It does not support all video-related node states, video model selection, processing-node inspection, or clear unsupported-node states.
• It blurs two product concepts: editing the selected node vs creating a downstream node.

For a node canvas product, implicit downstream creation from a bottom prompt box is risky because users expect graph topology to be visible and deliberate.

3. Current Architecture

3.1 Frontend Shell

• Framework: Next.js + React + TypeScript.
• Canvas: @xyflow/react via src/components/WorkflowCanvas.tsx.
• State: Zustand store in src/store/workflowStore.ts.
• Bottom composer entry: src/components/composer/GenerationComposer.tsx.
• Existing right/side settings: src/components/nodes/ControlPanel.tsx.

3.2 Graph Data Model

Node types are defined in src/types/nodes.ts.

Relevant generation node types:

• nanoBanana: image generation.
• generateVideo: video generation.
• generate3d: 3D generation.
• generateAudio: audio generation.
• llmGenerate: text/LLM generation.

Relevant node data fields:

• Image generation: NanoBananaNodeData

  • inputPrompt
  • inputImages
  • aspectRatio
  • resolution
  • model
  • selectedModel
  • parameters
  • inputSchema
  • outputImage
  • status
  • error

• Video generation: GenerateVideoNodeData

  • inputPrompt
  • inputImages
  • selectedModel
  • parameters
  • inputSchema
  • outputVideo
  • status
  • error

3.3 Store and Execution

Existing store APIs:

• addNode(type, position, initialData?)
• updateNodeData(nodeId, data)
• addEdgeWithType(connection, edgeType)
• regenerateNode(nodeId)
• executeWorkflow(startNodeId?)
• executeSelectedNodes(nodeIds)

Execution already supports:

• executeNanoBanana for nanoBanana
• executeGenerateVideo for generateVideo
• executeGenerate3D for generate3d
• executeGenerateAudio for generateAudio
• batch paths for selected generation node types

3.4 Model Layer

The model browser is src/components/modals/ModelSearchDialog.tsx.

It already supports capability filtering:

• all
• image
• video
• 3d
• audio

It fetches models through /api/models and maps filters to capabilities:

• image: text-to-image,image-to-image
• video: text-to-video,image-to-video,audio-to-video
• 3D: text-to-3d,image-to-3d
• audio: text-to-audio

Default NewAPI models exist in src/store/utils/defaultImageModel.ts:

• image: apiyi_nano_banana_2
• video: viduq2-pro-fast

3.5 Canvas Next-Step Creation

WorkflowCanvas.tsx already supports dragging from handles and dropping on blank canvas. The ConnectionDropMenu can create downstream nodes such as:

• nanoBanana
• generateVideo
• generateAudio
• output
• outputGallery
• media processing nodes

This should remain the canonical way to create visible workflow structure.

3.6 Node / Composer / Model API Relationship

flowchart LR
  S["Zustand workflow store\nnodes, edges, selected state"] --> C["ComposerContext\nmode, node, nodeType, capability"]
  C --> D["Composer draft\neditable local form state"]
  D --> P["Adapter buildPatch\nnode-type-specific patch"]
  P --> S
  S --> R["regenerateNode(nodeId)"]
  R --> E["Existing executor\nimage, video, 3D, audio"]
  E --> A["/api/generate\nprovider gateway"]
  A --> M["Model provider\nNewAPI / Gemini / fal / Replicate"]
  M --> S

Ownership:

• Nodes and edges are the source of truth for creative data and workflow structure.
• Composer is a derived view plus a local draft, not a second source of truth.
• selectedModel.capabilities defines which node type and model filter are valid.
• inputSchema and parameters define provider-specific controls.
• Executors and API routes remain the only layer allowed to call model providers.

4. Product Principles

P1. One Focus

The bottom composer must represent exactly one target:

• no selected node: create a new root generation node
• one supported generation node selected: edit and run that node
• one supported processing node selected: inspect inputs/status and run that node
• unsupported node selected: show non-generative context or disabled state
• multiple nodes selected: show batch/selection state, not a single-node editor

P2. No Hidden Graph Mutation

If a node is selected, the main generate button must not silently create a downstream branch.

Next-step creation must happen through visible graph actions:

• drag an output handle to blank canvas
• choose a node type from the connection drop menu
• use a node-level plus action
• select the newly created node
• configure and run it from the bottom composer

P3. Selected Node Is the Source of Truth

For supported generation nodes, the composer reads from and writes to the selected node's data.

The composer should display:

• prompt
• selected model
• model capability
• aspect ratio / resolution / duration where applicable
• reference inputs
• advanced parameters when available
• status and error

P4. NewAPI Is a Provider, Not a Separate UX

The composer should use the existing selectedModel contract and model browser. It should not hardcode direct provider calls.

For NewAPI:

• model selection still writes selectedModel.provider = "newapiwg"
• generation still flows through existing node executors and API routes
• capability filtering determines which NewAPI models appear in each context
• gateway credentials are configured through NEWAPIWG_API_KEY / NEWAPIWG_BASE_URL or project settings
• UI components never call NewApiWG directly; they use /api/models, /api/generate, and existing executors
• provider-specific media mapping belongs in the NewApiWG provider adapter, not in the composer

P5. Upstream Graph Inputs Win

When a generation node has connected upstream prompt/media inputs, those graph connections are the authoritative source for execution. The composer may display them, but it must not hide or override them.

P6. Decisions Must Be Explicit

The implementation should not leave major product behavior to incidental code paths. Draft sync, prompt precedence, video parameter ownership, model switching, and selection ownership are recorded as decision documents under docs/decisions/.

5. Goals

ID	Goal	Priority
G-01	Make bottom composer a clear current-node control surface	Must
G-02	Support editing and rerunning selected image generation nodes	Must
G-03	Support editing and rerunning selected video generation nodes	Must
G-04	Filter model browser by selected node capability	Must
G-05	Allow no-selection composer to create a new root node from any model capability	Should
G-06	Make canvas drag/drop the required path for downstream generation	Must
G-07	Show references and existing node parameters without hiding graph topology	Should
G-08	Preserve UI affordances for unsupported future capabilities without blocking MVP	Should

6. Non-Goals

• Do not use the composer generate button to create downstream nodes when an existing node is selected.
• Do not replace the entire advanced node settings system in the first pass.
• Do not introduce a new model provider abstraction.
• Do not add new dependencies.
• Do not make real API calls during automated tests.
• Do not implement unsupported provider-specific controls unless the model schema already exposes them.

7. User Stories

US-01: Edit an Image Node

As a creator, I select an existing image generation node and see its prompt, image model, aspect ratio, resolution, references, and status in the bottom composer. When I click generate, the selected node reruns with the visible values.

US-02: Edit a Video Node

As a creator, I select an existing video generation node and see its prompt, video model, references, and video parameters in the bottom composer. When I click generate, the selected video node reruns.

US-03: Create a New Root Generation

As a creator, when no node is selected, I can type a prompt, choose any supported model, and click generate. The app creates a new root generation node matching the selected model capability and runs it.

US-04: Create a Next Step

As a creator, I drag from an image or video output to blank canvas, choose the next node type, then edit and run the newly created node from the bottom composer.

US-05: Avoid Accidental Branches

As a creator, if I select an old node and click generate, I expect the old node to rerun. I do not expect a hidden downstream branch to appear.

7.1 Composer Context and Draft Lifecycle

7.1.1 Context Source

ComposerContext is always derived from the workflow store. It is not manually stored as independent component state.

Inputs that can change the context:

• selected node ids
• selected node type
• selected node model capability
• selected node deletion
• node type support in the adapter registry

Inputs that update visible status but must not reset the draft:

• selected node status
• selected node error
• selected node outputImage / outputVideo
• execution start, completion, or failure

Model catalog async loading does not change the selected node's context by itself. If a stored selectedModel lacks capabilities, the adapter's node type determines the capability until the user selects a new model.

7.1.2 Draft Source

Composer draft is initialized from the selected node through the node adapter. The draft is reinitialized only when the composer identity changes:

• mode changes
• selected node id changes
• selected node type changes
• empty-create draft is explicitly reset after creating a node

Status and output updates must not wipe unsaved prompt or parameter edits.

7.1.3 Draft Sync Decision

The product decision is autosave.

Autosave rules:

• On blur, write dirty draft fields back to the selected node.
• Before generate, write dirty draft fields back to the selected node, then call regenerateNode.
• Before switching from selected node A to selected node B, write dirty draft fields back to node A, then load node B.
• If node A is deleted before autosave can apply, discard the draft because the target no longer exists.
• No confirmation dialog is required in MVP.

This avoids hidden data loss without interrupting canvas exploration.

7.1.4 Composer State Transition Triggers

Trigger	Context Update	Draft Update	Notes
Select no nodes	empty-create	keep existing empty draft unless previous target was a node	Creation draft is independent
Select one supported node	node-edit	autosave previous dirty node draft, then read selected node	Adapter decides fields
Select one supported processing node	process-node	autosave previous dirty node draft, then clear editable generation draft	Inspector reads directly from node data and connected inputs
Select one unsupported node	unsupported-node	autosave previous dirty node draft, then clear editable draft	Show guidance only
Select multiple nodes	multi-select	autosave previous dirty node draft, then clear editable draft	Batch edit disabled in MVP
Node data changes externally	same context	update non-dirty fields only	Prevent status refresh from wiping edits
Model selected in dialog	same context	update draft model and selected node model in edit mode	Capability must match current context
Generate clicked	same context	autosave draft before execution	Then regenerateNode
Generation starts	same context	no draft reset	Show running state
Generation succeeds	same context	no draft reset	Show output/status
Generation fails	same context	no draft reset	Show node error
Cancel/escape composer draft	same context	reload from selected node	Optional MVP action

8. Composer States

State	Trigger	Composer Target	Model Filter	Main Button
Empty create	no selected nodes	new root node draft	all	Create and generate
Image node edit	exactly one nanoBanana node selected	selected node	image	Rerun selected image node
Video node edit	exactly one generateVideo node selected	selected node	video	Rerun selected video node
Video stitch inspector	exactly one videoStitch node selected	selected node + connected videos	none	Stitch selected processing node when at least 2 clips are connected
Ease curve inspector	exactly one easeCurve node selected	selected node + connected video/settings	none	Apply selected processing node when a video is connected
3D node edit	exactly one generate3d node selected	selected node	3d	Rerun selected 3D node, later phase
Audio node edit	exactly one generateAudio node selected	selected node	audio	Rerun selected audio node, later phase
Unsupported node	one non-generation node selected	selected node info	none	Disabled
Multi-select	multiple nodes selected	selection summary	none or batch	Disabled in MVP

9. Core Interaction Rules

9.1 No Selection

Default behavior:

1. Composer shows a creation draft.
2. Model browser opens with initialCapabilityFilter = "all".
3. User selects an image/video/3D/audio model.
4. Composer infers node type from selected model capability.
5. User clicks generate.
6. App creates a node at the viewport center.
7. App selects the new node.
8. App runs regenerateNode(newNodeId).

Model to node mapping:

Model Capability	Node Type
text-to-image, image-to-image	nanoBanana
text-to-video, image-to-video, audio-to-video	generateVideo
text-to-3d, image-to-3d	generate3d
text-to-audio	generateAudio

9.2 Selected Image Generation Node

Default behavior:

1. User selects a nanoBanana node.
2. Composer reads:
   • inputPrompt
   • selectedModel
   • aspectRatio
   • resolution
   • parameters
   • status
   • error
3. Model browser opens with initialCapabilityFilter = "image".
4. Edits write to the selected node, either immediately with debounce or on generate.
5. Main button calls regenerateNode(selectedNode.id).

The button label should be "生成" or "重新生成", but the tooltip must clarify: "重新生成当前图片节点".

9.3 Selected Video Generation Node

Default behavior:

1. User selects a generateVideo node.
2. Composer reads:
   • inputPrompt
   • selectedModel
   • parameters
   • status
   • error
   • connected image/text/audio references when available
3. Model browser opens with initialCapabilityFilter = "video".
4. Video controls render from known fields and model schema.
5. Main button calls regenerateNode(selectedNode.id).

The button tooltip must clarify: "重新生成当前视频节点".

9.4 Selected Video Stitch Processing Node

Default behavior:

1. User selects a videoStitch node.
2. Composer switches to process-node, not node-edit.
3. Composer displays:
   • connected video clip count
   • audio input presence
   • clip order summary from clipOrder / connected video edges
   • loopCount controls
   • output readiness
   • status and error
4. Composer hides prompt, model selector, reference image upload, and generation-only tools.
5. Main button calls regenerateNode(selectedNode.id) only when at least two connected videos resolve through getConnectedInputs.

The button tooltip must clarify: "拼接当前视频节点".

9.5 Selected Ease Curve Processing Node

Default behavior:

1. User selects an easeCurve node.
2. Composer switches to process-node, not node-edit.
3. Composer displays:
   • connected video input state
   • inherited vs local curve settings
   • current preset or custom bezier handles
   • outputDuration control
   • output readiness
   • status and error
4. Composer hides prompt, model selector, reference image upload, and generation-only tools.
5. Main button calls regenerateNode(selectedNode.id) only when at least one connected video resolves through getConnectedInputs.

The button tooltip must clarify: "应用当前缓动曲线节点".

9.6 Unsupported Node Selection

Examples:

• imageInput
• videoInput
• annotation
• prompt
• output
• outputGallery

Behavior:

• Composer must not imply that generate will create the next step.
• Composer may show compact context:
  • node type
  • filename / media preview if available
  • "从节点端口拖拽以创建下一步"
• Main generate button is disabled unless the node has an explicit supported processing inspector.

9.7 Multi-Selection

MVP behavior:

• Show selected count.
• Show a compact empty state: "Selected {count} nodes. Batch editing is not available in this version."
• Hide single-node parameter editing.
• Main generate button disabled.

Future behavior:

• Batch rerun selected supported generation nodes.
• Batch parameter edit only when all selected nodes share the same node type.

10. Next-Step Workflow

Next-step generation is a canvas operation, not a composer operation.

Preferred flow:

flowchart LR
  A["User drags from source node output"] --> B["ConnectionDropMenu opens"]
  B --> C["User selects Generate Image / Generate Video"]
  C --> D["Canvas creates downstream node"]
  D --> E["Canvas connects edge"]
  E --> F["Canvas selects new node"]
  F --> G["Bottom composer edits new node"]
  G --> H["Generate button reruns selected new node"]

Engineering requirement:

• After ConnectionDropMenu creates a new generation node, the app should select the new node automatically so the composer immediately targets it.
• If the new node is not auto-selected, the user must manually click it, which breaks the intended flow.
• Selection ownership is centralized in the workflow store. React Flow selection is treated as a UI mirror of nodes[].selected, not an independent source of truth.
• Typed processing handles must not fall back to generic media targets. Dragging from an easeCurve Settings output should offer Settings-compatible targets such as easeCurve or router, not videoInput.

11. Parameter Display

11.1 Image Controls

Required MVP controls:

• model selector
• prompt textarea
• aspect ratio
• resolution
• count display or selector
• reference thumbnail strip
• status/error indicator
• generate button

Current fields:

• inputPrompt
• selectedModel
• aspectRatio
• resolution
• inputImages
• parameters

11.2 Video Controls

Required MVP controls:

• model selector
• prompt textarea
• reference thumbnail strip
• generated video status/error
• generate button
• schema-driven advanced parameters when available

Semantic video controls are first-class product fields, even if the initial implementation stores them temporarily under parameters during migration.

First-class video fields to add before exposing fixed controls:

• aspect ratio
• resolution
• duration seconds
• fps
• sound enabled

Provider-specific controls remain schema-driven:

• camera/motion preset
• motion strength
• seed variants that do not map cleanly across providers
• provider-only quality or mode flags

Important constraint:

The current GenerateVideoNodeData does not define first-class fields such as duration, fps, or resolution. The decided direction is to add semantic first-class fields and let executors map them to provider payload names. Fixed video controls must not ship until they write to execution data.

11.3 Advanced Parameters

The composer should not hardcode every provider-specific field.

Use this precedence:

1. Known stable first-class fields, such as image aspectRatio and resolution.
2. Model schema via inputSchema.
3. Raw parameters editor or compact settings popover.
4. UI-only disabled placeholders for future capabilities.

12. Data Ownership

12.1 Prompt

Generation nodes own their prompt:

• image: NanoBananaNodeData.inputPrompt
• video: GenerateVideoNodeData.inputPrompt

Prompt nodes and connected text inputs still remain valid workflow inputs. Connected upstream text has priority over the node-local prompt.

Prompt precedence:

Node-local prompt	Upstream text connection	Execution prompt	Composer behavior
empty	none	missing prompt	editable, generate disabled
empty	present	upstream text	read-only display of upstream text
present	none	node-local prompt	editable
present	present	upstream text	read-only display of upstream text, show local prompt as fallback/overridden

When upstream text is connected, the composer prompt field must be disabled or visually read-only with a hint: "Prompt is provided by an upstream text node." This keeps the graph as the relationship source of truth.

12.2 References

The graph should remain the source of truth for relationships.

Composer should display references from:

• connected upstream media/text/audio nodes when possible
• selected generation node cached fields such as inputImages as fallback

Composer should not silently create new reference edges except in no-selection root creation, where there are no selected graph relationships.

MVP direct upload boundary:

• Image uploads in the composer may become node-local inputImages for no-selection root creation, or fallback references on a selected generation node when no upstream image connection exists.
• Upstream image connections remain authoritative. If a selected node has connected image inputs, composer uploads must not override those graph relationships.
• Video file uploads are not stored directly on generation nodes in this PRD. Until GenerationInput and model input schemas support video inputs, uploaded videos remain explicit videoInput nodes connected through the canvas.

12.3 Model

Selected generation nodes own their model:

• selectedModel.provider
• selectedModel.modelId
• selectedModel.displayName
• selectedModel.capabilities
• selectedModel.pricing

For image nodes, the legacy model field must still be maintained for backward compatibility.

Model switching rules:

• Same capability switch is allowed.
• Cross-capability switch is blocked in node-edit mode. For example, an image node cannot become a video node through the composer model browser.
• Empty-create mode allows any capability because there is no existing node data structure.
• Same capability switch preserves common fields such as prompt and references.
• Same capability switch clears provider-specific parameters, fallbackParameters, and inputSchema.
• Image node switches must continue to update the legacy model field when applicable.

13. Generate Button Contract

13.1 Enabled Conditions

State	Enabled When
Empty create	prompt is non-empty and selected model has a supported capability
Image node edit	selected node exists, prompt is non-empty, model is an image model, not running
Video node edit	selected node exists, prompt is non-empty, model is a video model, not running
Unsupported node	never
Multi-select	never in MVP

13.2 Click Behavior

No selected node:

1. Infer node type from selected model capability.
2. Build initial node data from composer draft.
3. addNode(nodeType, viewportCenter, initialData).
4. Select the newly created node.
5. regenerateNode(newNodeId).

Selected supported generation node:

1. Build a node patch from composer state.
2. updateNodeData(selectedNode.id, patch).
3. regenerateNode(selectedNode.id).

Selected unsupported node:

1. No-op.
2. Button disabled.

Multiple selected nodes:

1. No-op in MVP.
2. Button disabled.

13.3 Running State

While isRunning is true:

• composer controls remain visible
• generate button shows spinner
• generate button is disabled
• status text should identify the current running node when possible

13.4 Error Handling

Errors are stored on the node, not in composer-only state:

• image: NanoBananaNodeData.error
• video: GenerateVideoNodeData.error
• status: NodeStatus = "idle" | "loading" | "complete" | "error" | "skipped"

Composer behavior:

• If the selected node has status = "error", show the error message above the footer controls.
• The generate button remains enabled when the prompt/model requirements are valid and isRunning is false.
• Button tooltip changes to "Retry current node" when selected node status is error.
• Clearing or changing prompt/model/parameters does not clear the error immediately; the error is cleared when execution starts through the existing executor update.

Expected user-facing error categories:

• missing API key or provider configuration
• invalid model selection
• provider timeout or polling failure
• rate limit or quota
• safety/policy rejection
• invalid provider-specific parameters

14. Technical Implementation Plan

Phase 1: Composer Context Derivation

Create a small context derivation layer in GenerationComposer.tsx or a sibling utility.

Suggested types:

type ComposerMode =
  | "empty-create"
  | "node-edit"
  | "process-node"
  | "unsupported-node"
  | "multi-select";

type ComposerCapability = "all" | "image" | "video" | "3d" | "audio";

interface ComposerContext {
  mode: ComposerMode;
  node: WorkflowNode | null;
  capability: ComposerCapability;
  nodeType: NodeType | null;
}

Acceptance:

• no selection resolves to empty-create + all
• nanoBanana resolves to node-edit + image
• generateVideo resolves to node-edit + video
• videoStitch resolves to process-node + video
• easeCurve resolves to process-node + video
• unsupported single selection resolves to unsupported-node
• multiple selection resolves to multi-select
• status/output/error updates do not reset dirty draft fields

Phase 2: Adapter-Based Node Read/Write

Avoid adding one-off conditionals throughout the composer.

Suggested shape:

interface ComposerAdapter<TData> {
  nodeType: NodeType;
  capability: Exclude<ComposerCapability, "all">;
  defaultModel: () => SelectedModel;
  readDraft: (node: WorkflowNode) => ComposerDraft;
  buildPatch: (draft: ComposerDraft) => Partial<TData>;
  getModelDialogTitle: () => string;
}

MVP adapters:

• imageComposerAdapter for nanoBanana
• videoComposerAdapter for generateVideo

Adapter registration:

const COMPOSER_ADAPTERS = {
  nanoBanana: imageComposerAdapter,
  generateVideo: videoComposerAdapter,
} satisfies Partial<Record<NodeType, ComposerAdapter<WorkflowNodeData>>>;

Do not place adapters in Zustand. They are deterministic UI/domain mapping and should stay as testable module-level code.

Acceptance:

• selected image node loads existing prompt/model/ratio/resolution
• selected video node loads existing prompt/model/parameters
• switching between selected nodes updates composer values correctly
• adding a future 3D/audio adapter requires registering one adapter entry

Phase 3: Model Selection Rules

Update composer model browser usage:

• empty-create: initialCapabilityFilter = "all"
• image node: initialCapabilityFilter = "image"
• video node: initialCapabilityFilter = "video"
• 3D/audio later: use existing filters

For composer usage, ModelSearchDialog should receive onModelSelected so it selects a model but does not immediately create a node.

Acceptance:

• selected image node only shows image-capable models by default
• selected video node only shows video-capable models by default
• no-selection state can browse all models
• selecting a model updates composer state and selected node data when editing
• selecting a cross-capability model is impossible in node-edit mode because the dialog is filtered by current capability
• same-capability model changes clear provider-specific parameters and schema

Phase 4: Generate Button Refactor

Refactor handleSubmit so it branches by ComposerContext, not by selected image references.

Acceptance:

• no selection + image model creates nanoBanana
• no selection + video model creates generateVideo
• selected image node reruns itself
• selected video node reruns itself
• selected input/media/output node does not create hidden downstream nodes

Phase 5: Canvas Next-Step Auto-Select

When ConnectionDropMenu creates a new generation node:

• connect edge as today
• select the new node
• clear prior selection

Implementation options:

• add selectSingleNode(nodeId) to workflowStore

Decision:

• add selectSingleNode(nodeId) and clearSelection() to workflowStore
• use those actions after ConnectionDropMenu creates a node
• treat React Flow's selection callbacks as sync events into the store, not as the composer source of truth

Acceptance:

• drag image output to blank canvas, choose Generate Video, new video node is selected
• composer immediately switches to video editing mode
• generate button reruns the new video node

Phase 6: Video Parameter UI

Add video controls only for parameters already available in schema or node data.

MVP:

• prompt
• model
• reference previews
• raw/advanced parameter popover if schema exists
• status/error

Future:

• first-class GenerateVideoNodeData fields for semantic controls
• duration
• resolution
• aspect ratio
• fps
• audio
• seed

Acceptance:

• video node can select NewAPI video models
• video node can rerun through regenerateNode
• UI does not show fake controls that do not write to execution data

Phase 7: Test and Visual Verification

Add focused tests instead of broad snapshots.

Targets:

• composer context derivation
• model filter behavior
• selected image node rerun
• selected video node rerun
• unsupported selection disables generation
• connection drop creation selects new generation node

Manual validation:

• open local app
• select image node
• select video node
• drag image output to Generate Video
• verify bottom composer mode changes
• verify no hidden downstream branch is created by the main button

15. Acceptance Criteria

ID	Requirement	Acceptance
AC-01	Composer has one target	Every state resolves to exactly one composer mode
AC-02	Selected image node edit	Selecting nanoBanana loads its prompt/model/ratio/resolution
AC-03	Selected image node run	Main button calls regenerateNode on the selected image node
AC-04	Selected video node edit	Selecting generateVideo loads its prompt/model/parameters
AC-05	Selected video node run	Main button calls regenerateNode on the selected video node
AC-06	No hidden next step	With a node selected, main button never calls addNode
AC-07	No selection creation	With no selection, main button creates a new node based on model capability
AC-08	Model filters	Image nodes open image models, video nodes open video models, empty state opens all models
AC-09	Unsupported node state	Selecting imageInput, prompt, output, etc. disables generate
AC-10	Canvas next step	Drag/drop creation selects the new node so composer edits it
AC-11	NewAPI compatibility	NewAPI model selection writes selectedModel.provider = "newapiwg" and uses existing executors
AC-12	Tests	Component/store tests cover core state transitions and generate button behavior
AC-13	Draft safety	Dirty draft fields are autosaved on selection change, blur, and before generate
AC-14	Prompt precedence	Connected upstream text overrides node-local prompt and makes composer prompt read-only
AC-15	Model switching	Node-edit mode prevents cross-capability model selection
AC-16	Error display	Selected node error appears in composer and retry reruns the same node
AC-17	NewApiWG gateway payloads	Video generation sends gateway-compatible images, videos, and audios fields, and model-specific exceptions are handled in the provider adapter
AC-18	Video stitch inspector	Selecting videoStitch shows clip/input/loop/status controls and reruns the same node without requiring a prompt
AC-19	Ease curve inspector	Selecting easeCurve shows video/settings/duration/status controls and reruns the same node without requiring a prompt
AC-20	Settings handle menu	Dragging from an easeCurve Settings handle shows only Settings-compatible connection targets and sources

16. UX Copy

Suggested mode labels:

• Empty create: "新建生成"
• Image node edit: "图片节点"
• Video node edit: "视频节点"
• Video stitch inspector: "视频拼接"
• Ease curve inspector: "缓动曲线"
• Unsupported node: "当前节点"
• Multi-select: "已选择 {count} 个节点"

Suggested button tooltips:

• Empty create: "创建并生成"
• Image edit: "重新生成当前图片节点"
• Video edit: "重新生成当前视频节点"
• Video stitch: "拼接当前视频节点"
• Ease curve: "应用当前缓动曲线节点"
• Unsupported: "从节点端口拖拽以创建下一步"
• Multi-select: "多选暂不支持单节点生成"

Suggested unsupported hint:

下一步需要从节点端口拖拽创建，底部框只编辑生成节点或显示受支持处理节点的检查器。

17. Risks and Constraints

R-01: Video Fields Are Not First-Class Yet

GenerateVideoNodeData currently stores provider-specific controls mostly under parameters. Fixed duration/fps/resolution controls require a data-model migration before they can become reliable UI.

Mitigation:

• MVP uses prompt/model/references/status plus schema-driven controls.
• Add first-class semantic fields before exposing fixed video controls.
• Executors map semantic fields to provider payload names.

R-02: Composer and ControlPanel Duplicate Responsibilities

The existing ControlPanel already edits generation nodes. The composer should become the primary surface for common generation parameters, while ControlPanel can remain advanced/legacy during transition.

Mitigation:

• Do not delete ControlPanel in MVP.
• Move only high-frequency controls first.

R-03: Selection State Must Not Split

addNode does not appear to select newly created nodes by default.

Mitigation:

• Add explicit store selection helpers.
• Use workflow store as the composer source of truth.
• Test connection drop flows.

R-04: Count UI May Exceed Executor Semantics

The composer prototype shows image count. Existing executor behavior may not support native multi-count for every provider.

Mitigation:

• MVP may keep count as UI for image only if wired.
• Otherwise default to 1 and mark multi-count as future.

R-05: NewApiWG Video Models Do Not Share One Media Schema

NewApiWG exposes several upstream video families through one gateway. Some models accept top-level images, videos, and audios, while others require provider-native fields under metadata.

Mitigation:

• Composer emits semantic node fields and connected graph inputs only.
• The NewApiWG provider adapter normalizes common aliases into gateway fields.
• Model-family exceptions, such as Kling metadata media fields or Jimeng base64 constraints, must stay in the provider adapter or model schema layer.
• Do not expose provider-specific composer controls until they write to a tested execution payload.

18. Metrics

Product success metrics:

• A user can select an image node, edit prompt/model, and rerun it without opening the side control panel.
• A user can select a video node, select a video model, and rerun it from the composer.
• A user can create image-to-video by dragging from image output to blank canvas, choosing Generate Video, and running the new selected node.
• No user test participant interprets the selected-node generate button as "create next step".

Engineering success metrics:

• Composer behavior is covered by unit/component tests.
• No direct provider calls are added to composer UI.
• Existing regenerateNode execution path remains the single run path.
• Existing NewAPI image and video tests keep passing.

19. Final Product Decision

The bottom composer should be implemented as a current-node console.

It is acceptable for the composer to create a new root node only when there is no current node selection. Once a node is selected, the composer must only edit and run that selected node. All next-step workflow creation must remain visible on the canvas through dragging, node plus actions, and explicit connections.

20. Related Decision Records

• docs/decisions/0001-composer-draft-sync.md
• docs/decisions/0002-prompt-source-precedence.md
• docs/decisions/0003-video-parameter-ownership.md
• docs/decisions/0004-model-switching-boundaries.md
• docs/decisions/0005-selection-source-of-truth.md

21. Appendix: NewApiWG Gateway Documentation Summary

21.1 Product Boundary

NewApiWG is the unified provider gateway for model discovery and multimodal generation. In this PRD it is a provider implementation behind the existing app APIs, not a separate composer UX.

The intended call path is:

flowchart LR
  UI["Composer / Node UI"] --> E["regenerateNode(nodeId)"]
  E --> A["/api/generate"]
  A --> P["NewApiWG provider adapter"]
  P --> G["NewApiWG gateway"]

The composer must not build gateway-specific payloads directly. It writes selected model, prompt, references, semantic video fields, and provider parameters into the selected node. Executors and the provider adapter own request construction.

21.2 Configuration and Local API Surface

Configuration:

• Environment: NEWAPIWG_API_KEY
• Environment: NEWAPIWG_BASE_URL
• Project settings override: provider newapiwg.apiKey and newapiwg.baseUrl
• Request headers used internally: X-NewApiWG-Key, X-NewApiWG-Base-URL

Local routes:

App Route	Gateway Route	Purpose
GET /api/models?provider=newapiwg	GET {baseUrl}/models	Fetch gateway model catalog and infer app capabilities
POST /api/generate with image capability	POST {baseUrl}/images/generations, POST {baseUrl}/chat/completions, or Gemini-native image route	Generate or edit images depending on model family
POST /api/generate with video capability	POST {baseUrl}/video/generations	Submit async video generation task
provider polling	GET {baseUrl}/video/generations/{task_id}	Poll video task until success/failure/timeout
POST /api/generate with audio capability	POST {baseUrl}/audio/speech	Generate speech/audio
media playback / frame extraction	/api/proxy-media?url=...	Avoid remote video CORS/range failures in browser media pipelines

21.3 Model Discovery and Capability Inference

NewApiWG model objects may expose capabilities explicitly through capabilities, capability, tags, modalities, or metadata fields. If the gateway does not provide explicit capability tags, the app infers capabilities from model id/name/description.

Supported app capabilities:

• text-to-image
• image-to-image
• text-to-video
• image-to-video
• audio-to-video
• text-to-3d
• image-to-3d
• text-to-audio

Known inference keywords:

• image: image, img, vision, flux, sdxl, gpt-image, imagen, seedream, banana
• video: video, vidu, veo, sora, kling, hailuo, seedance, dreamactor, motion
• audio: audio, speech, voice, tts, music
• 3D: 3d, mesh, tripo, hunyuan3d

Chat-only models should not appear in image/video composer filters unless the gateway marks them with a compatible generation capability.

Default app models when NewApiWG is preferred:

Capability	Default Model
Image	apiyi_nano_banana_2 / APIYI Nano Banana 2
Video	viduq2-pro-fast / Vidu Q2 Pro Fast

21.4 Common Generation Payload Contract

Shared fields:

• model: gateway model id
• prompt: node-local prompt or upstream text prompt, with upstream graph text taking precedence
• parameters: provider-specific fields that do not have first-class node fields
• dynamicInputs: schema-derived inputs

Image generation:

• Standard route uses POST /images/generations.
• Image references use OpenAI-compatible image for image/edit models.
• Gemini-native image models route through the Gemini-compatible generation endpoint.
• Chat image models such as gpt-image-2-all route through /chat/completions; URL-only responses are accepted and may be converted to data URLs when fetchable.

Video generation:

• Standard route uses POST /video/generations.
• Common gateway media fields:
  • images: string[]
  • videos: string[]
  • audios: string[]
• Stale or provider-specific aliases should be normalized by the NewApiWG provider adapter. Examples:
  • image, image_url, first_frame_url, last_frame_url, reference_image_urls -> images
  • video, video_url, reference_video_urls -> videos
  • audio, audio_url, reference_audio_urls -> audios
• parameters may include fields such as duration, seconds, size, ratio, metadata, seed, frames, generate_audio, or model-family options.
• Connected graph inputs are authoritative. Composer uploads are only fallback references when no upstream media connection exists.

21.5 Video Model Family Summary

Family	Gateway Model Examples	Input Rules	Product Notes
Sora / VEO-like OpenAI video	sora-2, veo-3.1-landscape-fl	Text prompt plus optional or required reference image depending on model	Some modes use multipart input_reference; do not expose single-file controls until provider adapter supports them
Hailuo / MiniMax	MiniMax-Hailuo-2.3, I2V-01, S2V-01	Gateway prefers images; 0 image = text-to-video, 1 image = image-to-video, 2 images = first/tail for supported models	metadata.action can force task type; model-specific duration/resolution validation belongs in adapter/schema
Doubao / Seedance	doubao-seedance-2-0-260128, doubao-seedance-2-0-fast-260128	Top-level images, videos, audios; 0 image = text-to-video, 1 = first frame, 2 = first/tail, 3+ = references	Seedance 2.0 supports video/audio references; expected runtime can be several minutes
Jimeng	jimeng_t2v_v30, jimeng_i2v_first_v30, jimeng_dreamactor_m20_gen_video	Top-level images / videos; some base64 inputs require pure base64 without data:image/... prefix	Base64 normalization must be handled below the composer
Kling	kling-v2-6, kling-v3, kling-video-o1	Media should be sent in Kling-native metadata fields such as metadata.image, metadata.image_tail, metadata.image_list, metadata.video_list	This is a schema exception. Generic top-level images is not enough for all Kling modes
Vidu	viduq3, viduq3-mix, viduq2-pro-fast, vidu2.0	Top-level images; 0 = text-to-video, 1 = image-to-video, 2 = first/tail, 3+ = reference video; metadata.subjects for subject reference	Current default video model is Vidu Q2 Pro Fast

21.6 Response and Polling Contract

Video submission response may include:

• id or task_id
• status
• created_at
• immediate url, video_url, result_url, or nested metadata.url

Polling response may include:

• status: pending/queued/processing/in_progress/completed/SUCCESS/failed/FAILURE
• result_url
• metadata.url
• metadata.last_frame_url

Current app behavior:

• Polls NewApiWG video tasks through the provider adapter.
• Treats returned remote video URLs as valid outputs.
• Uses /api/proxy-media for browser playback and frame extraction when remote media would otherwise fail CORS or range requests.

Gateway timing guidance:

• Video generation is async.
• First poll should not be immediate for slow models.
• Seedance can take around 6 minutes; other families are often around 1 to 2 minutes, depending on mode and load.
• The app should keep timeout and retry behavior in provider/executor code, not in composer UI.

21.7 Error Handling Contract

Errors should be stored on the node and surfaced by the composer.

Expected NewApiWG error categories:

• missing NEWAPIWG_API_KEY or project key
• invalid model id or missing selected model metadata
• missing prompt or required media
• invalid model-specific duration/resolution/aspect ratio
• quota or insufficient credits
• upstream task failure
• polling timeout
• transient gateway/provider overload

Current adapter behavior includes transient retry normalization for system disk overloaded. User-facing copy should recommend retrying or switching models instead of exposing raw provider infrastructure text.

21.8 PRD Consequences

• NewApiWG is the recommended unified gateway when configured.
• The model browser should prefer NewApiWG over Gemini when a NewApiWG key exists.
• Composer mode and filters remain capability-driven, not provider-driven.
• Provider-specific media schema exceptions must be implemented in src/app/api/generate/providers/newapiwg.ts or model schema extraction, not in GenerationComposer.
• Tests must cover gateway field normalization for connected media, including images, videos, audios, and stale aliases such as first_frame_url.