Browse Source

优化视频节点首尾帧生成流程与注释

xuzhijie-popiart-node-canvas
xuzhijie 3 months ago
parent
commit
5f7b2a0dc7
  1. 60
      popiart-node-interaction-enhancement-implementation.md
  2. 4
      src/components/WorkflowCanvas.tsx
  3. 147
      src/components/__tests__/VideoInputNode.test.tsx
  4. 106
      src/components/nodes/VideoInputNode.tsx

60
popiart-node-interaction-enhancement-implementation.md

@ -1984,3 +1984,63 @@ npm.cmd run test:run -- ImageInputNode.test.tsx WorkflowCanvas.test.tsx ImageNod
``` ```
结果:4 个测试文件通过,95 个测试通过。测试日志里仍有已有的 React `act(...)` 警告、jsdom canvas/getContext 提示和测试环境相对 URL 提示,不影响本次功能通过。 结果:4 个测试文件通过,95 个测试通过。测试日志里仍有已有的 React `act(...)` 警告、jsdom canvas/getContext 提示和测试环境相对 URL 提示,不影响本次功能通过。
## 2026-04-30 更新:视频上传节点帧生成快捷入口
状态:已完成,可进入手动测试。
本次实现把参考图里的视频入口能力放到空 `videoInput` 节点中:
1. 空 `videoInput` 节点中间显示 `尝试:`、`首尾帧生成视频`、`首帧生成视频`。
2. 点击 `首帧生成视频` 会打开图片选择器,选择一张首帧图片。
3. 点击 `首尾帧生成视频` 会打开图片选择器,选择两张首尾帧图片。
4. 上传帧图片后自动在右侧创建 `generateVideo` 节点。
5. 上传的帧图片会写入新 `generateVideo` 节点的 `inputImages`,后续生成视频时可作为帧素材使用。
6. 原有上方 `Upload` 上传真实视频文件的能力保持不变。
涉及文件:
- `src/components/nodes/VideoInputNode.tsx`
- `src/components/__tests__/VideoInputNode.test.tsx`
验证:
```powershell
npm.cmd run test:run -- VideoInputNode.test.tsx WorkflowCanvas.test.tsx ImageNodeComposerPanel.test.tsx VideoNodeUploadOverlay.test.tsx GenerateVideoNode.test.tsx
```
结果:5 个测试文件通过,87 个测试通过。测试日志里仍有已有的 React `act(...)` 警告、jsdom 视频 `pause()` 未实现提示和测试环境相对 URL 提示,不影响本次功能通过。
## 2026-04-30 更新:首尾帧生成视频节点自动建链修正
本次把“首尾帧生成视频”从“只创建一个隐藏输入的 Generate Video 节点”修正为可见节点链路:
1. 点击 `首帧生成视频` 后,只允许选择视频文件;当前 `videoInput` 节点会变成 `首帧` 视频源节点,并在右侧创建 `generateVideo` 节点。
2. 点击 `首尾帧生成视频` 后,只允许选择两个视频文件;当前节点作为 `首帧`,自动在下方创建 `尾帧` 视频源节点,并在右侧居中创建 `generateVideo` 节点。
3. 自动创建 `video -> video` 连线:`首帧` 连接到生成视频节点;如果有 `尾帧`,也自动连接到同一个生成视频节点。
4. `WorkflowCanvas` 的浮动标题支持读取节点 `label`,所以自动生成的源节点会显示 `首帧` / `尾帧`
涉及文件:
- `src/components/nodes/VideoInputNode.tsx`
- `src/components/WorkflowCanvas.tsx`
- `src/components/__tests__/VideoInputNode.test.tsx`
验证:
```powershell
npm.cmd run test:run -- VideoInputNode.test.tsx WorkflowCanvas.test.tsx ImageNodeComposerPanel.test.tsx VideoNodeUploadOverlay.test.tsx GenerateVideoNode.test.tsx
```
结果:5 个测试文件、87 个测试通过。测试日志里仍有既有的 React `act(...)` 和 jsdom `HTMLMediaElement.pause()` 警告,不影响本次功能断言。
## 2026-04-30 更新:首尾帧视频入口改为纯建链
本次根据反馈移除了“首尾帧生成视频”点击后的即时文件选择:
1. 点击 `首帧生成视频` 不再弹上传框,只把当前视频上传节点标记为 `首帧`,并在右侧创建/连接 `Generate Video` 节点。
2. 点击 `首尾帧生成视频` 不再要求立刻选择两个视频,只创建一个空的 `尾帧` 视频上传节点,并把 `首帧`、`尾帧` 都连接到右侧 `Generate Video`
3. 视频文件由用户后续分别在 `首帧` / `尾帧` 视频节点上自行上传。
验证:
```powershell
npm.cmd run test:run -- VideoInputNode.test.tsx WorkflowCanvas.test.tsx ImageNodeComposerPanel.test.tsx VideoNodeUploadOverlay.test.tsx GenerateVideoNode.test.tsx
```
结果:5 个测试文件、87 个测试通过。

4
src/components/WorkflowCanvas.tsx

@ -561,6 +561,10 @@ export function WorkflowCanvas() {
// Helper to get node title (used for FloatingNodeHeader) // Helper to get node title (used for FloatingNodeHeader)
const getNodeTitle = useCallback((node: Node) => { const getNodeTitle = useCallback((node: Node) => {
// 自动创建的首帧/尾帧等素材节点优先使用业务标签作为浮动标题。
const label = (node.data as any)?.label;
if (typeof label === "string" && label.trim()) return label;
// For generate nodes, check for selectedModel display name // For generate nodes, check for selectedModel display name
if (node.type === "nanoBanana" || node.type === "generateVideo" || node.type === "generate3d" || node.type === "generateAudio") { if (node.type === "nanoBanana" || node.type === "generateVideo" || node.type === "generate3d" || node.type === "generateAudio") {
const model = (node.data as any)?.selectedModel; const model = (node.data as any)?.selectedModel;

147
src/components/__tests__/VideoInputNode.test.tsx

@ -0,0 +1,147 @@
import { describe, it, expect, vi, beforeEach } from "vitest";
import { fireEvent, render, screen } from "@testing-library/react";
import { ReactFlowProvider } from "@xyflow/react";
import { VideoInputNode } from "@/components/nodes/VideoInputNode";
const TRY_LABEL = "\u5c1d\u8bd5:";
const KEYFRAMES_LABEL = "\u9996\u5c3e\u5e27\u751f\u6210\u89c6\u9891";
const FIRST_FRAME_LABEL = "\u9996\u5e27\u751f\u6210\u89c6\u9891";
const FIRST_FRAME_NODE_LABEL = "\u9996\u5e27";
const LAST_FRAME_NODE_LABEL = "\u5c3e\u5e27";
const mockUpdateNodeData = vi.fn();
let addNodeSequence = 0;
const mockAddNode = vi.fn((type: string) => `${type}-${++addNodeSequence}`);
const mockOnConnect = vi.fn();
vi.mock("@/store/workflowStore", () => ({
useWorkflowStore: vi.fn((selector) => selector({
updateNodeData: mockUpdateNodeData,
addNode: mockAddNode,
onConnect: mockOnConnect,
currentNodeIds: [],
groups: {},
nodes: [
{
id: "video-1",
type: "videoInput",
position: { x: 120, y: 160 },
measured: { width: 300, height: 280 },
style: { width: 300, height: 280 },
data: {},
},
],
getNodesWithComments: vi.fn(() => []),
markCommentViewed: vi.fn(),
setNavigationTarget: vi.fn(),
})),
}));
vi.mock("@/utils/videoFile", () => ({
readVideoFile: vi.fn((file: File) => Promise.resolve({
video: `data:video/mp4;base64,${file.name}`,
filename: file.name,
format: file.type,
duration: 3,
dimensions: { width: 1280, height: 720 },
})),
}));
function TestWrapper({ children }: { children: React.ReactNode }) {
return <ReactFlowProvider>{children}</ReactFlowProvider>;
}
describe("VideoInputNode", () => {
const defaultProps = {
id: "video-1",
type: "videoInput" as const,
data: {
video: null,
videoRef: undefined,
filename: null,
duration: null,
dimensions: null,
format: null,
},
selected: false,
isConnectable: true,
positionAbsoluteX: 0,
positionAbsoluteY: 0,
zIndex: 0,
dragging: false,
deletable: true,
selectable: true,
parentId: undefined,
dragHandle: undefined,
};
beforeEach(() => {
addNodeSequence = 0;
vi.clearAllMocks();
});
it("renders video generation shortcuts when empty", () => {
render(
<TestWrapper>
<VideoInputNode {...defaultProps} />
</TestWrapper>
);
expect(screen.getByText(TRY_LABEL)).toBeInTheDocument();
expect(screen.getByRole("button", { name: KEYFRAMES_LABEL })).toBeInTheDocument();
expect(screen.getByRole("button", { name: FIRST_FRAME_LABEL })).toBeInTheDocument();
});
it("labels the current node as first frame and links it to a generate video node", () => {
render(
<TestWrapper>
<VideoInputNode {...defaultProps} />
</TestWrapper>
);
fireEvent.click(screen.getByRole("button", { name: FIRST_FRAME_LABEL }));
expect(mockUpdateNodeData).toHaveBeenCalledWith("video-1", {
label: FIRST_FRAME_NODE_LABEL,
});
expect(mockAddNode).toHaveBeenCalledWith("generateVideo", { x: 600, y: 160 });
expect(mockOnConnect).toHaveBeenCalledWith({
source: "video-1",
sourceHandle: "video",
target: "generateVideo-1",
targetHandle: "video",
});
});
it("creates an empty last-frame video node and links both videos to a generate video node", () => {
render(
<TestWrapper>
<VideoInputNode {...defaultProps} />
</TestWrapper>
);
fireEvent.click(screen.getByRole("button", { name: KEYFRAMES_LABEL }));
expect(mockUpdateNodeData).toHaveBeenCalledWith("video-1", {
label: FIRST_FRAME_NODE_LABEL,
});
expect(mockAddNode).toHaveBeenCalledWith(
"videoInput",
{ x: 120, y: 492 },
{ label: LAST_FRAME_NODE_LABEL }
);
expect(mockAddNode).toHaveBeenCalledWith("generateVideo", { x: 600, y: 316 });
expect(mockOnConnect).toHaveBeenCalledWith({
source: "video-1",
sourceHandle: "video",
target: "generateVideo-2",
targetHandle: "video",
});
expect(mockOnConnect).toHaveBeenCalledWith({
source: "videoInput-1",
sourceHandle: "video",
target: "generateVideo-2",
targetHandle: "video",
});
});
});

106
src/components/nodes/VideoInputNode.tsx

@ -5,6 +5,7 @@ import { Handle, Position, NodeProps, Node } from "@xyflow/react";
import { BaseNode } from "./BaseNode"; import { BaseNode } from "./BaseNode";
import { useWorkflowStore } from "@/store/workflowStore"; import { useWorkflowStore } from "@/store/workflowStore";
import { VideoInputNodeData } from "@/types"; import { VideoInputNodeData } from "@/types";
import { defaultNodeDimensions } from "@/store/utils/nodeDefaults";
import { useVideoBlobUrl } from "@/hooks/useVideoBlobUrl"; import { useVideoBlobUrl } from "@/hooks/useVideoBlobUrl";
import { downloadMedia } from "@/utils/downloadMedia"; import { downloadMedia } from "@/utils/downloadMedia";
import { useShowHandleLabels } from "@/hooks/useShowHandleLabels"; import { useShowHandleLabels } from "@/hooks/useShowHandleLabels";
@ -13,14 +14,81 @@ import { readVideoFile } from "@/utils/videoFile";
type VideoInputNodeType = Node<VideoInputNodeData, "videoInput">; type VideoInputNodeType = Node<VideoInputNodeData, "videoInput">;
// 空视频节点的快捷入口文案;使用转义写法,避免不同终端编码导致中文被写坏。
const TRY_LABEL = "\u5c1d\u8bd5:";
const KEYFRAMES_ACTION_LABEL = "\u9996\u5c3e\u5e27\u751f\u6210\u89c6\u9891";
const FIRST_FRAME_ACTION_LABEL = "\u9996\u5e27\u751f\u6210\u89c6\u9891";
const FIRST_FRAME_LABEL = "\u9996\u5e27";
const LAST_FRAME_LABEL = "\u5c3e\u5e27";
export function VideoInputNode({ id, data, selected }: NodeProps<VideoInputNodeType>) { export function VideoInputNode({ id, data, selected }: NodeProps<VideoInputNodeType>) {
const nodeData = data; const nodeData = data;
const updateNodeData = useWorkflowStore((state) => state.updateNodeData); const updateNodeData = useWorkflowStore((state) => state.updateNodeData);
const addNode = useWorkflowStore((state) => state.addNode);
const onConnect = useWorkflowStore((state) => state.onConnect);
const nodes = useWorkflowStore((state) => state.nodes);
const showLabels = useShowHandleLabels(selected); const showLabels = useShowHandleLabels(selected);
// Use blob URL for efficient playback of large base64 videos // Use blob URL for efficient playback of large base64 videos
const playbackUrl = useVideoBlobUrl(nodeData.video ?? null); const playbackUrl = useVideoBlobUrl(nodeData.video ?? null);
// 只负责搭建“首帧/尾帧 -> 生成视频”的节点链路,视频文件由用户后续在各节点上传。
const createGenerateVideoChain = useCallback(
(hasLastFrame: boolean) => {
const currentNode = nodes.find((node) => node.id === id);
const currentWidth =
currentNode?.measured?.width ??
(typeof currentNode?.style?.width === "number" ? currentNode.style.width : defaultNodeDimensions.videoInput.width);
const currentHeight =
currentNode?.measured?.height ??
(typeof currentNode?.style?.height === "number" ? currentNode.style.height : defaultNodeDimensions.videoInput.height);
const currentPosition = currentNode?.position ?? { x: 0, y: 0 };
const generateVideoHeight = defaultNodeDimensions.generateVideo.height;
const verticalGap = 52;
// 复用当前空视频节点作为首帧节点,避免点击快捷入口后留下多余空节点。
updateNodeData(id, { label: FIRST_FRAME_LABEL });
const lastFrameNodeId = hasLastFrame
? addNode(
"videoInput",
{
x: currentPosition.x,
y: currentPosition.y + currentHeight + verticalGap,
},
{ label: LAST_FRAME_LABEL }
)
: null;
const targetY = hasLastFrame
? currentPosition.y + currentHeight + verticalGap / 2 - generateVideoHeight / 2
: currentPosition.y;
// 生成节点放在素材节点右侧;双帧模式下按两帧整体高度居中。
const generateVideoNodeId = addNode("generateVideo", {
x: currentPosition.x + currentWidth + 180,
y: targetY,
});
onConnect({
source: id,
sourceHandle: "video",
target: generateVideoNodeId,
targetHandle: "video",
});
if (lastFrameNodeId) {
onConnect({
source: lastFrameNodeId,
sourceHandle: "video",
target: generateVideoNodeId,
targetHandle: "video",
});
}
},
[addNode, id, nodes, onConnect, updateNodeData]
);
const applyVideoFile = useCallback( const applyVideoFile = useCallback(
async (file: File) => { async (file: File) => {
if (!file) return; if (!file) return;
@ -117,10 +185,44 @@ export function VideoInputNode({ id, data, selected }: NodeProps<VideoInputNodeT
onDragOver={handleDragOver} onDragOver={handleDragOver}
className={`w-full h-full bg-neutral-900/40 flex flex-col items-center justify-center transition-colors ${nodeData.isOptional ? "border-2 border-dashed border-neutral-600" : ""}`} className={`w-full h-full bg-neutral-900/40 flex flex-col items-center justify-center transition-colors ${nodeData.isOptional ? "border-2 border-dashed border-neutral-600" : ""}`}
> >
<svg className="w-8 h-8 text-neutral-600" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={1.5}> <svg className="w-10 h-10 text-neutral-600" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={1.5}>
<path strokeLinecap="round" strokeLinejoin="round" d="m15.75 10.5 4.72-4.72a.75.75 0 0 1 1.28.53v11.38a.75.75 0 0 1-1.28.53l-4.72-4.72M4.5 18.75h9a2.25 2.25 0 0 0 2.25-2.25v-9a2.25 2.25 0 0 0-2.25-2.25h-9A2.25 2.25 0 0 0 2.25 7.5v9a2.25 2.25 0 0 0 2.25 2.25Z" /> <path strokeLinecap="round" strokeLinejoin="round" d="m15.75 10.5 4.72-4.72a.75.75 0 0 1 1.28.53v11.38a.75.75 0 0 1-1.28.53l-4.72-4.72M4.5 18.75h9a2.25 2.25 0 0 0 2.25-2.25v-9a2.25 2.25 0 0 0-2.25-2.25h-9A2.25 2.25 0 0 0 2.25 7.5v9a2.25 2.25 0 0 0 2.25 2.25Z" />
</svg> </svg>
<span className="text-xs text-neutral-500 mt-2">{nodeData.isOptional ? "Optional" : "Drop video"}</span> {nodeData.isOptional ? (
<span className="text-xs text-neutral-500 mt-2">Optional</span>
) : (
<div className="mt-8 w-full max-w-[190px] self-start ml-7">
<div className="mb-3 text-sm text-neutral-400">{TRY_LABEL}</div>
<button
type="button"
className="nodrag nopan mb-4 flex items-center gap-2 text-sm font-semibold text-white transition-colors hover:text-blue-200 focus:outline-none focus:text-blue-200"
onClick={(event) => {
event.preventDefault();
event.stopPropagation();
createGenerateVideoChain(true);
}}
>
<svg className="h-4 w-4 shrink-0" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={1.8}>
<path strokeLinecap="round" strokeLinejoin="round" d="M4 7.5 12 3l8 4.5-8 4.5L4 7.5Zm0 4.5 8 4.5 8-4.5M4 16.5 12 21l8-4.5" />
</svg>
{KEYFRAMES_ACTION_LABEL}
</button>
<button
type="button"
className="nodrag nopan flex items-center gap-2 text-sm font-semibold text-white transition-colors hover:text-blue-200 focus:outline-none focus:text-blue-200"
onClick={(event) => {
event.preventDefault();
event.stopPropagation();
createGenerateVideoChain(false);
}}
>
<svg className="h-4 w-4 shrink-0" fill="currentColor" viewBox="0 0 24 24">
<path d="M12 2.75 14.65 9.35 21.25 12 14.65 14.65 12 21.25 9.35 14.65 2.75 12 9.35 9.35 12 2.75Z" />
</svg>
{FIRST_FRAME_ACTION_LABEL}
</button>
</div>
)}
</div> </div>
)} )}

Loading…
Cancel
Save