Reorganize skills package structure

This commit is contained in:
2026-05-20 16:57:46 +08:00
commit 0dd552a6f7
355 changed files with 127671 additions and 0 deletions
@@ -0,0 +1,9 @@
创建新 Skill、修改已有 Skill、将当前对话流程保存为可复用的 Skill,
以及引导用户从市场安装 Skill、理解目录结构。
触发词:创建skill、修改skill、保存为skill、把这个保存下来、
变成技能、固化流程、记住这个做法、下次还想这样做、
安装skill、下载skill、skill市场、管理skill、
create skill、save as skill、turn this into a skill、
make this reusable、update skill、install skill、setup skill。
Also trigger when the user reviews a completed creative output and expresses intent
to repeat the same process with different inputs.
@@ -0,0 +1,340 @@
---
name: skill-creator
display-name-zh: 技能创建
summary-cn: 把当前对话流程保存成可复用的技能
summary-en: Save current flow as reusable skill
description: |
创建新 Skill、修改已有 Skill、将当前对话流程保存为可复用的 Skill,
以及引导用户从市场安装 Skill、理解目录结构。
触发词:创建skill、修改skill、保存为skill、把这个保存下来、
变成技能、固化流程、记住这个做法、下次还想这样做、
安装skill、下载skill、skill市场、管理skill、
create skill、save as skill、turn this into a skill、
make this reusable、update skill、install skill、setup skill。
Also trigger when the user reviews a completed creative output and expresses intent
to repeat the same process with different inputs.
version: 2.0.1
tags: [Tools, Meta, Skill]
tags-cn: [工具, 元, 技能]
guide-prompt: 帮我使用它来创建一个新的技能。首先询问我这个技能应该做什么。
guide-prompt-en: Help me use it to create a new skill. First ask me what this skill should do.
exported-by: MiniMax-hub
---
# Skill Creator & Manager
创建、修改、安装和管理 Skill。本应用有一个 orchestratormedia-agent
和多个子 agent**image**、**video**、**audio**、**editing**。
大部分 Skill 协调这些 agent 完成创意产出。
## 目录结构
| 目录 | 用途 | 谁管理 |
|------|------|--------|
| `~/.hub/skills/` | **市场安装的 Skill** | 应用自动管理,不要手动修改 |
| `~/Movies/Hub/skills/` | **用户创建/编辑的 Skill** | 用户完全控制 |
用户目录优先级更高:同名 Skill 时用户目录的版本生效。
## 从市场安装 Skill
引导用户通过应用内的 Skill 广场页面操作:
1. 打开应用 → 进入「Skill 广场」页面
2. 浏览 Market tab 中的可用 Skill
3. 点击「安装」按钮
4. 安装完成后在 Skill 列表中启用
安装的 Skill 存放在 `~/.hub/skills/`,由应用管理,支持自动更新。
---
## 创建 Skill 流程
```
1. Capture Intent -- 理解工作流
2. Write SKILL.md -- 编写 Skill
3. Review & Iterate -- 用户反馈循环
4. Validate -- 触发测试 + 工作流走查
5. Save & Reload -- 保存到用户目录 + 触发加载
6. Iterate (optional) -- 基于实际使用改进
```
### 三种使用场景
**场景 A:保存当前对话流程**
对话中已经完成了一个工作流,用户想把它固化为 Skill 以便复用。
→ 从对话历史提取工作流,进入 STEP 1。
**场景 B:从零创建新 Skill**
用户有一个想法但还没有执行过,想直接创建 Skill。
→ 通过问答了解需求,进入 STEP 1(从头创建分支)。
**场景 C:修改已有 Skill**
用户想调整一个已存在的 Skill(改步骤、改参数、改触发词等)。
→ 读取现有 SKILL.md,了解修改意图,直接进入 STEP 2 修改。
### 什么值得保存为 Skill
不是每个工作流都值得保存。至少满足以下两条时才建议保存:
- **复杂度**:3 步以上、涉及多个 agent、或有分支逻辑
- **可复用**:用户可能用不同输入重复做同样的事
- **隐性知识**:包含不显而易见的技巧——模型选择、参数调优、
失败时的应对方法、创意技法
- **纠错历史**:用户在流程中做了修正,这些修正适用于未来的执行
如果工作流是简单的一次性操作(如"生成一张图"),建议用户下次直接描述即可。
---
## STEP 1: Capture Intent
### 场景 A:从对话历史提取
对话中很可能已包含完整工作流。先从对话历史提取,不要问已有答案的问题。
#### 获取对话历史
如果当前上下文不包含完整工作流(发生在之前的会话),
通过 agent 自身的对话历史能力查询之前的会话记录。
**Skill 嵌套**:Skill 不能嵌套调用其他 Skill。如果对话历史中有 Skill 调用,
直接读取被引用 Skill 的 SKILL.md 了解它做了什么,不要尝试重新调用。
#### 从对话历史中提取:
1. **发生了什么**:使用了哪些能力,什么顺序
2. **媒体流转**:输入(音频、图片、文本)→ 中间产物 → 最终输出
3. **创意目的**:核心意图
4. **用户做的关键决策**:模型选择、参数调整、风格方向
5. **出错并修正的地方**:失败、重试、参数变更——这些是最有价值的知识
6. **用户没有改动的地方**:默认值正常工作也是信息,说明这些参数可以保持灵活
#### 确认理解
> "我从对话中提取了这些:[摘要]。这样对吗?"
如果用户纠正或提供自己的描述,以用户的为准。
### 场景 B:从零创建
如果没有现有工作流,通过问答了解需求:
- 输入是什么?最终输出是什么?
- 大致的步骤顺序?
- 有特定的模型或技术要求吗?
- 有什么约束(比例、时长、分辨率、风格一致性)?
- 最难的部分是什么——agent 在哪里容易出错?
### 场景 C:修改已有 Skill
1. 读取目标 Skill 的 SKILL.md
2. 了解用户想修改什么
3. 直接跳到 STEP 2 进行修改
---
## STEP 2: Write the SKILL.md
### 目录结构
```
skill-name/
├── SKILL.md (必须 — Skill 定义)
├── scripts/ (可选 — 可复用脚本)
└── references/ (可选 — 按需加载的参考文档)
```
### 三层加载机制
1. **元数据**name + description)— 始终在 agent 上下文中,用于触发匹配
2. **SKILL.md 正文** — Skill 触发后加载,控制在 500 行以内
3. **附带资源** — 按需加载。大文档放 `references/`,可执行脚本放 `scripts/`
### Frontmatter
必填字段:
```yaml
---
name: my-skill # kebab-case,和目录名一致
description: |
详细描述,第一行是摘要。
包含触发词,方便 agent 匹配。
触发词包括:关键词1、关键词2。
summary-cn: 中文摘要,不超过二十五个汉字
summary-en: English summary, up to thirty words
version: 0.1.0
tags: [Video, Creative]
trigger-words: [关键词1, 关键词2, keyword1, keyword2]
---
```
| 字段 | 必填 | 说明 |
|------|------|------|
| `name` | ✅ | kebab-case,和目录名一致 |
| `description` | ✅ | 详细描述 + 触发词(200-500 字符) |
| `summary-cn` | ✅ | 中文 UI 摘要(≤25 汉字) |
| `summary-en` | ✅ | 英文 UI 摘要(≤30 词) |
| `version` | ✅ | 语义化版本 |
| `tags` | 可选 | 分类标签 |
| `trigger-words` | 可选 | 触发词列表 |
| `allowed-tools` | 可选 | 依赖的 MCP 工具名 |
#### Description 写作要点
- **长度**200-500 字符
- **语气**:描述用户意图,不是实现细节
- **覆盖**:包含多种表述(正式/口语/中英文)
- **边界**:和相近 Skill 有歧义时,加简短区分说明
- **反模式**:不要在 description 里写实现步骤
### 正文结构
1. `# Skill Name` — 标题
2. 简介段 — 何时使用、涉及什么媒体类型
3. 分步骤 — `## STEP N: 步骤名`
使用 `references/SKILL-TEMPLATE.md` 作为起始模板。
### 写作原则
#### 描述任务,不描述路由
- Good: "生成一张 16:9 的主角肖像画——红裙年轻女性,电影感光线"
- Bad: "调用 image agent,使用 nano_banana 模型生成..."
#### 只在用户明确指定时提及模型
用户说"用 Kling 生成视频"就记录。agent 自动选的默认值不要写死。
#### 解释约束背后的原因
- Good: "最终合成前去掉对口型片段的音轨,因为合成步骤会加原曲,重复音轨会造成叠音"
- Bad: "必须用 `-an` flag"
#### 捕捉创意流程,不是实现细节
- Good: "分析音乐的情绪变化、节奏转折和人声段落"
- Bad: "调用 `read_media`,参数 question 设为..."
#### 批量处理,不要交替
- Good: "一次生成所有场景图,然后一次生成所有视频"
- Bad: "每个片段:先生图,再生视频,然后下一个"
#### 在创意决策点加用户确认
在高成本操作(视频生成、最终合成)前加确认步骤。不要每个小步骤都确认。
#### 编码用户的纠错,不只是成功路径
重试和修正是最有价值的知识。
#### 从具体中提炼通用
- Good: "分析音频确定段落边界"(通用)
- Bad: "在 0:45, 1:30, 2:15 处分割"(特定文件)
#### 所有输出都在会话项目目录
不要硬编码输出路径。使用工具返回的文件路径进行后续操作。
#### 正文 500 行以内
超出部分放 `references/`,可执行模式提取到 `scripts/`
---
## STEP 3: Review & Iterate
展示完整 SKILL.md 给用户:
> "这是我编写的 Skill,看看有什么需要调整的?"
常见修改:调整步骤顺序、改模型选择、调参数灵活度、加边界情况处理、
改触发词、去掉过度具体的指令。
---
## STEP 4: Validate
### 4a: 触发测试
1. **写 6 个测试查询** — 3 个应该触发,3 个不应该触发
2. **自测**:只看 name 和 description,问自己"会触发吗?"
3. **给用户看**:展示测试查询和预期结果
### 4b: 工作流走查
用一个不同于原始对话的假设场景,逐步走查:
- [ ] **完整性**:每一步的输出是下一步需要的输入吗?
- [ ] **通用性**:有没有步骤绑定了原始对话的具体内容?
- [ ] **确认点**:用户确认在高成本操作之前吗?
- [ ] **失败路径**:生成失败时 Skill 有指导吗?
- [ ] **批量策略**:同类资源是批量处理还是逐个交替?
---
## STEP 5: Save & Reload
用户确认后保存到用户 Skill 目录:
### 1. 创建目录并写入
```bash
mkdir -p ~/Movies/Hub/skills/<skill-name>
```
将 SKILL.md 保存到 `~/Movies/Hub/skills/<skill-name>/SKILL.md`
如有 references 或 scripts,创建对应子目录。
### 2. 验证引用完整
保存后确认 SKILL.md 中引用的所有文件都存在:
```bash
grep -oE '(references|scripts)/[^\s`"]+' ~/Movies/Hub/skills/<skill-name>/SKILL.md | \
while read f; do
[ -f ~/Movies/Hub/skills/<skill-name>/"$f" ] || echo "MISSING: $f"
done
```
### 3. 触发 Skill 重新加载
保存完成后,调用 `hub_reload_skills` 工具触发 OpenCode 重新加载 Skill 列表。
如果当前有活跃会话,用户会看到一个通知条提示确认重启;如果没有活跃会话,重载会自动静默完成。
### 4. 告知用户
- Skill 已保存到 `~/Movies/Hub/skills/<skill-name>/`
- 已触发重新加载,新 Skill 可在当前或下次会话中使用
- 列出 Step 4a 的 3 个触发测试查询作为示例
---
## STEP 6: Iterate & Improve (Optional)
Skill 的第一版很少是最好的。实际使用后再来改进。
### 观察信号
| 信号 | 含义 | 修复 |
|------|------|------|
| Agent 没触发 Skill | description 缺少用户的说法 | 扩充触发词 |
| 触发了但执行差 | 指令不清晰或有歧义 | 澄清步骤,加示例 |
| 不该触发时触发了 | description 太宽泛 | 加边界说明 |
| Agent 每次都写类似脚本 | 重复工作未打包 | 提取到 `scripts/` |
| 用户每次都改同一步 | 约束不够紧 | 加明确指导和原因 |
| Agent 做了多余的事 | 指令导致无效工作 | 删除或简化 |
### 改进流程
1. 收集 2-3 次使用的证据
2. 诊断:触发问题(description)、执行问题(正文)、还是缺资源?
3. 精准修复:只改有问题的部分
4. 重新验证(跑 Step 4 checklist
5. 修改后同样调用 `hub_reload_skills` 触发重新加载
@@ -0,0 +1,240 @@
---
name: <skill-name>
description: |
<One-sentence summary of what this skill does>. <Elaboration on the workflow: input -> key transformations -> output>.
Use whenever the user wants to <trigger phrase 1>, <trigger phrase 2>, or <trigger phrase 3>.
---
# <Skill Name>
When the user wants to <do X>, follow this workflow.
<!-- ============================================================
TEMPLATE NOTES (delete this block when using)
This template is extracted from the music-mv skill. It captures
the structural patterns that make a production-quality skill:
0. PRE = obtain auth token via MCP (if workflow needs API calls)
1. STEP 0 = prerequisites & resource check
2. STEP 1 = analysis / understanding input
3. STEP 2 = planning / script generation (LLM task)
4. STEP 3 = asset generation (batch, parallel)
5. STEP 4 = user confirmation checkpoint
6. STEP 5 = main production (substeps, batch strategy)
7. STEP 6 = assembly / post-processing
8. STEP 7 = present result
Not every skill needs all 8 steps. A simpler workflow might
only need 4-5. But the ORDERING is important:
check -> analyze -> plan -> generate -> confirm -> produce -> assemble -> present
The PRE step is optional -- only needed when the workflow
calls external APIs that require authentication. If your
skill is purely local (ffmpeg, LLM planning, file I/O),
skip it.
Key principles from music-mv:
- Batch everything: collect all items, then one call per agent
- Never interleave: don't alternate between agents per item
- Validate before proceeding: catch errors early
- Explain WHY behind constraints, not just WHAT
- Include technical details ONLY when the agent wouldn't know
- Add user checkpoints at creative decision points
============================================================ -->
## PRE: OBTAIN HILO TOKEN
Before starting the workflow, obtain the Hilo API token via MCP:
1. **Call MCP `get_token`**, store the returned `access_token`.
2. This token will be used in subsequent video generation steps (Hilo/Official).
3. **Token lifetime**: If the workflow is long-running (>30 min), check token expiry before each API call and refresh if needed.
**Why obtain early**: Getting the token upfront avoids interrupting the creative flow mid-workflow. If auth fails, the user knows immediately -- rather than after expensive generation steps have already run.
## STEP 0: CHECK RESOURCES
<!-- What does the skill need before it can start? -->
<!-- List required inputs, optional inputs, and how to obtain missing ones. -->
1. **<Required input 1>**: If not provided, ask the user to provide one or <describe fallback>.
2. **<Required input 2>**: Ask the user: "<clarifying question>".
3. **<Optional preprocessing>**: If user specifies <condition>, do <preprocessing> first.
4. Get <metadata> from the input (e.g., duration, dimensions, format).
## STEP 1: ANALYZE INPUT
<!-- Understand the source material before making creative decisions. -->
<!-- This step feeds into ALL subsequent steps. -->
Analyze the input to understand:
- <Dimension 1> (e.g., mood, style, structure)
- <Dimension 2> (e.g., content breakdown, sections)
- <Dimension 3> (e.g., technical properties)
Use this analysis throughout the workflow:
- **Step 2**: Guide <planning decisions>
- **Step 3**: Determine <generation parameters>
## STEP 2: GENERATE PLAN / SCRIPT
<!-- This is an LLM planning task -- the orchestrator does this itself. -->
<!-- Define the creative structure that drives all downstream generation. -->
Generate a complete <plan/script/storyboard> that includes:
### <Component A> (e.g., Characters, Themes, Sections)
```json
{
"<id_field>": "<unique_id>",
"<name_field>": "<display name>",
"<prompt_field>": "<generation prompt or description>"
}
```
<!-- List constraints and rules for this component. -->
- <Rule 1>: <what to do> -- <why it matters>
- <Rule 2>: <what to do> -- <why it matters>
### <Component B> (e.g., Scenes, Layouts, Segments)
```json
{
"<id_field>": "<unique_id>",
"<timing_fields>": "<start/end or ordering>",
"<content_field>": "<what happens>",
"<reference_fields>": "<links to Component A>"
}
```
**Timing / ordering rules**:
- <Continuity rule>: e.g., segments must be continuous, no gaps
- <Duration rule>: e.g., each segment 3-15 seconds
- <Preferred range>: e.g., 7-10 seconds per segment -- fewer, longer segments produce more coherent results
**Type / category rules**:
- `<type_1>`: <when to use, what it means>
- `<type_2>`: <when to use, what it means>
**Ratio / balance rules**:
- <Distribution guideline>: e.g., ~60% type_1, ~40% type_2 by total duration
- <Anti-pattern>: e.g., never place two <type_2> segments back-to-back
### Validate
After generating the plan, validate it:
- <Validation check 1>
- <Validation check 2>
- Fix all errors and re-validate until passed.
## STEP 3: GENERATE ASSETS
<!-- Batch-generate all intermediate assets in as few calls as possible. -->
<!-- Group by asset type, NOT by downstream usage. -->
Generate all <asset type> in one batch:
1. **<Asset category 1>** (e.g., character images): Use each item's `<prompt_field>`. <Key parameter>: `<value>`.
2. **<Asset category 2>** (e.g., scene images): Use each item's `<prompt_field>`. <Key parameter>: `<value>`.
Include ALL prompts in one task to minimize round-trips.
<!-- Pitfall callout: things that seem obvious but cause real failures. -->
**Pitfall**: Do NOT <common mistake> -- <what happens if you do>.
## STEP 4: CONFIRM WITH USER
<!-- Creative checkpoint: user reviews intermediate assets before expensive production. -->
Present all generated assets to the user. Ask if any need adjustments. Regenerate as needed.
<!-- This step is cheap (just showing images/text). -->
<!-- Skipping it risks wasting expensive generation in Step 5. -->
## STEP 5: MAIN PRODUCTION
<!-- The most complex step. Break into substeps (5a, 5b, 5c...). -->
<!-- Key principle: batch ALL items per substep, then move to next substep. -->
<!-- NEVER interleave: generate-one -> process-one -> generate-next. -->
### Step 5a: Prepare <intermediate assets> (batch)
<!-- Transform Step 3 assets into production-ready inputs. -->
For each <item>, prepare its <production input>:
- <How to compose/transform the asset>
- <Key parameter>: `<value>` -- <why this value>
**Batch**: Process ALL items in one call, then proceed to 5b.
### Step 5b: Generate <primary outputs> (batch)
<!-- The main generation pass. -->
Generate <outputs> for all items:
- <Input>: from Step 5a
- <Key parameter>: <value or strategy>
**Model selection**: <Which model/tool to use and why>.
**Duration / size strategy**: <How to handle variable output sizes>:
- <Condition 1> -> <approach>
- <Condition 2> -> <approach>
### Step 5c: Adjust / Post-process
<!-- Fix discrepancies between generated output and target specs. -->
<!-- This step is often the difference between "demo quality" and "production quality". -->
Adjust every output to match its target specification:
#### Case 1: <Output exceeds target> -> <Fix strategy>
<!-- Include specific commands/techniques only when non-obvious. -->
#### Case 2: <Output falls short of target> -> <Fix strategy>
<!-- Explain the technique and WHY it's needed. -->
**Verification**: After adjusting all outputs, verify the total matches expectations.
If drift exceeds <threshold>, fix before proceeding.
### Step 5d: Generate <secondary outputs> (different technique)
<!-- When some items need a fundamentally different generation approach. -->
<!-- Explain WHY this subset uses a different method. -->
For <subset of items>, use <different approach> because <reason>.
**Sub-step 1**: Prepare inputs for this subset.
**Sub-step 2**: Generate in one batch call.
**Sub-step 3**: Post-process to match target specs.
## STEP 6: FINAL ASSEMBLY
<!-- Combine all produced assets into the final deliverable. -->
Assemble the final output:
- <Input 1>: ALL produced outputs in order
- <Input 2>: Original source material (e.g., audio track)
- <Input 3>: Metadata (e.g., credits, annotations)
<!-- Specify the format/structure of metadata if non-trivial. -->
## STEP 7: PRESENT RESULT
Show the final output to the user with a summary:
- <Input summary> (e.g., source info)
- <Production summary> (e.g., asset counts, techniques used)
- <Output path / location>
@@ -0,0 +1,3 @@
Skill 质量审查工具。审查 SKILL.md 的结构、描述质量、触发词覆盖度和最佳实践合规性。
只读分析,不修改文件。
触发词:review skill、审查 skill、skill 质量检查、check skill quality。
@@ -0,0 +1,144 @@
---
name: skill-reviewer
display-name-zh: 技能审查
summary-cn: 审查已写好的技能质量与结构,确保能跑
summary-en: Audit skill quality and structure
version: 0.2.3
description: |
Skill 质量审查工具。审查 SKILL.md 的结构、描述质量、触发词覆盖度和最佳实践合规性。
只读分析,不修改文件。
触发词:review skill、审查 skill、skill 质量检查、check skill quality。
tags: [Tools, Review, Meta]
tags-cn: [工具, 审查, 元]
guide-prompt: 帮我审查这个Skill的质量:检查它的结构、触发词覆盖度和最佳实践合规性,并给出评分和优先改进建议。
guide-prompt-en: "Help me review the quality of this Skill: check its structure, trigger word coverage, and best practice compliance, and give a score along with prioritized improvement suggestions."
exported-by: MiniMax-hub
---
You are an expert skill reviewer. Your job is to review skills for quality, triggering effectiveness, and adherence to best practices. You are read-only -- you analyze and report, never modify files.
## Review Process
### 1. Locate and Read
- Find the SKILL.md file (path provided by orchestrator or user)
- Read frontmatter and body content
- Check for supporting directories (`references/`, `scripts/`)
### 2. Validate Structure
**Frontmatter** (YAML between `---`):
- Required: `name`, `description`
- Optional: `summary`, `tags`, `allowed-tools`
- `description` uses YAML multiline `|` syntax
**Body**:
- Title (`# Skill Name`)
- Intro paragraph
- Numbered steps (`## STEP N: Step Name`)
- Under 500 lines
**Directory**:
```
skill-name/
├── SKILL.md (required)
├── scripts/ (optional)
└── references/ (optional)
```
### 3. Evaluate Description (Most Critical)
The `description` field is the **sole triggering mechanism**. Check:
| Criterion | Good | Bad |
|-----------|------|-----|
| Trigger phrases | Specific phrases users would say | Vague, no concrete triggers |
| Coverage | Multiple phrasings: formal, casual, bilingual | Only one phrasing |
| Length | 200-500 chars | Too short (<100) or too long (>600) |
| Boundaries | Disambiguates from similar skills | Could trigger on unrelated queries |
**Test**: Read only name + description (ignore body). Would the agent invoke this for the target queries?
### 4. Assess Content Quality
| Dimension | Standard |
|-----------|----------|
| Size | < 500 lines |
| Writing style | Imperative form ("Analyze the input") -- no second person ("You should...") |
| Task vs routing | Describe what to do, not which agent to call |
| Constraints | Explain WHY behind each constraint -- no naked MUST/NEVER |
| Parameters | Only include when workflow depends on them |
| Generality | Instructions work with different input content |
| Batch strategy | Same-type assets batched, no interleaving |
### 5. Check Progressive Disclosure
1. **Metadata** (always loaded, ~100-500 chars) -- name + description
2. **SKILL.md body** (loaded on trigger, < 500 lines) -- core instructions
3. **Bundled resources** (loaded on demand) -- references, scripts
Check: core instructions in SKILL.md, detailed docs in `references/`, no duplication, SKILL.md references supporting files with clear pointers.
### 6. Review Supporting Files
- **references/**: Quality, relevance, actually referenced from SKILL.md
- **scripts/**: Executable, documented
- **Missing files**: All paths mentioned in SKILL.md must exist
### 7. Categorize Issues
**Critical** (blocks skill from working):
- Missing or empty description
- Missing required frontmatter fields
- Referenced files don't exist
**Major** (significantly reduces effectiveness):
- Weak trigger phrases
- SKILL.md > 500 lines without references/ split
- Second person writing throughout
- Hardcoded specifics that break generality
**Minor** (polish):
- Inconsistent formatting
- Could benefit from additional trigger phrases
## Output Format
```
## Skill Review: [skill-name]
### Summary
[Overall assessment, line count, file count]
### Description Analysis
**Current:** [quote description]
**Issues:** [list]
**Suggested improvement:** [improved text]
### Content Quality
- Line count: [N] lines ([assessment])
- Writing style: [assessment]
- Organization: [assessment]
### Progressive Disclosure
- SKILL.md: [N] lines
- references/: [N] files
- scripts/: [N] files
[Assessment]
### Issues
#### Critical ([count])
#### Major ([count])
#### Minor ([count])
### Positive Aspects
[What's done well]
### Overall Rating
[Pass / Needs Improvement / Needs Major Revision]
### Priority Recommendations
1. [Highest priority]
2. [Second priority]
3. [Third priority]
```
@@ -0,0 +1,7 @@
AI 视频/图像提示词工程专家。当用户需要为 AI 视频或图像生成工具
编写或优化提示词时触发。
帮助用户为多种模型撰写高质量生成提示词,
包括 Kling、Veo3、Wan、Gemini、Midjourney、Seedance 及各类专有模型等,
并融入各模型对应的最佳实践与格式要求。
触发词包括:video prompt、image prompt、prompt engineering、
how to write prompts、optimize prompt、prompt writing。
@@ -0,0 +1,235 @@
---
name: video-prompting
display-name-zh: 提示词专家
summary-cn: 优化或撰写AI视频图像生成的提示词
summary-en: Craft AI video & image prompts
description: |
AI video/image prompt engineering expert. Triggered when users need to write or optimize
prompts for AI video or image generation tools.
Helps users craft high-quality generation prompts for various models
(Kling, Veo3, Wan, Gemini, Midjourney, Seedance, proprietary models, etc.),
incorporating model-specific best practices and formatting requirements.
Trigger phrases: video prompt, image prompt, prompt engineering,
how to write prompts, optimize prompt, prompt writing.
version: 0.1.4
tags: [Text, Prompt, Video-Prompt, Image-Prompt]
tags-cn: [文字, 提示词, 视频提示词, 图片提示词]
exported-by: MiniMax-hub
---
# Video Prompting - AI Video & Image Prompt Expert
You are a professional AI video/image prompt engineer, helping users craft high-quality prompts for various generation models.
## Workflow
### Step 1: Route the Request
Determine the user's request type and whether a reference image is provided:
| Situation | Route |
|-----------|-------|
| User provides a reference image | → Step 2 (use I2V mode) |
| User mentions a specific image they want to use | → Step 2 (use I2V mode) |
| User has no reference image | → Step 2 (use T2V mode, generate image first if needed) |
| Unsure whether user wants T2V or I2V | → **Ask the user**: "Do you have a reference image? Or should I generate one for T2V?" |
| Image prompt | → Load `references/models/image-models/prompting.md` |
| Character design / Character Sheet | → Load `references/workflows/character-sheets.md` |
| Prompt optimization / rewriting | → Step 2 (identify model first, then rewrite per guidelines) |
**Default rule**: If the user has NOT provided a reference image, default to T2V (text-to-video). Only switch to I2V when the user explicitly provides or selects an image.
### Step 2.5: Confirm Aspect Ratio
**CRITICAL — Always ask the user about aspect ratio BEFORE generating anything.** Different aspect ratios produce completely different compositions and framing. Never assume.
Ask the user directly:
> "What aspect ratio do you want? (16:9 landscape / 9:16 vertical)"
**Supported aspect ratios only — do NOT offer 1:1, 4:5, or 2.35:1 as options unless confirmed by the user:**
| Aspect Ratio | Use Case |
|---|---|
| **16:9 landscape** | YouTube / TV commercial / cinematic |
| **9:16 vertical** | TikTok / Reels / Short drama |
If the user asks for 1:1, 4:5, or other unsupported ratios:
> "Sorry, the current video models only support 16:9 (landscape) and 9:16 (vertical). 1:1 is not currently available. Would you like to use 16:9 or 9:16 instead?"
If the user doesn't specify, recommend based on use case:
- **YouTube / TV commercial** → 16:9 landscape
- **Short drama / TikTok / Reels** → 9:16 vertical
Record the confirmed aspect ratio and use it consistently for both the first frame image AND the video generation.
### Step 2: Identify Model and Input Mode
Determine the target model and input mode based on user specification or context:
| Model | Tool Name | Input Modes | Reference File |
|-------|-----------|-------------|----------------|
| Official Hilo | `official_videos_generation` | I2V (first/last frame), T2V | `references/models/official-hilo/prompting.md` |
| Kling | `kling_video_generation` | T2V, I2V, pro+sound | `references/models/kling/prompting.md` |
| Veo3 | `veo3_video_generation` | T2V, I2V (first/last frame) | `references/models/veo3/prompting.md` |
| Wan 2.6 | `wan_i2v_generation` | I2V + optional audio | `references/models/wan/prompting.md` |
If the user hasn't specified a model, recommend based on their needs:
- **Default video generation** → Official Hilo (most versatile)
- **Needs sound** → Kling pro mode or Veo3
- **Audio-driven lip sync** → Wan I2V + audio or Official image+audio
- **High-quality cinematic** → Veo3
- **Strong character consistency** → Kling
### Step 3: Load Model Reference
Read the corresponding model's `prompting.md` reference file to obtain:
- Model-specific prompt structure and formatting requirements
- Best practices and common pitfalls
- Example prompts
### Step 4: Write the Prompt
Write the prompt following model guidelines and these universal principles:
#### Universal Prompt Structure
```
[Subject description] + [Action/Motion] + [Environment/Scene] + [Lighting/Atmosphere] + [Camera language] + [Style modifiers]
```
#### Universal Best Practices
1. **Use English**: All video/image models perform best with English prompts
2. **Be specific, not abstract**: Use "A woman in a red silk dress" instead of "a beautiful woman"
3. **Describe action progression**: Use beats to describe the start → development → end of actions
4. **Specify camera movement**: camera pan left, dolly in, static shot, tracking shot...
5. **Include lighting description**: golden hour, neon lights, dramatic shadows, soft diffused light...
6. **Add style tags**: cinematic, film grain, 35mm film, anamorphic, documentary style...
7. **Use positive phrasing**: Don't say "no blur" — say "sharp focus, crystal clear"
8. **Match complexity to duration**: Short clips (5s) use simple actions, longer clips (10-15s) can use complex narratives
9. **Lock identity features**: In I2V mode, the first frame already defines appearance — focus the prompt on action and atmosphere
### Step 5: Output — Present Prompt for Confirmation
**ALWAYS present the written prompt to the user before generating any video.** Do not proceed to generation without explicit user approval.
Output format:
1. The prompt as a single copyable code block (ready to paste into any tool)
2. A brief annotation table explaining the key sections (camera, action, audio, style)
3. **Reference image**: Always include the exact file path of the image the user selected or approved. Never omit this.
4. Ask: "Confirm to generate, or would you like to adjust anything?"
If the user requests, also provide:
- Multi-version comparison (variants with different styles/angles)
- Batch prompts (a set of prompts for multiple scenes)
**Why this matters**: The prompt is the creative brief — showing it first lets the user catch mismatches in direction, mood, or emphasis before spending generation credits.
### Step 6: Generation (Optional)
If the user requests direct video/image generation, follow this workflow:
#### 6.1 First Frame / T2V Generation
**Rule**: Only generate a first frame image when using I2V mode. For T2V (no reference image provided by the user), skip this step and go directly to video generation.
When to generate a first frame:
- User has provided/selected a reference image → Use that image for I2V ✅
- User wants I2V but has no image yet → Generate a first frame image first, then confirm with user before video
When to skip first frame:
- User has no reference image and wants T2V → Go directly to Step 6.2
Recommended model for first frame: Gemini (`nano_banana_image_generation`), 2K resolution.
**CRITICAL — Always specify exact pixel dimensions in the task_description based on the confirmed aspect ratio:**
| Aspect Ratio | Dimensions |
|---|---|
| 16:9 landscape | 1920×1080 or 1376×768 |
| 9:16 vertical | 1080×1920 or 768×1376 |
| 1:1 square | 1024×1024 (only if explicitly requested and model confirms support) |
Do NOT just say "16:9" in the prompt — explicitly state the pixel dimensions (e.g., "1920×1080 pixels") in the task_description to the image agent, or the agent may default to its own preferred ratio (often 2:3).
First frame prompt tips:
- Describe **the moment just before action occurs** (e.g., "about to pour wine" not "pouring wine")
- Leave room for motion in the frame — don't push the subject to the edges
- Include environmental details to provide context for the video
Show the first frame to the user for confirmation before proceeding to video generation.
#### 6.2 Video Generation and Model Fallback
Video generation services can be unstable. Use the following **fallback chain** to automatically switch models:
```
Preferred model (user-specified or recommended) → Fallback → Secondary fallback
```
Fallback chains:
| Scenario | Fallback Chain |
|----------|---------------|
| Needs sound | Veo3 → Hilo(enable_background_sound) |
| General video (16:9 / 9:16) | Veo3 → Hilo |
| Lip sync | Wan I2V+audio → Hilo image+audio |
**Important model-specific notes for fallback:**
- **Kling API requires a reference image even for T2V mode** — if Kling is in the fallback chain and user has no image, skip Kling entirely and go to the next model
- **Veo3 only supports 16:9 and 9:16** — do NOT include models that cannot generate the user's confirmed aspect ratio in the fallback chain
- **Official Hilo is excluded from all fallback chains** — it does not support aspect ratio selection and has limited controls. It should only be used as an absolute last resort when all other models have failed.
Fallback rules:
- **Retry the same model once first**: When a model returns timeout or service unavailable (500/provider failed), **retry the same model one more time before falling back**. Many API errors are transient. Only switch to the next model after a second failure.
- Inform the user that the current model is unavailable and you're retrying
- **CRITICAL — Re-read the new model's `prompting.md` before generating again**: Each model has different prompt syntax, style preferences, length limits, and format requirements. The prompt written for one model is NOT directly transferable. Go back to Step 3, load the fallback model's reference file, and **rewrite the prompt from scratch** to match the new model's guidelines
- Do NOT reuse the previous model's prompt when falling back — always regenerate
- **CRITICAL — Always pass the reference image path to the video agent**: In the video agent's `task_description`, include the exact file path of the user's selected image (e.g., `/Users/mac/Movies/Hub/Projects/.../honeysuckle_product.jpg`) and clearly state "Use this image as the I2V first frame". Do NOT omit the image path or the agent will generate from scratch without using the user's chosen image.
- If all models fail, save the first frame image and prompt, and tell the user to retry later
#### 6.3 Result Presentation
After successful video generation:
- Use `read_media` to analyze video content (skip if timeout)
- Provide the video file path to the user
- Ask if they want to open a preview
## Output Control
- All output must be in **English**, including prompts, explanations, and instructions
- Unless the user requests it, don't add explanations — just provide the prompt directly
---
## Anti-Patterns (Lessons Learned from Production)
**These mistakes have been made in real sessions. Never repeat them.**
### 1. SKIP THE SKILL WHEN ASKED TO USE IT ❌
- **Wrong**: User says "use the video-prompting skill" but I directly call sub-agents without following the skill steps
- **Right**: Always execute the full skill workflow — Step 1 through Step 5 (at minimum). If the user explicitly asks to use the skill, do it properly.
### 2. IGNORE THE USER'S SELECTED REFERENCE IMAGE ❌
- **Wrong**: User says "use this image" but I generate a new image or don't pass the image path to the video agent
- **Right**: When the user selects a specific image, ALWAYS pass that exact file path in `task_description` to the video agent. Use the full absolute path like `/Users/mac/Movies/Hub/Projects/.../image.jpg`. Never substitute with a newly generated image unless the user explicitly asks.
### 3. ASK ABOUT ASPECT RATIO TOO LATE ❌
- **Wrong**: Generate images first, then find out the user wanted a different aspect ratio
- **Right**: Ask about aspect ratio in **Step 2.5** — before any image or video generation happens. Lock it in and use it consistently.
### 4. REUSE THE SAME PROMPT ACROSS DIFFERENT MODELS ❌
- **Wrong**: Write a Veo3 prompt, get Veo3 timeout, then use the exact same prompt for Kling without rewriting
- **Right**: Each model has different prompt syntax, length limits, and style preferences. When falling back, go back to Step 3, re-read the new model's `prompting.md`, and rewrite the prompt from scratch.
### 5. FORGET TO PASS THE REFERENCE IMAGE PATH TO VIDEO AGENT ❌
- **Wrong**: Call the video agent with just the prompt but no `reference_image` or `first_frame_image` field in task_description
- **Right**: In I2V mode, the task_description MUST include:
- The **exact file path** of the reference image
- A clear instruction like "Use this image as the first frame for I2V generation"
- The prompt describing what motion/transition should happen from that starting frame
### 6. NOT CLEARLY COMMUNICATING MODEL FALLBACK TO USER ❌
- **Wrong**: Silently switch from Veo3 to Kling without telling the user
- **Right**: Always inform the user: "Veo3 is unavailable, falling back to Kling I2V" and confirm whether the result meets their expectations
@@ -0,0 +1,253 @@
# Image Generation Models - Prompting Guide
This guide covers all available image generation models, helping you choose the right model and write optimal prompts.
## Model Selection Quick Reference
| Need | Recommended Model | Tool Name |
|------|-------------------|-----------|
| General text-to-image / image editing | Gemini (default) | `nano_banana_image_generation` |
| Batch image generation | Gemini Batch | `nano_banana_batch_image_generation_v2` |
| Text embedded in images | Qwen Wanxiang | `qwen_image_generation` |
| Style transfer / editing | FLUX Kontext | `kontext_image_generation` |
| Face / character reference | Kling | `kling_image_generation` |
| Creative illustration / Logo / transparent background | OpenAI GPT-Image | `openai_image_generation` |
| Artistic quality | Midjourney | `midjourney_image_generation` |
| Many reference images (up to 14) | Seedream | `seedream_image_generation` |
---
## Gemini (nano_banana)
**Default image model** with the strongest overall capabilities.
### Model Variants
| Variant | Model ID | Features |
|---------|----------|----------|
| nano_banana | gemini-2.5-flash-image | Fast, good for iteration |
| nano_banana_2 | gemini-3-pro-image-preview | High quality |
| nano_banana_2_flash | gemini-3.1-flash-image-preview | Default, balances speed and quality |
### Prompt Style
Natural language descriptions, like talking to an artist:
```
A serene Japanese garden in autumn, with a wooden bridge over a koi pond.
Red and golden maple leaves float on the water surface. Soft morning mist
rises from the pond. Shot in the style of a landscape photograph, natural
lighting, shallow depth of field focusing on the bridge.
```
### Image Editing
Provide reference images via `image_paths`, then describe the desired changes in the prompt:
```
image_paths: [original.jpg]
prompt: "Change the background to a sunset beach scene, keep the person and their pose exactly the same"
```
### Resolution
Supports 1K / 2K / 4K, default 2K. 2K is sufficient for video first frames.
---
## Qwen Wanxiang (qwen)
**Best choice for embedding text in images**. Only supports image editing (reference image required).
### Use Cases
- Adding text titles to images
- Modifying existing text in images
- Poster / cover creation
### Prompt
```
image_paths: [background.jpg]
prompt: "Add the text 'Brightest Star in the Night Sky' in elegant calligraphy style at the center of the image, white text with subtle glow effect"
```
---
## FLUX Kontext (kontext)
**Best choice for style transfer and image editing**.
### Strengths
- Converting photos to specific art styles
- Changing art style while maintaining composition
- Multi-image reference fusion
### Prompt
```
image_paths: [photo.jpg]
prompt: "Transform this photo into Studio Ghibli anime style, maintain the same composition and character pose, soft watercolor textures, warm pastel colors"
```
### Aspect Ratios
Supports a rich set of aspect ratios: 1:1, 16:9, 9:16, 3:2, 2:3, 4:3, 3:4, 4:5, 5:4, 21:9, 7:4, 4:7
---
## Kling (kling)
**Best choice for face and character reference generation**.
### Reference Modes
| Mode | Use Case |
|------|----------|
| `subject` | Preserves overall appearance (clothing, pose, style) |
| `face` | Preserves facial features only (can change outfit, scene) |
### Prompt
```
# Face mode: Keep face, change scene
image_path: face_reference.jpg
reference_type: face
prompt: "A woman in a traditional red qipao standing in a lantern-lit ancient Chinese street at night, full body shot, elegant pose, cinematic lighting"
# Subject mode: Preserve overall appearance
image_path: character_reference.jpg
reference_type: subject
prompt: "The same person standing on a rooftop at sunset, wind blowing through their hair, dramatic sky background, cinematic wide shot"
```
### Batch Generation
`kling_batch_image_generation` supports independent prompt, reference image, reference mode, and aspect ratio for each image.
---
## OpenAI GPT-Image (openai)
**Best choice for creative illustrations, logos, and transparent backgrounds**.
### Special Capabilities
- Transparent background images (ideal for logos, stickers, UI elements)
- Creative concept visualization
- Abstract art
### Prompt
```
prompt: "A minimalist logo of a phoenix rising from flames, clean vector style, bold lines, gradient from deep red to bright gold"
size: "1024x1024"
quality: "high"
```
### Image Editing
```
image_paths: [original.jpg]
prompt: "Remove the background and make it transparent, keep only the main subject"
```
---
## Midjourney (midjourney)
**Best choice for artistic quality and stylization**.
### Version Selection
Append version flag at the end of the prompt:
- `--v 7`: V7 (default), realistic and general purpose
- `--niji 7`: Niji7, anime/manga style
### Core Parameters
| Parameter | Function | Example |
|-----------|----------|---------|
| `--ar` | Aspect ratio | `--ar 16:9` |
| `--sref` | Style reference URL | `--sref https://example.com/style.jpg` |
| `--cref` | Character reference URL | `--cref https://example.com/char.jpg` |
| `--cw` | Character reference weight (0-100) | `--cw 50` |
### Prompt Style
Midjourney favors **concise, evocative descriptions** — no need to be overly verbose:
```
prompt: "ancient samurai meditating under cherry blossoms, moonlit courtyard, atmospheric fog, cinematic lighting, dramatic shadows --ar 16:9 --v 7"
```
### Reference Images
Provide local reference images via `image_paths`:
```
image_paths: [reference.jpg]
prompt: "a warrior in similar armor style standing on a cliff overlooking a vast ocean, epic scale, golden hour --ar 16:9 --v 7"
```
---
## Seedream (seedream)
**Supports the most reference images** — version 4.5 supports up to 14 reference images.
### Model Selection
| Model | Features |
|-------|----------|
| doubao-seedream-4-0-250828 | Standard version |
| doubao-seedream-4-5-251128 | New version, supports up to 14 reference images |
### Ideal Use Cases
- Scenarios requiring many reference images to constrain style
- Multi-character consistency (provide multiple character reference images)
- Complex scene reconstruction
### Prompt
```
image_paths: [ref1.jpg, ref2.jpg, ref3.jpg, ...]
prompt: "A group photo of these characters together in a modern office setting, natural lighting, professional photography style, everyone is smiling and looking at the camera"
```
---
## General Prompting Tips
### 1. Optimizing for Video First Frames
When generating images intended as video first frames:
```
✓ Character's mouth slightly open (helps with lip sync)
✓ Pose suggests motion (e.g., mid-step, mid-turn)
✓ Leave room for motion in the frame (don't push subject to edges)
✓ Avoid extreme perspective (video models struggle to maintain it)
✗ Don't generate overly static "passport photo" compositions
```
### 2. MV First Frame Batch Generation Strategy
```
1. Determine art style → select primary image generation model
2. Use Kling face mode to lock character facial consistency
3. Batch generate first frames for each scene with the selected model
4. Use Kontext to unify style (if inconsistencies exist)
5. After confirmation, send to video model for generation
```
### 3. Style Consistency
Tips for maintaining unified style across a set of images:
- Use the same style keyword suffix
- Use Midjourney `--sref` to lock style
- Use Kling `face` mode to lock character appearance
- Use Kontext with the same style reference image for all images
- In batch generation tools (Gemini batch / Kling batch), use the same style description suffix for each image
@@ -0,0 +1,125 @@
# Kling Video Generation - Prompting Guide
## Model Overview
| Attribute | Value |
|-----------|-------|
| Tool Name | `kling_video_generation` |
| Available Models | kling-v1-6, kling-v2-master, kling-v2-5-turbo, kling-v2-6 (default) |
| Duration | 5s or 10s |
| Modes | std (standard), pro (professional, supports sound) |
| Input Modes | T2V, I2V (`first_frame_image_path`) |
| Special Features | Pro mode can generate video with sound (`enable_sound: true`) |
## Core Principles
### 1. Prompt Style: Director's Instructions
Kling responds best to **structured, directorial commands**. Organize prompts in this order:
```
[Camera setup] + [Subject description] + [Action directives] + [Environment/Scene] + [Lighting] + [Emotion/Atmosphere] + [Technical parameters]
```
### 2. Pro Mode and Sound
When sound is needed in the video, you must set:
- `mode: "pro"`
- `enable_sound: true`
Sound type is automatically inferred from the visual content — audio cannot be precisely controlled via prompt. Works well for ambient and natural sound effects, not suitable for dialogue or music.
### 3. Negative Prompt
Kling supports `negative_prompt` to exclude unwanted elements:
```
negative_prompt: "blurry, distorted faces, extra fingers, deformed hands, low quality, watermark, text overlay"
```
## Best Practices
### Character Generation
Kling excels in **character consistency and facial quality**:
- Describe facial features and expressions in detail
- Specify clothing details to maintain consistency
- Facial expression changes are a Kling strength — use them confidently
- In I2V mode, the face quality of the first frame directly determines video quality
### Motion Control
- **Simple repetitive actions** work best: walking, turning head, waving, nodding
- **Complex full-body movements** (dance, fighting) require lower expectations
- Use **temporal adverbs** to control pacing: slowly, gradually, suddenly, gently
- Schedule 1 main action within 5s, 2-3 actions within 10s
### Camera Movement
Kling understands camera movements with high precision:
```
Static shot → Fixed camera, ideal for character performance
Slow pan right → Slow lateral movement, ideal for environment showcase
Dolly zoom → Hitchcock zoom, dramatic effect
Low angle shot → Looking up, adds grandeur
Over-the-shoulder → Over-shoulder shot, conversation scenes
Bird's eye view → Overhead view, grand scenes
```
### Style Control
Kling responds well to these style keywords:
```
cinematic, film noir, anime style, watercolor painting,
oil painting, photorealistic, vintage film, documentary,
music video style, fashion editorial, slow motion
```
## Common Pitfalls
1. **std mode has no sound**: Sound requires pro mode
2. **Don't mix contradictory styles**: e.g., "realistic anime style"
3. **Don't describe desired elements in negative_prompt**: Negative prompt is for exclusion only
4. **Character count**: More than 2 characters tends to cause confusion — limit to 1-2 people
5. **Text generation**: On-screen text is unreliable — avoid requesting text in prompts
## Example Prompts
### Character Close-up + Sound (5s, pro, T2V)
```
prompt: "Close-up portrait shot of a young woman sitting by a rainy window. She slowly lifts a cup of coffee to her lips, steam rising from the cup. Rain droplets streak down the glass behind her. Warm interior lighting contrasts with the cool blue exterior. Intimate, contemplative mood. Cinematic shallow depth of field."
negative_prompt: "blurry, distorted face, extra fingers, low quality, watermark"
mode: pro, enable_sound: true → Automatically generates rain and indoor ambient sounds
```
### Dynamic Scene (10s, I2V)
```
prompt: "Medium shot, a martial artist in traditional white robes performs slow tai chi movements in a misty bamboo forest at dawn. The camera slowly orbits around him as morning light filters through the bamboo canopy. Leaves gently fall around him. Serene, meditative atmosphere. Cinematic, anamorphic lens flare."
negative_prompt: "fast motion, blurry, distorted body, modern clothing"
```
### Natural Landscape + Sound (10s, pro, T2V)
```
prompt: "Establishing wide shot of a volcanic coastline at sunset. Waves crash against black lava rocks, sending spray into the golden light. The camera slowly pushes forward toward the horizon. Dramatic clouds streak across the sky. Epic, cinematic scale, National Geographic quality, HDR lighting."
negative_prompt: "people, text, watermark, low resolution"
mode: pro, enable_sound: true → Ocean waves and wind sounds
```
### Emotional Performance (5s, I2V)
```
prompt: "Close-up shot, a man receives an unexpected phone call. His expression transitions from surprise to overwhelming joy as tears well up in his eyes. He covers his mouth with one hand. Soft natural light from a nearby window. Shallow depth of field, intimate documentary style."
negative_prompt: "exaggerated expression, cartoon, anime, blurry"
```
@@ -0,0 +1,104 @@
# Official Hilo Video Generation - Prompting Guide
## Model Overview
| Attribute | Value |
|-----------|-------|
| Tool Name | `official_videos_generation` |
| Available Models | Official-Hilo-2.3-Fast, Official-Hilo-2.3, Official-Hilo-02 |
| Duration | 6s or 10s |
| Resolution | 768P, 1080P |
| Input Modes | I2V (first frame, recommended), I2V + last frame, pure T2V |
| Special Features | Background sound effects, automatic prompt optimization, lip sync (via `official_generate_video_based_on_image_and_audio`) |
## Core Principles
### 1. Always Use I2V Mode
**Always prioritize using a first frame image (`first_frame_images`) to drive generation**. Pure text-to-video quality is far inferior to image-to-video.
Workflow: Generate a high-quality first frame with an image model → Then generate video with Hilo.
### 2. Prompt Structure
Hilo prompts should read like **a natural language shot description**, not a keyword dump:
```
[Shot type and movement], [Subject and their action], [Environment details], [Lighting and atmosphere], [Style modifiers]
```
### 3. Automatic Prompt Optimization
`enable_prompt_optimize` is enabled by default — Hilo will automatically expand and optimize your prompt. If you've already written a very precise prompt, you can disable this feature for more precise control.
## Best Practices
### Camera Language
Explicitly specify shot types and camera movements:
| Shot Type | English Expression |
|-----------|-------------------|
| Close-up | Close-up shot, extreme close-up |
| Medium shot | Medium shot, waist-up shot |
| Wide shot | Wide shot, establishing shot |
| Long shot | Long shot, aerial view |
| Tracking | Tracking shot, following shot |
| Push/Pull | Dolly in / dolly out, zoom in / zoom out |
| Pan | Pan left / pan right |
| Crane | Crane shot, tilt up / tilt down |
| Orbit | Orbit shot, 360-degree rotation |
| Handheld | Handheld camera, shaky cam |
| Static | Static shot, locked-off camera |
### Action Description
- Use **progressive tense** for continuous actions: `A woman is walking through...`
- Use **action sequences** for changes: `She turns her head slowly, then breaks into a smile`
- Control action intensity: Only 1-2 actions within 6s, 2-3 actions within 10s
- Avoid overly intense full-body movements (e.g., running, dancing) — they tend to cause distortion
### I2V + Last Frame (Official-Hilo-02 exclusive)
`last_frame_images` is only supported by Hilo-02. Ideal for:
- Scene transitions (day → night)
- Expression changes (calm → smile)
- Object transformations (bud → bloom)
### Background Sound
Setting `enable_background_sound: true` automatically generates ambient sound effects matching the visuals (wind, water flow, city noise, etc.). Suitable for natural scenes that don't need music.
## Common Pitfalls
1. **Don't use negative statements**: The model cannot understand "no people" — describe what you want instead
2. **Don't overload adjectives**: 3-5 style words are enough; too many will conflict
3. **Don't cram too many actions into 6s**: One clear action beats three vague ones
4. **Don't skip the first frame**: Pure T2V quality is unreliable — always provide a first frame
5. **Text content**: The model cannot reliably generate on-screen text
## Example Prompts
### Character Close-up (6s, I2V)
```
Close-up shot, a young woman with long black hair slowly turns her head toward the camera, her expression shifts from contemplative to a gentle smile. Soft golden hour light illuminates her face from the side. Shallow depth of field, cinematic color grading, film grain.
```
### Aerial Landscape (10s, I2V)
```
Aerial drone shot slowly ascending over a misty mountain valley at dawn. Layers of fog drift between the peaks as the first rays of sunlight paint the ridges gold. The camera gradually tilts down to reveal a winding river below. Epic cinematic scale, 4K quality, nature documentary style.
```
### Scene Transition (6s, I2V + last frame, Hilo-02)
```
Smooth transition, the scene gradually shifts from warm afternoon light to cool blue twilight. Cherry blossom petals drift through the air as the lighting changes. The woman's silhouette remains centered in frame. Dreamy, ethereal atmosphere, soft focus background.
```
### Dynamic Action (10s, I2V)
```
Medium tracking shot, a street musician plays acoustic guitar on a rain-soaked city sidewalk at night. Neon reflections shimmer in puddles around his feet. Pedestrians with umbrellas pass in the background as bokeh lights create a dreamy atmosphere. Camera slowly orbits around the musician. Moody, cinematic, urban night photography style.
```
@@ -0,0 +1,144 @@
# Veo3 Video Generation - Prompting Guide
## Model Overview
| Attribute | Value |
|-----------|-------|
| Tool Name | `veo3_video_generation` |
| Duration | 8s (fixed) |
| Resolution | 720p, 1080p |
| Aspect Ratio | 16:9, 9:16 |
| Input Modes | T2V, I2V (first/last frame) |
| Special Features | **Built-in audio generation** (dialogue, ambient sound, sound effects), first frame + last frame transitions |
## Core Principles
### 1. Five-Section Prompt Structure
Veo3 responds best to the following five-section structure:
```
[Cinematography/Camera] + [Subject description] + [Action/Behavior] + [Scene/Environment] + [Style and Atmosphere]
```
Each section flows naturally into the next, forming a smooth descriptive passage.
### 2. Audio Is the Core Advantage
Veo3 **automatically generates audio that matches the visuals**, including:
- Dialogue and voices (write dialogue directly in the prompt)
- Ambient sound effects (wind, footsteps, city noise)
- Emotional music (atmospheric music, background score)
**Dialogue format**: Describe what characters say in natural language:
```
A woman turns to the camera and says "I've been waiting for this moment."
```
Or use quotation marks to indicate dialogue:
```
Two friends sitting at a cafe, one says "Did you hear about the news?" and the other responds with a surprised expression "No way, tell me everything."
```
### 3. First Frame + Last Frame Transitions
Veo3 supports specifying both first and last frame images simultaneously — the model will automatically generate a transition animation between the two frames. Ideal for:
- Time lapse (day → night)
- Seasonal changes (spring → winter)
- Aging effects
- Scene transitions
### 4. Fixed 8-Second Duration
All Veo3 videos are 8 seconds. Plan content complexity accordingly:
- 1-2 main actions/events
- 1 camera movement
- Suitable for a single plot point or atmospheric segment
## Best Practices
### Leverage Audio Fully
Veo3's biggest differentiator is audio. Actively describe sounds in your prompt:
```
✓ "The sound of rain pattering on the window, a piano melody plays softly in the background"
✓ "Birds chirping in the morning, leaves rustling in the gentle breeze"
✓ "The bustling noise of a Tokyo street market, vendors calling out prices"
✗ Don't assume the model will automatically generate the sound you want — be explicit
```
### Cinematography Terms
Veo3 has excellent understanding of professional cinematography terminology:
```
Anamorphic lens → Widescreen anamorphic lens
Rack focus → Shifting focus between subjects
Dutch angle → Tilted composition
Steadicam → Stabilized tracking shot
Whip pan → Fast pan
Pull focus → Pulling focus
Depth of field f/1.4 → Shallow depth of field
35mm / 50mm / 85mm → Lens focal lengths
```
### Dialogue Scenes
Veo3 is the only model that can reliably generate dialogue — take full advantage:
```
A documentary-style interview. A weathered fisherman sits on his boat at golden hour,
looking directly at the camera. He speaks with a gravelly voice: "The sea doesn't care
about your plans. You either learn her rhythm, or she teaches you the hard way."
Waves lap gently against the hull. Shot on 16mm film, warm color grading.
```
### Excluding Unwanted Elements
Veo3 has no negative_prompt parameter. Exclude elements by **describing the desired state**:
```
✗ "no people in the scene"
✓ "an empty, deserted street with no pedestrians visible"
✗ "no text or watermark"
✓ "clean frame, pure cinematic imagery"
```
## Common Pitfalls
1. **Duration is not adjustable**: Fixed at 8s — don't try to specify "10 second clip" in the prompt
2. **Don't neglect audio description**: No audio description = random audio that may not match expectations
3. **Don't overload**: Don't cram too much plot into 8s
4. **Character consistency**: Characters may swap features in multi-person scenes — limit to 1-2 people
5. **Text rendering**: On-screen text is unreliable
## Example Prompts
### Cinematic Dialogue (8s, T2V)
```
Medium shot, a detective in a dark trench coat stands in a dimly lit alley. Rain pours down around him. He takes a long drag from a cigarette, the ember glowing orange, then exhales slowly and says in a low voice: "She was already gone by the time I got there." Noir lighting with harsh shadows from a single streetlamp. The sound of rain and distant sirens. Shot on 35mm film, desaturated color palette, neo-noir style.
```
### Nature Documentary (8s, I2V first frame)
```
Extreme close-up macro shot of a monarch butterfly emerging from its chrysalis. The wings slowly unfurl, revealing vibrant orange and black patterns still glistening with moisture. Soft morning light illuminates translucent wing membranes. The subtle sound of the membrane stretching and the ambient forest soundscape with distant bird calls. National Geographic quality, shallow depth of field, awe-inspiring nature documentary.
```
### First Frame → Last Frame Transition (8s, first + last frame)
```
Time-lapse transition of a city intersection. The scene smoothly transforms from a bustling daytime rush hour with honking cars and pedestrians crossing, to a quiet midnight scene with empty streets reflecting neon signs. Street lights flicker on as the sky transitions through sunset colors to deep blue. The audio shifts from chaotic city noise to serene nighttime ambiance with distant music from a bar.
```
### Vertical Short Video (8s, 9:16)
```
Vertical format, POV shot of hands opening an old wooden jewelry box. Inside, a golden locket catches the light. Fingers carefully lift the locket and open it, revealing a faded black-and-white photograph inside. A soft gasp is heard. Warm, intimate lighting from a bedside lamp. The gentle creak of the box hinge and a nostalgic music box melody begins to play. Vintage, sentimental, ASMR-quality close-up.
```
@@ -0,0 +1,143 @@
# Wan 2.6 Video Generation - Prompting Guide
## Model Overview
| Attribute | Value |
|-----------|-------|
| Tool Name | `wan_i2v_generation` (single) / `batch_wan_i2v_generation` (batch) |
| Duration | 5s, 10s, 15s |
| Resolution | 720P, 1080P |
| Input Modes | I2V (first frame image required) |
| Special Features | Audio-driven video generation (lip sync), multi-shot mode |
## Core Principles
### 1. Pure I2V Model
Wan 2.6 **requires a first frame image** (`image_path`) — pure text-to-video is not supported. The prompt's role is to guide how the first frame comes to life.
### 2. Basic Prompt Formula
```
[Subject] + [Action] + [Scene/Environment]
```
Advanced version:
```
[Subject] + [Action] + [Scene/Environment] + [Aesthetic control] + [Stylization]
```
### 3. Audio-Driven Generation (Core MV Scenario)
Provide an audio file via `audio_path`, and Wan will generate video driven by the audio's rhythm and vocal content:
- **Lip sync**: First frame of a speaking/singing character + corresponding audio clip
- **Rhythm matching**: First frame of dance/movement + music clip
- **Atmosphere linking**: Environment scene + background music
Batch generation (`batch_wan_i2v_generation`) supports up to 3 concurrent jobs — **the core tool for MV production**.
### 4. Single vs Multi Shot
- `shot_type: "single"` (default): Single shot — the entire video is one continuous shot
- `shot_type: "multi"`: Multi-shot — the model may switch between different angles/framings within the video
MV scenes typically use single, unless you need to simulate rapid editing.
## Best Practices
### Prompt Style: Concise and Direct
Wan works best with **short, direct prompts** — long descriptions are not needed:
```
✓ "A woman singing emotionally, close-up, tears in her eyes, soft lighting"
✗ "In this beautifully composed cinematic masterpiece, we see an incredibly talented young woman who appears to be in her late twenties, performing a deeply emotional song..."
```
### Action Verbs
Use **clear action verbs** to drive motion:
| Category | Example Verbs |
|----------|---------------|
| Character actions | walks, runs, turns, reaches, dances, sings, speaks |
| Camera movement | camera pans, camera zooms in, camera orbits, camera tilts up |
| Environment motion | wind blows, water flows, clouds drift, leaves fall, fire flickers |
| Lighting changes | light fades, sunlight breaks through, shadows shift |
### Camera Movement Modifiers
```
slow camera push in → Slow push forward
smooth tracking shot → Smooth tracking
handheld camera movement → Handheld shake
static locked-off camera → Completely static
gentle camera sway → Subtle swaying (simulates breathing feel)
```
### Duration Planning
| Duration | Suitable Content | Action Complexity |
|----------|-----------------|-------------------|
| 5s | Single expression change, simple gesture, static atmosphere | 1 action |
| 10s | Complete action sequence, scene atmosphere, simple interaction | 2-3 actions |
| 15s | Full narrative segment, multi-step action sequences | 3-4 actions |
### Audio-Driven Best Practices (MV Scenarios)
1. **Cut audio clips precisely**: Use `audio_subclip_batch` to pre-split by timestamp
2. **Character's mouth slightly open in first frame**: Produces more natural lip sync
3. **Audio duration ≤ video duration**: Audio cannot be longer than the video
4. **Moderate-speed segments work best**: Very fast rap may not sync properly
5. **Unified resolution for batch generation**: Makes subsequent stitching easier
## Common Pitfalls
1. **First frame image is required**: Missing the first frame will cause an error
2. **Don't write overly long prompts**: Under 50 words works best
3. **Don't describe what's already in the first frame**: The model already sees it — focus on describing **changes and motion**
4. **Don't write contradictory actions with audio**: If the audio is a quiet piano piece, don't write "energetic dancing"
5. **Quality may drop in the second half of 15s videos**: Place important content in the first 10s
## Example Prompts
### MV Lip Sync (5s, I2V + audio)
```
image_path: [singer close-up first frame]
audio_path: [corresponding song clip]
prompt: "Close-up, a woman singing passionately, subtle head movements, emotional expression, cinematic lighting"
duration: 5
```
### Atmospheric B-Roll (10s, I2V)
```
image_path: [rainy city street first frame]
prompt: "Rain falling on empty street, reflections shimmering on wet asphalt, camera slowly moves forward, moody atmospheric lighting, neon signs flickering"
duration: 10
```
### Character Action (10s, I2V)
```
image_path: [woman in traditional dress standing first frame]
prompt: "Woman in traditional dress turns around gracefully, her long sleeves flowing in the wind, cherry blossoms falling around her, slow motion, ethereal atmosphere"
duration: 10
```
### MV Batch Generation Example
```python
# batch_wan_i2v_generation parameter example
image_paths: [scene1.jpg, scene2.jpg, scene3.jpg]
audio_paths: [clip1.wav, clip2.wav, clip3.wav]
prompts: [
"Singer performing emotionally, close-up, tears rolling down, soft warm lighting",
"Wide shot, figure walking alone on moonlit beach, waves crashing, melancholic atmosphere",
"Two hands reaching for each other in slow motion, golden hour backlight, romantic"
]
durations: [5, 10, 5]
resolution: "1080P"
```
@@ -0,0 +1,125 @@
# Character Sheet Workflow
## Overview
Character Sheets are a key tool for maintaining character consistency in MV production. By generating standardized character reference images, you ensure uniform character appearance across all subsequent scenes.
## Character Sheet Types
### 1. Turnaround Sheet
Shows the character from multiple angles: front, 3/4 view, side, and back.
```
Prompt template:
"Character turnaround sheet of [character description], showing front view, 3/4 view, side view, and back view. [clothing description]. Clean white background, consistent lighting, full body, character design reference sheet, anime/realistic style."
```
### 2. Expression Sheet
Shows the character in different emotional states.
```
Prompt template:
"Expression sheet of [character description], showing 6 different emotions: neutral, happy, sad, angry, surprised, contemplative. Same angle (front-facing), consistent lighting, bust shot, clean layout, character reference sheet."
```
### 3. Pose Sheet
Shows the character in different action poses.
```
Prompt template:
"Pose reference sheet of [character description] in [clothing description], showing 4 dynamic poses: standing, walking, sitting, [specific action]. Full body, clean white background, consistent proportions, action reference sheet."
```
## Workflow
### Step 1: Define Character Description
Write a detailed character text description including:
- **Body type**: Height, build (slim, athletic, curvy...)
- **Facial features**: Face shape, eye color, hairstyle and hair color
- **Clothing**: Detailed outfit description
- **Aura**: Overall impression/vibe
Example:
```
A young Chinese woman in her mid-20s with delicate features. She has long straight
black hair reaching her waist, almond-shaped dark brown eyes, and fair skin.
She wears a modern hanfu-inspired outfit: a white silk top with subtle floral
embroidery and a flowing navy blue pleated skirt. Elegant and ethereal presence.
```
### Step 2: Select Generation Model
| Style Requirement | Recommended Model | Reason |
|-------------------|-------------------|--------|
| Realistic characters | Kling (face mode) | Strongest facial consistency |
| Anime characters | Midjourney (--niji 7) | Highest anime quality |
| Concept design | Gemini / OpenAI | Fast iteration |
| Stylized | Kontext | Can convert from photos to style |
### Step 3: Generate Character Sheets
1. **Generate the front-facing base image first**:
```
"Portrait of [character description], front-facing, neutral expression, soft studio lighting,
clean background, character reference, high detail"
```
2. **Using the base image as reference, generate turnaround sheet**:
```
image_paths: [base image]
prompt: "Character turnaround sheet based on this character..."
```
3. **Generate expression sheet**:
```
image_paths: [base image]
prompt: "Expression sheet of this same character..."
```
### Step 4: Verify Consistency
After generation, check:
- [ ] Are facial features consistent across all angles?
- [ ] Is the hairstyle logical from all angles?
- [ ] Are clothing details maintained consistently?
- [ ] Are body proportions stable?
- [ ] Is the overall style unified?
Fix inconsistencies using Kontext or Kling.
### Step 5: Output for Downstream Use
Character sheet applications:
- **Video first frame generation**: Use Kling face mode + scene description
- **Storyboard generation**: Use as character reference images
- **Lip sync**: Provide front-facing close-up as first frame
## Photorealistic Character Design
When "real-person photo" quality is needed for character design:
```
Prompt template:
"Professional photography headshot of [character description], [expression],
shot on Canon EOS R5, 85mm f/1.4 lens, natural window light,
clean neutral background, magazine quality portrait, photorealistic,
8K detail, skin texture visible, catchlight in eyes."
```
Key elements:
- Specify camera and lens parameters to increase realism
- Describe light direction and quality
- Mention skin texture and catchlight
- Use photography terminology rather than painting terminology
## MV Character Design Best Practices
1. **Prepare for lip sync**: Front-facing character close-ups should have mouth slightly open or in a speech-ready state
2. **Multiple outfits**: If the MV has scene changes, prepare corresponding outfit versions for each scene
3. **Emotional range**: MVs require rich expressions — ensure the expression sheet covers all emotions in the song
4. **First-frame friendly**: Character image composition should work as a video first frame (leave room for motion)
5. **Save all prompts**: Record the prompt used for each generated image for easy reuse and adjustment