Reorganize skills package structure
This commit is contained in:
@@ -0,0 +1,7 @@
|
||||
AI 视频/图像提示词工程专家。当用户需要为 AI 视频或图像生成工具
|
||||
编写或优化提示词时触发。
|
||||
帮助用户为多种模型撰写高质量生成提示词，
|
||||
包括 Kling、Veo3、Wan、Gemini、Midjourney、Seedance 及各类专有模型等，
|
||||
并融入各模型对应的最佳实践与格式要求。
|
||||
触发词包括：video prompt、image prompt、prompt engineering、
|
||||
how to write prompts、optimize prompt、prompt writing。
|
||||
@@ -0,0 +1,235 @@
|
||||
---
|
||||
name: video-prompting
|
||||
display-name-zh: 提示词专家
|
||||
summary-cn: 优化或撰写AI视频图像生成的提示词
|
||||
summary-en: Craft AI video & image prompts
|
||||
description: |
|
||||
AI video/image prompt engineering expert. Triggered when users need to write or optimize
|
||||
prompts for AI video or image generation tools.
|
||||
Helps users craft high-quality generation prompts for various models
|
||||
(Kling, Veo3, Wan, Gemini, Midjourney, Seedance, proprietary models, etc.),
|
||||
incorporating model-specific best practices and formatting requirements.
|
||||
Trigger phrases: video prompt, image prompt, prompt engineering,
|
||||
how to write prompts, optimize prompt, prompt writing.
|
||||
version: 0.1.4
|
||||
tags: [Text, Prompt, Video-Prompt, Image-Prompt]
|
||||
tags-cn: [文字, 提示词, 视频提示词, 图片提示词]
|
||||
exported-by: MiniMax-hub
|
||||
---
|
||||
|
||||
# Video Prompting - AI Video & Image Prompt Expert
|
||||
|
||||
You are a professional AI video/image prompt engineer, helping users craft high-quality prompts for various generation models.
|
||||
|
||||
## Workflow
|
||||
|
||||
### Step 1: Route the Request
|
||||
|
||||
Determine the user's request type and whether a reference image is provided:
|
||||
|
||||
| Situation | Route |
|
||||
|-----------|-------|
|
||||
| User provides a reference image | → Step 2 (use I2V mode) |
|
||||
| User mentions a specific image they want to use | → Step 2 (use I2V mode) |
|
||||
| User has no reference image | → Step 2 (use T2V mode, generate image first if needed) |
|
||||
| Unsure whether user wants T2V or I2V | → **Ask the user**: "Do you have a reference image? Or should I generate one for T2V?" |
|
||||
| Image prompt | → Load `references/models/image-models/prompting.md` |
|
||||
| Character design / Character Sheet | → Load `references/workflows/character-sheets.md` |
|
||||
| Prompt optimization / rewriting | → Step 2 (identify model first, then rewrite per guidelines) |
|
||||
|
||||
**Default rule**: If the user has NOT provided a reference image, default to T2V (text-to-video). Only switch to I2V when the user explicitly provides or selects an image.
|
||||
|
||||
### Step 2.5: Confirm Aspect Ratio
|
||||
|
||||
**CRITICAL — Always ask the user about aspect ratio BEFORE generating anything.** Different aspect ratios produce completely different compositions and framing. Never assume.
|
||||
|
||||
Ask the user directly:
|
||||
> "What aspect ratio do you want? (16:9 landscape / 9:16 vertical)"
|
||||
|
||||
**Supported aspect ratios only — do NOT offer 1:1, 4:5, or 2.35:1 as options unless confirmed by the user:**
|
||||
|
||||
| Aspect Ratio | Use Case |
|
||||
|---|---|
|
||||
| **16:9 landscape** | YouTube / TV commercial / cinematic |
|
||||
| **9:16 vertical** | TikTok / Reels / Short drama |
|
||||
|
||||
If the user asks for 1:1, 4:5, or other unsupported ratios:
|
||||
> "Sorry, the current video models only support 16:9 (landscape) and 9:16 (vertical). 1:1 is not currently available. Would you like to use 16:9 or 9:16 instead?"
|
||||
|
||||
If the user doesn't specify, recommend based on use case:
|
||||
- **YouTube / TV commercial** → 16:9 landscape
|
||||
- **Short drama / TikTok / Reels** → 9:16 vertical
|
||||
|
||||
Record the confirmed aspect ratio and use it consistently for both the first frame image AND the video generation.
|
||||
|
||||
### Step 2: Identify Model and Input Mode
|
||||
|
||||
Determine the target model and input mode based on user specification or context:
|
||||
|
||||
| Model | Tool Name | Input Modes | Reference File |
|
||||
|-------|-----------|-------------|----------------|
|
||||
| Official Hilo | `official_videos_generation` | I2V (first/last frame), T2V | `references/models/official-hilo/prompting.md` |
|
||||
| Kling | `kling_video_generation` | T2V, I2V, pro+sound | `references/models/kling/prompting.md` |
|
||||
| Veo3 | `veo3_video_generation` | T2V, I2V (first/last frame) | `references/models/veo3/prompting.md` |
|
||||
| Wan 2.6 | `wan_i2v_generation` | I2V + optional audio | `references/models/wan/prompting.md` |
|
||||
|
||||
If the user hasn't specified a model, recommend based on their needs:
|
||||
- **Default video generation** → Official Hilo (most versatile)
|
||||
- **Needs sound** → Kling pro mode or Veo3
|
||||
- **Audio-driven lip sync** → Wan I2V + audio or Official image+audio
|
||||
- **High-quality cinematic** → Veo3
|
||||
- **Strong character consistency** → Kling
|
||||
|
||||
### Step 3: Load Model Reference
|
||||
|
||||
Read the corresponding model's `prompting.md` reference file to obtain:
|
||||
- Model-specific prompt structure and formatting requirements
|
||||
- Best practices and common pitfalls
|
||||
- Example prompts
|
||||
|
||||
### Step 4: Write the Prompt
|
||||
|
||||
Write the prompt following model guidelines and these universal principles:
|
||||
|
||||
#### Universal Prompt Structure
|
||||
|
||||
```
|
||||
[Subject description] + [Action/Motion] + [Environment/Scene] + [Lighting/Atmosphere] + [Camera language] + [Style modifiers]
|
||||
```
|
||||
|
||||
#### Universal Best Practices
|
||||
|
||||
1. **Use English**: All video/image models perform best with English prompts
|
||||
2. **Be specific, not abstract**: Use "A woman in a red silk dress" instead of "a beautiful woman"
|
||||
3. **Describe action progression**: Use beats to describe the start → development → end of actions
|
||||
4. **Specify camera movement**: camera pan left, dolly in, static shot, tracking shot...
|
||||
5. **Include lighting description**: golden hour, neon lights, dramatic shadows, soft diffused light...
|
||||
6. **Add style tags**: cinematic, film grain, 35mm film, anamorphic, documentary style...
|
||||
7. **Use positive phrasing**: Don't say "no blur" — say "sharp focus, crystal clear"
|
||||
8. **Match complexity to duration**: Short clips (5s) use simple actions, longer clips (10-15s) can use complex narratives
|
||||
9. **Lock identity features**: In I2V mode, the first frame already defines appearance — focus the prompt on action and atmosphere
|
||||
|
||||
### Step 5: Output — Present Prompt for Confirmation
|
||||
|
||||
**ALWAYS present the written prompt to the user before generating any video.** Do not proceed to generation without explicit user approval.
|
||||
|
||||
Output format:
|
||||
1. The prompt as a single copyable code block (ready to paste into any tool)
|
||||
2. A brief annotation table explaining the key sections (camera, action, audio, style)
|
||||
3. **Reference image**: Always include the exact file path of the image the user selected or approved. Never omit this.
|
||||
4. Ask: "Confirm to generate, or would you like to adjust anything?"
|
||||
|
||||
If the user requests, also provide:
|
||||
- Multi-version comparison (variants with different styles/angles)
|
||||
- Batch prompts (a set of prompts for multiple scenes)
|
||||
|
||||
**Why this matters**: The prompt is the creative brief — showing it first lets the user catch mismatches in direction, mood, or emphasis before spending generation credits.
|
||||
|
||||
### Step 6: Generation (Optional)
|
||||
|
||||
If the user requests direct video/image generation, follow this workflow:
|
||||
|
||||
#### 6.1 First Frame / T2V Generation
|
||||
|
||||
**Rule**: Only generate a first frame image when using I2V mode. For T2V (no reference image provided by the user), skip this step and go directly to video generation.
|
||||
|
||||
When to generate a first frame:
|
||||
- User has provided/selected a reference image → Use that image for I2V ✅
|
||||
- User wants I2V but has no image yet → Generate a first frame image first, then confirm with user before video
|
||||
|
||||
When to skip first frame:
|
||||
- User has no reference image and wants T2V → Go directly to Step 6.2
|
||||
|
||||
Recommended model for first frame: Gemini (`nano_banana_image_generation`), 2K resolution.
|
||||
|
||||
**CRITICAL — Always specify exact pixel dimensions in the task_description based on the confirmed aspect ratio:**
|
||||
|
||||
| Aspect Ratio | Dimensions |
|
||||
|---|---|
|
||||
| 16:9 landscape | 1920×1080 or 1376×768 |
|
||||
| 9:16 vertical | 1080×1920 or 768×1376 |
|
||||
| 1:1 square | 1024×1024 (only if explicitly requested and model confirms support) |
|
||||
|
||||
Do NOT just say "16:9" in the prompt — explicitly state the pixel dimensions (e.g., "1920×1080 pixels") in the task_description to the image agent, or the agent may default to its own preferred ratio (often 2:3).
|
||||
|
||||
First frame prompt tips:
|
||||
- Describe **the moment just before action occurs** (e.g., "about to pour wine" not "pouring wine")
|
||||
- Leave room for motion in the frame — don't push the subject to the edges
|
||||
- Include environmental details to provide context for the video
|
||||
|
||||
Show the first frame to the user for confirmation before proceeding to video generation.
|
||||
|
||||
#### 6.2 Video Generation and Model Fallback
|
||||
|
||||
Video generation services can be unstable. Use the following **fallback chain** to automatically switch models:
|
||||
|
||||
```
|
||||
Preferred model (user-specified or recommended) → Fallback → Secondary fallback
|
||||
```
|
||||
|
||||
Fallback chains:
|
||||
|
||||
| Scenario | Fallback Chain |
|
||||
|----------|---------------|
|
||||
| Needs sound | Veo3 → Hilo(enable_background_sound) |
|
||||
| General video (16:9 / 9:16) | Veo3 → Hilo |
|
||||
| Lip sync | Wan I2V+audio → Hilo image+audio |
|
||||
|
||||
**Important model-specific notes for fallback:**
|
||||
|
||||
- **Kling API requires a reference image even for T2V mode** — if Kling is in the fallback chain and user has no image, skip Kling entirely and go to the next model
|
||||
- **Veo3 only supports 16:9 and 9:16** — do NOT include models that cannot generate the user's confirmed aspect ratio in the fallback chain
|
||||
- **Official Hilo is excluded from all fallback chains** — it does not support aspect ratio selection and has limited controls. It should only be used as an absolute last resort when all other models have failed.
|
||||
|
||||
Fallback rules:
|
||||
- **Retry the same model once first**: When a model returns timeout or service unavailable (500/provider failed), **retry the same model one more time before falling back**. Many API errors are transient. Only switch to the next model after a second failure.
|
||||
- Inform the user that the current model is unavailable and you're retrying
|
||||
- **CRITICAL — Re-read the new model's `prompting.md` before generating again**: Each model has different prompt syntax, style preferences, length limits, and format requirements. The prompt written for one model is NOT directly transferable. Go back to Step 3, load the fallback model's reference file, and **rewrite the prompt from scratch** to match the new model's guidelines
|
||||
- Do NOT reuse the previous model's prompt when falling back — always regenerate
|
||||
- **CRITICAL — Always pass the reference image path to the video agent**: In the video agent's `task_description`, include the exact file path of the user's selected image (e.g., `/Users/mac/Movies/Hub/Projects/.../honeysuckle_product.jpg`) and clearly state "Use this image as the I2V first frame". Do NOT omit the image path or the agent will generate from scratch without using the user's chosen image.
|
||||
- If all models fail, save the first frame image and prompt, and tell the user to retry later
|
||||
|
||||
#### 6.3 Result Presentation
|
||||
|
||||
After successful video generation:
|
||||
- Use `read_media` to analyze video content (skip if timeout)
|
||||
- Provide the video file path to the user
|
||||
- Ask if they want to open a preview
|
||||
|
||||
## Output Control
|
||||
|
||||
- All output must be in **English**, including prompts, explanations, and instructions
|
||||
- Unless the user requests it, don't add explanations — just provide the prompt directly
|
||||
|
||||
---
|
||||
|
||||
## Anti-Patterns (Lessons Learned from Production)
|
||||
|
||||
**These mistakes have been made in real sessions. Never repeat them.**
|
||||
|
||||
### 1. SKIP THE SKILL WHEN ASKED TO USE IT ❌
|
||||
- **Wrong**: User says "use the video-prompting skill" but I directly call sub-agents without following the skill steps
|
||||
- **Right**: Always execute the full skill workflow — Step 1 through Step 5 (at minimum). If the user explicitly asks to use the skill, do it properly.
|
||||
|
||||
### 2. IGNORE THE USER'S SELECTED REFERENCE IMAGE ❌
|
||||
- **Wrong**: User says "use this image" but I generate a new image or don't pass the image path to the video agent
|
||||
- **Right**: When the user selects a specific image, ALWAYS pass that exact file path in `task_description` to the video agent. Use the full absolute path like `/Users/mac/Movies/Hub/Projects/.../image.jpg`. Never substitute with a newly generated image unless the user explicitly asks.
|
||||
|
||||
### 3. ASK ABOUT ASPECT RATIO TOO LATE ❌
|
||||
- **Wrong**: Generate images first, then find out the user wanted a different aspect ratio
|
||||
- **Right**: Ask about aspect ratio in **Step 2.5** — before any image or video generation happens. Lock it in and use it consistently.
|
||||
|
||||
### 4. REUSE THE SAME PROMPT ACROSS DIFFERENT MODELS ❌
|
||||
- **Wrong**: Write a Veo3 prompt, get Veo3 timeout, then use the exact same prompt for Kling without rewriting
|
||||
- **Right**: Each model has different prompt syntax, length limits, and style preferences. When falling back, go back to Step 3, re-read the new model's `prompting.md`, and rewrite the prompt from scratch.
|
||||
|
||||
### 5. FORGET TO PASS THE REFERENCE IMAGE PATH TO VIDEO AGENT ❌
|
||||
- **Wrong**: Call the video agent with just the prompt but no `reference_image` or `first_frame_image` field in task_description
|
||||
- **Right**: In I2V mode, the task_description MUST include:
|
||||
- The **exact file path** of the reference image
|
||||
- A clear instruction like "Use this image as the first frame for I2V generation"
|
||||
- The prompt describing what motion/transition should happen from that starting frame
|
||||
|
||||
### 6. NOT CLEARLY COMMUNICATING MODEL FALLBACK TO USER ❌
|
||||
- **Wrong**: Silently switch from Veo3 to Kling without telling the user
|
||||
- **Right**: Always inform the user: "Veo3 is unavailable, falling back to Kling I2V" and confirm whether the result meets their expectations
|
||||
+253
@@ -0,0 +1,253 @@
|
||||
# Image Generation Models - Prompting Guide
|
||||
|
||||
This guide covers all available image generation models, helping you choose the right model and write optimal prompts.
|
||||
|
||||
## Model Selection Quick Reference
|
||||
|
||||
| Need | Recommended Model | Tool Name |
|
||||
|------|-------------------|-----------|
|
||||
| General text-to-image / image editing | Gemini (default) | `nano_banana_image_generation` |
|
||||
| Batch image generation | Gemini Batch | `nano_banana_batch_image_generation_v2` |
|
||||
| Text embedded in images | Qwen Wanxiang | `qwen_image_generation` |
|
||||
| Style transfer / editing | FLUX Kontext | `kontext_image_generation` |
|
||||
| Face / character reference | Kling | `kling_image_generation` |
|
||||
| Creative illustration / Logo / transparent background | OpenAI GPT-Image | `openai_image_generation` |
|
||||
| Artistic quality | Midjourney | `midjourney_image_generation` |
|
||||
| Many reference images (up to 14) | Seedream | `seedream_image_generation` |
|
||||
|
||||
---
|
||||
|
||||
## Gemini (nano_banana)
|
||||
|
||||
**Default image model** with the strongest overall capabilities.
|
||||
|
||||
### Model Variants
|
||||
|
||||
| Variant | Model ID | Features |
|
||||
|---------|----------|----------|
|
||||
| nano_banana | gemini-2.5-flash-image | Fast, good for iteration |
|
||||
| nano_banana_2 | gemini-3-pro-image-preview | High quality |
|
||||
| nano_banana_2_flash | gemini-3.1-flash-image-preview | Default, balances speed and quality |
|
||||
|
||||
### Prompt Style
|
||||
|
||||
Natural language descriptions, like talking to an artist:
|
||||
|
||||
```
|
||||
A serene Japanese garden in autumn, with a wooden bridge over a koi pond.
|
||||
Red and golden maple leaves float on the water surface. Soft morning mist
|
||||
rises from the pond. Shot in the style of a landscape photograph, natural
|
||||
lighting, shallow depth of field focusing on the bridge.
|
||||
```
|
||||
|
||||
### Image Editing
|
||||
|
||||
Provide reference images via `image_paths`, then describe the desired changes in the prompt:
|
||||
|
||||
```
|
||||
image_paths: [original.jpg]
|
||||
prompt: "Change the background to a sunset beach scene, keep the person and their pose exactly the same"
|
||||
```
|
||||
|
||||
### Resolution
|
||||
|
||||
Supports 1K / 2K / 4K, default 2K. 2K is sufficient for video first frames.
|
||||
|
||||
---
|
||||
|
||||
## Qwen Wanxiang (qwen)
|
||||
|
||||
**Best choice for embedding text in images**. Only supports image editing (reference image required).
|
||||
|
||||
### Use Cases
|
||||
|
||||
- Adding text titles to images
|
||||
- Modifying existing text in images
|
||||
- Poster / cover creation
|
||||
|
||||
### Prompt
|
||||
|
||||
```
|
||||
image_paths: [background.jpg]
|
||||
prompt: "Add the text 'Brightest Star in the Night Sky' in elegant calligraphy style at the center of the image, white text with subtle glow effect"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## FLUX Kontext (kontext)
|
||||
|
||||
**Best choice for style transfer and image editing**.
|
||||
|
||||
### Strengths
|
||||
|
||||
- Converting photos to specific art styles
|
||||
- Changing art style while maintaining composition
|
||||
- Multi-image reference fusion
|
||||
|
||||
### Prompt
|
||||
|
||||
```
|
||||
image_paths: [photo.jpg]
|
||||
prompt: "Transform this photo into Studio Ghibli anime style, maintain the same composition and character pose, soft watercolor textures, warm pastel colors"
|
||||
```
|
||||
|
||||
### Aspect Ratios
|
||||
|
||||
Supports a rich set of aspect ratios: 1:1, 16:9, 9:16, 3:2, 2:3, 4:3, 3:4, 4:5, 5:4, 21:9, 7:4, 4:7
|
||||
|
||||
---
|
||||
|
||||
## Kling (kling)
|
||||
|
||||
**Best choice for face and character reference generation**.
|
||||
|
||||
### Reference Modes
|
||||
|
||||
| Mode | Use Case |
|
||||
|------|----------|
|
||||
| `subject` | Preserves overall appearance (clothing, pose, style) |
|
||||
| `face` | Preserves facial features only (can change outfit, scene) |
|
||||
|
||||
### Prompt
|
||||
|
||||
```
|
||||
# Face mode: Keep face, change scene
|
||||
image_path: face_reference.jpg
|
||||
reference_type: face
|
||||
prompt: "A woman in a traditional red qipao standing in a lantern-lit ancient Chinese street at night, full body shot, elegant pose, cinematic lighting"
|
||||
|
||||
# Subject mode: Preserve overall appearance
|
||||
image_path: character_reference.jpg
|
||||
reference_type: subject
|
||||
prompt: "The same person standing on a rooftop at sunset, wind blowing through their hair, dramatic sky background, cinematic wide shot"
|
||||
```
|
||||
|
||||
### Batch Generation
|
||||
|
||||
`kling_batch_image_generation` supports independent prompt, reference image, reference mode, and aspect ratio for each image.
|
||||
|
||||
---
|
||||
|
||||
## OpenAI GPT-Image (openai)
|
||||
|
||||
**Best choice for creative illustrations, logos, and transparent backgrounds**.
|
||||
|
||||
### Special Capabilities
|
||||
|
||||
- Transparent background images (ideal for logos, stickers, UI elements)
|
||||
- Creative concept visualization
|
||||
- Abstract art
|
||||
|
||||
### Prompt
|
||||
|
||||
```
|
||||
prompt: "A minimalist logo of a phoenix rising from flames, clean vector style, bold lines, gradient from deep red to bright gold"
|
||||
size: "1024x1024"
|
||||
quality: "high"
|
||||
```
|
||||
|
||||
### Image Editing
|
||||
|
||||
```
|
||||
image_paths: [original.jpg]
|
||||
prompt: "Remove the background and make it transparent, keep only the main subject"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Midjourney (midjourney)
|
||||
|
||||
**Best choice for artistic quality and stylization**.
|
||||
|
||||
### Version Selection
|
||||
|
||||
Append version flag at the end of the prompt:
|
||||
- `--v 7`: V7 (default), realistic and general purpose
|
||||
- `--niji 7`: Niji7, anime/manga style
|
||||
|
||||
### Core Parameters
|
||||
|
||||
| Parameter | Function | Example |
|
||||
|-----------|----------|---------|
|
||||
| `--ar` | Aspect ratio | `--ar 16:9` |
|
||||
| `--sref` | Style reference URL | `--sref https://example.com/style.jpg` |
|
||||
| `--cref` | Character reference URL | `--cref https://example.com/char.jpg` |
|
||||
| `--cw` | Character reference weight (0-100) | `--cw 50` |
|
||||
|
||||
### Prompt Style
|
||||
|
||||
Midjourney favors **concise, evocative descriptions** — no need to be overly verbose:
|
||||
|
||||
```
|
||||
prompt: "ancient samurai meditating under cherry blossoms, moonlit courtyard, atmospheric fog, cinematic lighting, dramatic shadows --ar 16:9 --v 7"
|
||||
```
|
||||
|
||||
### Reference Images
|
||||
|
||||
Provide local reference images via `image_paths`:
|
||||
|
||||
```
|
||||
image_paths: [reference.jpg]
|
||||
prompt: "a warrior in similar armor style standing on a cliff overlooking a vast ocean, epic scale, golden hour --ar 16:9 --v 7"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Seedream (seedream)
|
||||
|
||||
**Supports the most reference images** — version 4.5 supports up to 14 reference images.
|
||||
|
||||
### Model Selection
|
||||
|
||||
| Model | Features |
|
||||
|-------|----------|
|
||||
| doubao-seedream-4-0-250828 | Standard version |
|
||||
| doubao-seedream-4-5-251128 | New version, supports up to 14 reference images |
|
||||
|
||||
### Ideal Use Cases
|
||||
|
||||
- Scenarios requiring many reference images to constrain style
|
||||
- Multi-character consistency (provide multiple character reference images)
|
||||
- Complex scene reconstruction
|
||||
|
||||
### Prompt
|
||||
|
||||
```
|
||||
image_paths: [ref1.jpg, ref2.jpg, ref3.jpg, ...]
|
||||
prompt: "A group photo of these characters together in a modern office setting, natural lighting, professional photography style, everyone is smiling and looking at the camera"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## General Prompting Tips
|
||||
|
||||
### 1. Optimizing for Video First Frames
|
||||
|
||||
When generating images intended as video first frames:
|
||||
|
||||
```
|
||||
✓ Character's mouth slightly open (helps with lip sync)
|
||||
✓ Pose suggests motion (e.g., mid-step, mid-turn)
|
||||
✓ Leave room for motion in the frame (don't push subject to edges)
|
||||
✓ Avoid extreme perspective (video models struggle to maintain it)
|
||||
✗ Don't generate overly static "passport photo" compositions
|
||||
```
|
||||
|
||||
### 2. MV First Frame Batch Generation Strategy
|
||||
|
||||
```
|
||||
1. Determine art style → select primary image generation model
|
||||
2. Use Kling face mode to lock character facial consistency
|
||||
3. Batch generate first frames for each scene with the selected model
|
||||
4. Use Kontext to unify style (if inconsistencies exist)
|
||||
5. After confirmation, send to video model for generation
|
||||
```
|
||||
|
||||
### 3. Style Consistency
|
||||
|
||||
Tips for maintaining unified style across a set of images:
|
||||
- Use the same style keyword suffix
|
||||
- Use Midjourney `--sref` to lock style
|
||||
- Use Kling `face` mode to lock character appearance
|
||||
- Use Kontext with the same style reference image for all images
|
||||
- In batch generation tools (Gemini batch / Kling batch), use the same style description suffix for each image
|
||||
@@ -0,0 +1,125 @@
|
||||
# Kling Video Generation - Prompting Guide
|
||||
|
||||
## Model Overview
|
||||
|
||||
| Attribute | Value |
|
||||
|-----------|-------|
|
||||
| Tool Name | `kling_video_generation` |
|
||||
| Available Models | kling-v1-6, kling-v2-master, kling-v2-5-turbo, kling-v2-6 (default) |
|
||||
| Duration | 5s or 10s |
|
||||
| Modes | std (standard), pro (professional, supports sound) |
|
||||
| Input Modes | T2V, I2V (`first_frame_image_path`) |
|
||||
| Special Features | Pro mode can generate video with sound (`enable_sound: true`) |
|
||||
|
||||
## Core Principles
|
||||
|
||||
### 1. Prompt Style: Director's Instructions
|
||||
|
||||
Kling responds best to **structured, directorial commands**. Organize prompts in this order:
|
||||
|
||||
```
|
||||
[Camera setup] + [Subject description] + [Action directives] + [Environment/Scene] + [Lighting] + [Emotion/Atmosphere] + [Technical parameters]
|
||||
```
|
||||
|
||||
### 2. Pro Mode and Sound
|
||||
|
||||
When sound is needed in the video, you must set:
|
||||
- `mode: "pro"`
|
||||
- `enable_sound: true`
|
||||
|
||||
Sound type is automatically inferred from the visual content — audio cannot be precisely controlled via prompt. Works well for ambient and natural sound effects, not suitable for dialogue or music.
|
||||
|
||||
### 3. Negative Prompt
|
||||
|
||||
Kling supports `negative_prompt` to exclude unwanted elements:
|
||||
|
||||
```
|
||||
negative_prompt: "blurry, distorted faces, extra fingers, deformed hands, low quality, watermark, text overlay"
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Character Generation
|
||||
|
||||
Kling excels in **character consistency and facial quality**:
|
||||
|
||||
- Describe facial features and expressions in detail
|
||||
- Specify clothing details to maintain consistency
|
||||
- Facial expression changes are a Kling strength — use them confidently
|
||||
- In I2V mode, the face quality of the first frame directly determines video quality
|
||||
|
||||
### Motion Control
|
||||
|
||||
- **Simple repetitive actions** work best: walking, turning head, waving, nodding
|
||||
- **Complex full-body movements** (dance, fighting) require lower expectations
|
||||
- Use **temporal adverbs** to control pacing: slowly, gradually, suddenly, gently
|
||||
- Schedule 1 main action within 5s, 2-3 actions within 10s
|
||||
|
||||
### Camera Movement
|
||||
|
||||
Kling understands camera movements with high precision:
|
||||
|
||||
```
|
||||
Static shot → Fixed camera, ideal for character performance
|
||||
Slow pan right → Slow lateral movement, ideal for environment showcase
|
||||
Dolly zoom → Hitchcock zoom, dramatic effect
|
||||
Low angle shot → Looking up, adds grandeur
|
||||
Over-the-shoulder → Over-shoulder shot, conversation scenes
|
||||
Bird's eye view → Overhead view, grand scenes
|
||||
```
|
||||
|
||||
### Style Control
|
||||
|
||||
Kling responds well to these style keywords:
|
||||
|
||||
```
|
||||
cinematic, film noir, anime style, watercolor painting,
|
||||
oil painting, photorealistic, vintage film, documentary,
|
||||
music video style, fashion editorial, slow motion
|
||||
```
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
1. **std mode has no sound**: Sound requires pro mode
|
||||
2. **Don't mix contradictory styles**: e.g., "realistic anime style"
|
||||
3. **Don't describe desired elements in negative_prompt**: Negative prompt is for exclusion only
|
||||
4. **Character count**: More than 2 characters tends to cause confusion — limit to 1-2 people
|
||||
5. **Text generation**: On-screen text is unreliable — avoid requesting text in prompts
|
||||
|
||||
## Example Prompts
|
||||
|
||||
### Character Close-up + Sound (5s, pro, T2V)
|
||||
|
||||
```
|
||||
prompt: "Close-up portrait shot of a young woman sitting by a rainy window. She slowly lifts a cup of coffee to her lips, steam rising from the cup. Rain droplets streak down the glass behind her. Warm interior lighting contrasts with the cool blue exterior. Intimate, contemplative mood. Cinematic shallow depth of field."
|
||||
|
||||
negative_prompt: "blurry, distorted face, extra fingers, low quality, watermark"
|
||||
|
||||
mode: pro, enable_sound: true → Automatically generates rain and indoor ambient sounds
|
||||
```
|
||||
|
||||
### Dynamic Scene (10s, I2V)
|
||||
|
||||
```
|
||||
prompt: "Medium shot, a martial artist in traditional white robes performs slow tai chi movements in a misty bamboo forest at dawn. The camera slowly orbits around him as morning light filters through the bamboo canopy. Leaves gently fall around him. Serene, meditative atmosphere. Cinematic, anamorphic lens flare."
|
||||
|
||||
negative_prompt: "fast motion, blurry, distorted body, modern clothing"
|
||||
```
|
||||
|
||||
### Natural Landscape + Sound (10s, pro, T2V)
|
||||
|
||||
```
|
||||
prompt: "Establishing wide shot of a volcanic coastline at sunset. Waves crash against black lava rocks, sending spray into the golden light. The camera slowly pushes forward toward the horizon. Dramatic clouds streak across the sky. Epic, cinematic scale, National Geographic quality, HDR lighting."
|
||||
|
||||
negative_prompt: "people, text, watermark, low resolution"
|
||||
|
||||
mode: pro, enable_sound: true → Ocean waves and wind sounds
|
||||
```
|
||||
|
||||
### Emotional Performance (5s, I2V)
|
||||
|
||||
```
|
||||
prompt: "Close-up shot, a man receives an unexpected phone call. His expression transitions from surprise to overwhelming joy as tears well up in his eyes. He covers his mouth with one hand. Soft natural light from a nearby window. Shallow depth of field, intimate documentary style."
|
||||
|
||||
negative_prompt: "exaggerated expression, cartoon, anime, blurry"
|
||||
```
|
||||
+104
@@ -0,0 +1,104 @@
|
||||
# Official Hilo Video Generation - Prompting Guide
|
||||
|
||||
## Model Overview
|
||||
|
||||
| Attribute | Value |
|
||||
|-----------|-------|
|
||||
| Tool Name | `official_videos_generation` |
|
||||
| Available Models | Official-Hilo-2.3-Fast, Official-Hilo-2.3, Official-Hilo-02 |
|
||||
| Duration | 6s or 10s |
|
||||
| Resolution | 768P, 1080P |
|
||||
| Input Modes | I2V (first frame, recommended), I2V + last frame, pure T2V |
|
||||
| Special Features | Background sound effects, automatic prompt optimization, lip sync (via `official_generate_video_based_on_image_and_audio`) |
|
||||
|
||||
## Core Principles
|
||||
|
||||
### 1. Always Use I2V Mode
|
||||
|
||||
**Always prioritize using a first frame image (`first_frame_images`) to drive generation**. Pure text-to-video quality is far inferior to image-to-video.
|
||||
|
||||
Workflow: Generate a high-quality first frame with an image model → Then generate video with Hilo.
|
||||
|
||||
### 2. Prompt Structure
|
||||
|
||||
Hilo prompts should read like **a natural language shot description**, not a keyword dump:
|
||||
|
||||
```
|
||||
[Shot type and movement], [Subject and their action], [Environment details], [Lighting and atmosphere], [Style modifiers]
|
||||
```
|
||||
|
||||
### 3. Automatic Prompt Optimization
|
||||
|
||||
`enable_prompt_optimize` is enabled by default — Hilo will automatically expand and optimize your prompt. If you've already written a very precise prompt, you can disable this feature for more precise control.
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Camera Language
|
||||
|
||||
Explicitly specify shot types and camera movements:
|
||||
|
||||
| Shot Type | English Expression |
|
||||
|-----------|-------------------|
|
||||
| Close-up | Close-up shot, extreme close-up |
|
||||
| Medium shot | Medium shot, waist-up shot |
|
||||
| Wide shot | Wide shot, establishing shot |
|
||||
| Long shot | Long shot, aerial view |
|
||||
| Tracking | Tracking shot, following shot |
|
||||
| Push/Pull | Dolly in / dolly out, zoom in / zoom out |
|
||||
| Pan | Pan left / pan right |
|
||||
| Crane | Crane shot, tilt up / tilt down |
|
||||
| Orbit | Orbit shot, 360-degree rotation |
|
||||
| Handheld | Handheld camera, shaky cam |
|
||||
| Static | Static shot, locked-off camera |
|
||||
|
||||
### Action Description
|
||||
|
||||
- Use **progressive tense** for continuous actions: `A woman is walking through...`
|
||||
- Use **action sequences** for changes: `She turns her head slowly, then breaks into a smile`
|
||||
- Control action intensity: Only 1-2 actions within 6s, 2-3 actions within 10s
|
||||
- Avoid overly intense full-body movements (e.g., running, dancing) — they tend to cause distortion
|
||||
|
||||
### I2V + Last Frame (Official-Hilo-02 exclusive)
|
||||
|
||||
`last_frame_images` is only supported by Hilo-02. Ideal for:
|
||||
- Scene transitions (day → night)
|
||||
- Expression changes (calm → smile)
|
||||
- Object transformations (bud → bloom)
|
||||
|
||||
### Background Sound
|
||||
|
||||
Setting `enable_background_sound: true` automatically generates ambient sound effects matching the visuals (wind, water flow, city noise, etc.). Suitable for natural scenes that don't need music.
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
1. **Don't use negative statements**: The model cannot understand "no people" — describe what you want instead
|
||||
2. **Don't overload adjectives**: 3-5 style words are enough; too many will conflict
|
||||
3. **Don't cram too many actions into 6s**: One clear action beats three vague ones
|
||||
4. **Don't skip the first frame**: Pure T2V quality is unreliable — always provide a first frame
|
||||
5. **Text content**: The model cannot reliably generate on-screen text
|
||||
|
||||
## Example Prompts
|
||||
|
||||
### Character Close-up (6s, I2V)
|
||||
|
||||
```
|
||||
Close-up shot, a young woman with long black hair slowly turns her head toward the camera, her expression shifts from contemplative to a gentle smile. Soft golden hour light illuminates her face from the side. Shallow depth of field, cinematic color grading, film grain.
|
||||
```
|
||||
|
||||
### Aerial Landscape (10s, I2V)
|
||||
|
||||
```
|
||||
Aerial drone shot slowly ascending over a misty mountain valley at dawn. Layers of fog drift between the peaks as the first rays of sunlight paint the ridges gold. The camera gradually tilts down to reveal a winding river below. Epic cinematic scale, 4K quality, nature documentary style.
|
||||
```
|
||||
|
||||
### Scene Transition (6s, I2V + last frame, Hilo-02)
|
||||
|
||||
```
|
||||
Smooth transition, the scene gradually shifts from warm afternoon light to cool blue twilight. Cherry blossom petals drift through the air as the lighting changes. The woman's silhouette remains centered in frame. Dreamy, ethereal atmosphere, soft focus background.
|
||||
```
|
||||
|
||||
### Dynamic Action (10s, I2V)
|
||||
|
||||
```
|
||||
Medium tracking shot, a street musician plays acoustic guitar on a rain-soaked city sidewalk at night. Neon reflections shimmer in puddles around his feet. Pedestrians with umbrellas pass in the background as bokeh lights create a dreamy atmosphere. Camera slowly orbits around the musician. Moody, cinematic, urban night photography style.
|
||||
```
|
||||
@@ -0,0 +1,144 @@
|
||||
# Veo3 Video Generation - Prompting Guide
|
||||
|
||||
## Model Overview
|
||||
|
||||
| Attribute | Value |
|
||||
|-----------|-------|
|
||||
| Tool Name | `veo3_video_generation` |
|
||||
| Duration | 8s (fixed) |
|
||||
| Resolution | 720p, 1080p |
|
||||
| Aspect Ratio | 16:9, 9:16 |
|
||||
| Input Modes | T2V, I2V (first/last frame) |
|
||||
| Special Features | **Built-in audio generation** (dialogue, ambient sound, sound effects), first frame + last frame transitions |
|
||||
|
||||
## Core Principles
|
||||
|
||||
### 1. Five-Section Prompt Structure
|
||||
|
||||
Veo3 responds best to the following five-section structure:
|
||||
|
||||
```
|
||||
[Cinematography/Camera] + [Subject description] + [Action/Behavior] + [Scene/Environment] + [Style and Atmosphere]
|
||||
```
|
||||
|
||||
Each section flows naturally into the next, forming a smooth descriptive passage.
|
||||
|
||||
### 2. Audio Is the Core Advantage
|
||||
|
||||
Veo3 **automatically generates audio that matches the visuals**, including:
|
||||
- Dialogue and voices (write dialogue directly in the prompt)
|
||||
- Ambient sound effects (wind, footsteps, city noise)
|
||||
- Emotional music (atmospheric music, background score)
|
||||
|
||||
**Dialogue format**: Describe what characters say in natural language:
|
||||
|
||||
```
|
||||
A woman turns to the camera and says "I've been waiting for this moment."
|
||||
```
|
||||
|
||||
Or use quotation marks to indicate dialogue:
|
||||
|
||||
```
|
||||
Two friends sitting at a cafe, one says "Did you hear about the news?" and the other responds with a surprised expression "No way, tell me everything."
|
||||
```
|
||||
|
||||
### 3. First Frame + Last Frame Transitions
|
||||
|
||||
Veo3 supports specifying both first and last frame images simultaneously — the model will automatically generate a transition animation between the two frames. Ideal for:
|
||||
- Time lapse (day → night)
|
||||
- Seasonal changes (spring → winter)
|
||||
- Aging effects
|
||||
- Scene transitions
|
||||
|
||||
### 4. Fixed 8-Second Duration
|
||||
|
||||
All Veo3 videos are 8 seconds. Plan content complexity accordingly:
|
||||
- 1-2 main actions/events
|
||||
- 1 camera movement
|
||||
- Suitable for a single plot point or atmospheric segment
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Leverage Audio Fully
|
||||
|
||||
Veo3's biggest differentiator is audio. Actively describe sounds in your prompt:
|
||||
|
||||
```
|
||||
✓ "The sound of rain pattering on the window, a piano melody plays softly in the background"
|
||||
✓ "Birds chirping in the morning, leaves rustling in the gentle breeze"
|
||||
✓ "The bustling noise of a Tokyo street market, vendors calling out prices"
|
||||
|
||||
✗ Don't assume the model will automatically generate the sound you want — be explicit
|
||||
```
|
||||
|
||||
### Cinematography Terms
|
||||
|
||||
Veo3 has excellent understanding of professional cinematography terminology:
|
||||
|
||||
```
|
||||
Anamorphic lens → Widescreen anamorphic lens
|
||||
Rack focus → Shifting focus between subjects
|
||||
Dutch angle → Tilted composition
|
||||
Steadicam → Stabilized tracking shot
|
||||
Whip pan → Fast pan
|
||||
Pull focus → Pulling focus
|
||||
Depth of field f/1.4 → Shallow depth of field
|
||||
35mm / 50mm / 85mm → Lens focal lengths
|
||||
```
|
||||
|
||||
### Dialogue Scenes
|
||||
|
||||
Veo3 is the only model that can reliably generate dialogue — take full advantage:
|
||||
|
||||
```
|
||||
A documentary-style interview. A weathered fisherman sits on his boat at golden hour,
|
||||
looking directly at the camera. He speaks with a gravelly voice: "The sea doesn't care
|
||||
about your plans. You either learn her rhythm, or she teaches you the hard way."
|
||||
Waves lap gently against the hull. Shot on 16mm film, warm color grading.
|
||||
```
|
||||
|
||||
### Excluding Unwanted Elements
|
||||
|
||||
Veo3 has no negative_prompt parameter. Exclude elements by **describing the desired state**:
|
||||
|
||||
```
|
||||
✗ "no people in the scene"
|
||||
✓ "an empty, deserted street with no pedestrians visible"
|
||||
|
||||
✗ "no text or watermark"
|
||||
✓ "clean frame, pure cinematic imagery"
|
||||
```
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
1. **Duration is not adjustable**: Fixed at 8s — don't try to specify "10 second clip" in the prompt
|
||||
2. **Don't neglect audio description**: No audio description = random audio that may not match expectations
|
||||
3. **Don't overload**: Don't cram too much plot into 8s
|
||||
4. **Character consistency**: Characters may swap features in multi-person scenes — limit to 1-2 people
|
||||
5. **Text rendering**: On-screen text is unreliable
|
||||
|
||||
## Example Prompts
|
||||
|
||||
### Cinematic Dialogue (8s, T2V)
|
||||
|
||||
```
|
||||
Medium shot, a detective in a dark trench coat stands in a dimly lit alley. Rain pours down around him. He takes a long drag from a cigarette, the ember glowing orange, then exhales slowly and says in a low voice: "She was already gone by the time I got there." Noir lighting with harsh shadows from a single streetlamp. The sound of rain and distant sirens. Shot on 35mm film, desaturated color palette, neo-noir style.
|
||||
```
|
||||
|
||||
### Nature Documentary (8s, I2V first frame)
|
||||
|
||||
```
|
||||
Extreme close-up macro shot of a monarch butterfly emerging from its chrysalis. The wings slowly unfurl, revealing vibrant orange and black patterns still glistening with moisture. Soft morning light illuminates translucent wing membranes. The subtle sound of the membrane stretching and the ambient forest soundscape with distant bird calls. National Geographic quality, shallow depth of field, awe-inspiring nature documentary.
|
||||
```
|
||||
|
||||
### First Frame → Last Frame Transition (8s, first + last frame)
|
||||
|
||||
```
|
||||
Time-lapse transition of a city intersection. The scene smoothly transforms from a bustling daytime rush hour with honking cars and pedestrians crossing, to a quiet midnight scene with empty streets reflecting neon signs. Street lights flicker on as the sky transitions through sunset colors to deep blue. The audio shifts from chaotic city noise to serene nighttime ambiance with distant music from a bar.
|
||||
```
|
||||
|
||||
### Vertical Short Video (8s, 9:16)
|
||||
|
||||
```
|
||||
Vertical format, POV shot of hands opening an old wooden jewelry box. Inside, a golden locket catches the light. Fingers carefully lift the locket and open it, revealing a faded black-and-white photograph inside. A soft gasp is heard. Warm, intimate lighting from a bedside lamp. The gentle creak of the box hinge and a nostalgic music box melody begins to play. Vintage, sentimental, ASMR-quality close-up.
|
||||
```
|
||||
@@ -0,0 +1,143 @@
|
||||
# Wan 2.6 Video Generation - Prompting Guide
|
||||
|
||||
## Model Overview
|
||||
|
||||
| Attribute | Value |
|
||||
|-----------|-------|
|
||||
| Tool Name | `wan_i2v_generation` (single) / `batch_wan_i2v_generation` (batch) |
|
||||
| Duration | 5s, 10s, 15s |
|
||||
| Resolution | 720P, 1080P |
|
||||
| Input Modes | I2V (first frame image required) |
|
||||
| Special Features | Audio-driven video generation (lip sync), multi-shot mode |
|
||||
|
||||
## Core Principles
|
||||
|
||||
### 1. Pure I2V Model
|
||||
|
||||
Wan 2.6 **requires a first frame image** (`image_path`) — pure text-to-video is not supported. The prompt's role is to guide how the first frame comes to life.
|
||||
|
||||
### 2. Basic Prompt Formula
|
||||
|
||||
```
|
||||
[Subject] + [Action] + [Scene/Environment]
|
||||
```
|
||||
|
||||
Advanced version:
|
||||
|
||||
```
|
||||
[Subject] + [Action] + [Scene/Environment] + [Aesthetic control] + [Stylization]
|
||||
```
|
||||
|
||||
### 3. Audio-Driven Generation (Core MV Scenario)
|
||||
|
||||
Provide an audio file via `audio_path`, and Wan will generate video driven by the audio's rhythm and vocal content:
|
||||
- **Lip sync**: First frame of a speaking/singing character + corresponding audio clip
|
||||
- **Rhythm matching**: First frame of dance/movement + music clip
|
||||
- **Atmosphere linking**: Environment scene + background music
|
||||
|
||||
Batch generation (`batch_wan_i2v_generation`) supports up to 3 concurrent jobs — **the core tool for MV production**.
|
||||
|
||||
### 4. Single vs Multi Shot
|
||||
|
||||
- `shot_type: "single"` (default): Single shot — the entire video is one continuous shot
|
||||
- `shot_type: "multi"`: Multi-shot — the model may switch between different angles/framings within the video
|
||||
|
||||
MV scenes typically use single, unless you need to simulate rapid editing.
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Prompt Style: Concise and Direct
|
||||
|
||||
Wan works best with **short, direct prompts** — long descriptions are not needed:
|
||||
|
||||
```
|
||||
✓ "A woman singing emotionally, close-up, tears in her eyes, soft lighting"
|
||||
✗ "In this beautifully composed cinematic masterpiece, we see an incredibly talented young woman who appears to be in her late twenties, performing a deeply emotional song..."
|
||||
```
|
||||
|
||||
### Action Verbs
|
||||
|
||||
Use **clear action verbs** to drive motion:
|
||||
|
||||
| Category | Example Verbs |
|
||||
|----------|---------------|
|
||||
| Character actions | walks, runs, turns, reaches, dances, sings, speaks |
|
||||
| Camera movement | camera pans, camera zooms in, camera orbits, camera tilts up |
|
||||
| Environment motion | wind blows, water flows, clouds drift, leaves fall, fire flickers |
|
||||
| Lighting changes | light fades, sunlight breaks through, shadows shift |
|
||||
|
||||
### Camera Movement Modifiers
|
||||
|
||||
```
|
||||
slow camera push in → Slow push forward
|
||||
smooth tracking shot → Smooth tracking
|
||||
handheld camera movement → Handheld shake
|
||||
static locked-off camera → Completely static
|
||||
gentle camera sway → Subtle swaying (simulates breathing feel)
|
||||
```
|
||||
|
||||
### Duration Planning
|
||||
|
||||
| Duration | Suitable Content | Action Complexity |
|
||||
|----------|-----------------|-------------------|
|
||||
| 5s | Single expression change, simple gesture, static atmosphere | 1 action |
|
||||
| 10s | Complete action sequence, scene atmosphere, simple interaction | 2-3 actions |
|
||||
| 15s | Full narrative segment, multi-step action sequences | 3-4 actions |
|
||||
|
||||
### Audio-Driven Best Practices (MV Scenarios)
|
||||
|
||||
1. **Cut audio clips precisely**: Use `audio_subclip_batch` to pre-split by timestamp
|
||||
2. **Character's mouth slightly open in first frame**: Produces more natural lip sync
|
||||
3. **Audio duration ≤ video duration**: Audio cannot be longer than the video
|
||||
4. **Moderate-speed segments work best**: Very fast rap may not sync properly
|
||||
5. **Unified resolution for batch generation**: Makes subsequent stitching easier
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
1. **First frame image is required**: Missing the first frame will cause an error
|
||||
2. **Don't write overly long prompts**: Under 50 words works best
|
||||
3. **Don't describe what's already in the first frame**: The model already sees it — focus on describing **changes and motion**
|
||||
4. **Don't write contradictory actions with audio**: If the audio is a quiet piano piece, don't write "energetic dancing"
|
||||
5. **Quality may drop in the second half of 15s videos**: Place important content in the first 10s
|
||||
|
||||
## Example Prompts
|
||||
|
||||
### MV Lip Sync (5s, I2V + audio)
|
||||
|
||||
```
|
||||
image_path: [singer close-up first frame]
|
||||
audio_path: [corresponding song clip]
|
||||
prompt: "Close-up, a woman singing passionately, subtle head movements, emotional expression, cinematic lighting"
|
||||
duration: 5
|
||||
```
|
||||
|
||||
### Atmospheric B-Roll (10s, I2V)
|
||||
|
||||
```
|
||||
image_path: [rainy city street first frame]
|
||||
prompt: "Rain falling on empty street, reflections shimmering on wet asphalt, camera slowly moves forward, moody atmospheric lighting, neon signs flickering"
|
||||
duration: 10
|
||||
```
|
||||
|
||||
### Character Action (10s, I2V)
|
||||
|
||||
```
|
||||
image_path: [woman in traditional dress standing first frame]
|
||||
prompt: "Woman in traditional dress turns around gracefully, her long sleeves flowing in the wind, cherry blossoms falling around her, slow motion, ethereal atmosphere"
|
||||
duration: 10
|
||||
```
|
||||
|
||||
### MV Batch Generation Example
|
||||
|
||||
```python
|
||||
# batch_wan_i2v_generation parameter example
|
||||
image_paths: [scene1.jpg, scene2.jpg, scene3.jpg]
|
||||
audio_paths: [clip1.wav, clip2.wav, clip3.wav]
|
||||
prompts: [
|
||||
"Singer performing emotionally, close-up, tears rolling down, soft warm lighting",
|
||||
"Wide shot, figure walking alone on moonlit beach, waves crashing, melancholic atmosphere",
|
||||
"Two hands reaching for each other in slow motion, golden hour backlight, romantic"
|
||||
]
|
||||
durations: [5, 10, 5]
|
||||
resolution: "1080P"
|
||||
```
|
||||
+125
@@ -0,0 +1,125 @@
|
||||
# Character Sheet Workflow
|
||||
|
||||
## Overview
|
||||
|
||||
Character Sheets are a key tool for maintaining character consistency in MV production. By generating standardized character reference images, you ensure uniform character appearance across all subsequent scenes.
|
||||
|
||||
## Character Sheet Types
|
||||
|
||||
### 1. Turnaround Sheet
|
||||
|
||||
Shows the character from multiple angles: front, 3/4 view, side, and back.
|
||||
|
||||
```
|
||||
Prompt template:
|
||||
"Character turnaround sheet of [character description], showing front view, 3/4 view, side view, and back view. [clothing description]. Clean white background, consistent lighting, full body, character design reference sheet, anime/realistic style."
|
||||
```
|
||||
|
||||
### 2. Expression Sheet
|
||||
|
||||
Shows the character in different emotional states.
|
||||
|
||||
```
|
||||
Prompt template:
|
||||
"Expression sheet of [character description], showing 6 different emotions: neutral, happy, sad, angry, surprised, contemplative. Same angle (front-facing), consistent lighting, bust shot, clean layout, character reference sheet."
|
||||
```
|
||||
|
||||
### 3. Pose Sheet
|
||||
|
||||
Shows the character in different action poses.
|
||||
|
||||
```
|
||||
Prompt template:
|
||||
"Pose reference sheet of [character description] in [clothing description], showing 4 dynamic poses: standing, walking, sitting, [specific action]. Full body, clean white background, consistent proportions, action reference sheet."
|
||||
```
|
||||
|
||||
## Workflow
|
||||
|
||||
### Step 1: Define Character Description
|
||||
|
||||
Write a detailed character text description including:
|
||||
- **Body type**: Height, build (slim, athletic, curvy...)
|
||||
- **Facial features**: Face shape, eye color, hairstyle and hair color
|
||||
- **Clothing**: Detailed outfit description
|
||||
- **Aura**: Overall impression/vibe
|
||||
|
||||
Example:
|
||||
```
|
||||
A young Chinese woman in her mid-20s with delicate features. She has long straight
|
||||
black hair reaching her waist, almond-shaped dark brown eyes, and fair skin.
|
||||
She wears a modern hanfu-inspired outfit: a white silk top with subtle floral
|
||||
embroidery and a flowing navy blue pleated skirt. Elegant and ethereal presence.
|
||||
```
|
||||
|
||||
### Step 2: Select Generation Model
|
||||
|
||||
| Style Requirement | Recommended Model | Reason |
|
||||
|-------------------|-------------------|--------|
|
||||
| Realistic characters | Kling (face mode) | Strongest facial consistency |
|
||||
| Anime characters | Midjourney (--niji 7) | Highest anime quality |
|
||||
| Concept design | Gemini / OpenAI | Fast iteration |
|
||||
| Stylized | Kontext | Can convert from photos to style |
|
||||
|
||||
### Step 3: Generate Character Sheets
|
||||
|
||||
1. **Generate the front-facing base image first**:
|
||||
```
|
||||
"Portrait of [character description], front-facing, neutral expression, soft studio lighting,
|
||||
clean background, character reference, high detail"
|
||||
```
|
||||
|
||||
2. **Using the base image as reference, generate turnaround sheet**:
|
||||
```
|
||||
image_paths: [base image]
|
||||
prompt: "Character turnaround sheet based on this character..."
|
||||
```
|
||||
|
||||
3. **Generate expression sheet**:
|
||||
```
|
||||
image_paths: [base image]
|
||||
prompt: "Expression sheet of this same character..."
|
||||
```
|
||||
|
||||
### Step 4: Verify Consistency
|
||||
|
||||
After generation, check:
|
||||
- [ ] Are facial features consistent across all angles?
|
||||
- [ ] Is the hairstyle logical from all angles?
|
||||
- [ ] Are clothing details maintained consistently?
|
||||
- [ ] Are body proportions stable?
|
||||
- [ ] Is the overall style unified?
|
||||
|
||||
Fix inconsistencies using Kontext or Kling.
|
||||
|
||||
### Step 5: Output for Downstream Use
|
||||
|
||||
Character sheet applications:
|
||||
- **Video first frame generation**: Use Kling face mode + scene description
|
||||
- **Storyboard generation**: Use as character reference images
|
||||
- **Lip sync**: Provide front-facing close-up as first frame
|
||||
|
||||
## Photorealistic Character Design
|
||||
|
||||
When "real-person photo" quality is needed for character design:
|
||||
|
||||
```
|
||||
Prompt template:
|
||||
"Professional photography headshot of [character description], [expression],
|
||||
shot on Canon EOS R5, 85mm f/1.4 lens, natural window light,
|
||||
clean neutral background, magazine quality portrait, photorealistic,
|
||||
8K detail, skin texture visible, catchlight in eyes."
|
||||
```
|
||||
|
||||
Key elements:
|
||||
- Specify camera and lens parameters to increase realism
|
||||
- Describe light direction and quality
|
||||
- Mention skin texture and catchlight
|
||||
- Use photography terminology rather than painting terminology
|
||||
|
||||
## MV Character Design Best Practices
|
||||
|
||||
1. **Prepare for lip sync**: Front-facing character close-ups should have mouth slightly open or in a speech-ready state
|
||||
2. **Multiple outfits**: If the MV has scene changes, prepare corresponding outfit versions for each scene
|
||||
3. **Emotional range**: MVs require rich expressions — ensure the expression sheet covers all emotions in the song
|
||||
4. **First-frame friendly**: Character image composition should work as a video first frame (leave room for motion)
|
||||
5. **Save all prompts**: Record the prompt used for each generated image for easy reuse and adjustment
|
||||
Binary file not shown.
Reference in New Issue
Block a user