Flatten skill category directory structure
This commit is contained in:
@@ -0,0 +1,253 @@
|
||||
# Image Generation Models - Prompting Guide
|
||||
|
||||
This guide covers all available image generation models, helping you choose the right model and write optimal prompts.
|
||||
|
||||
## Model Selection Quick Reference
|
||||
|
||||
| Need | Recommended Model | Tool Name |
|
||||
|------|-------------------|-----------|
|
||||
| General text-to-image / image editing | Gemini (default) | `nano_banana_image_generation` |
|
||||
| Batch image generation | Gemini Batch | `nano_banana_batch_image_generation_v2` |
|
||||
| Text embedded in images | Qwen Wanxiang | `qwen_image_generation` |
|
||||
| Style transfer / editing | FLUX Kontext | `kontext_image_generation` |
|
||||
| Face / character reference | Kling | `kling_image_generation` |
|
||||
| Creative illustration / Logo / transparent background | OpenAI GPT-Image | `openai_image_generation` |
|
||||
| Artistic quality | Midjourney | `midjourney_image_generation` |
|
||||
| Many reference images (up to 14) | Seedream | `seedream_image_generation` |
|
||||
|
||||
---
|
||||
|
||||
## Gemini (nano_banana)
|
||||
|
||||
**Default image model** with the strongest overall capabilities.
|
||||
|
||||
### Model Variants
|
||||
|
||||
| Variant | Model ID | Features |
|
||||
|---------|----------|----------|
|
||||
| nano_banana | gemini-2.5-flash-image | Fast, good for iteration |
|
||||
| nano_banana_2 | gemini-3-pro-image-preview | High quality |
|
||||
| nano_banana_2_flash | gemini-3.1-flash-image-preview | Default, balances speed and quality |
|
||||
|
||||
### Prompt Style
|
||||
|
||||
Natural language descriptions, like talking to an artist:
|
||||
|
||||
```
|
||||
A serene Japanese garden in autumn, with a wooden bridge over a koi pond.
|
||||
Red and golden maple leaves float on the water surface. Soft morning mist
|
||||
rises from the pond. Shot in the style of a landscape photograph, natural
|
||||
lighting, shallow depth of field focusing on the bridge.
|
||||
```
|
||||
|
||||
### Image Editing
|
||||
|
||||
Provide reference images via `image_paths`, then describe the desired changes in the prompt:
|
||||
|
||||
```
|
||||
image_paths: [original.jpg]
|
||||
prompt: "Change the background to a sunset beach scene, keep the person and their pose exactly the same"
|
||||
```
|
||||
|
||||
### Resolution
|
||||
|
||||
Supports 1K / 2K / 4K, default 2K. 2K is sufficient for video first frames.
|
||||
|
||||
---
|
||||
|
||||
## Qwen Wanxiang (qwen)
|
||||
|
||||
**Best choice for embedding text in images**. Only supports image editing (reference image required).
|
||||
|
||||
### Use Cases
|
||||
|
||||
- Adding text titles to images
|
||||
- Modifying existing text in images
|
||||
- Poster / cover creation
|
||||
|
||||
### Prompt
|
||||
|
||||
```
|
||||
image_paths: [background.jpg]
|
||||
prompt: "Add the text 'Brightest Star in the Night Sky' in elegant calligraphy style at the center of the image, white text with subtle glow effect"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## FLUX Kontext (kontext)
|
||||
|
||||
**Best choice for style transfer and image editing**.
|
||||
|
||||
### Strengths
|
||||
|
||||
- Converting photos to specific art styles
|
||||
- Changing art style while maintaining composition
|
||||
- Multi-image reference fusion
|
||||
|
||||
### Prompt
|
||||
|
||||
```
|
||||
image_paths: [photo.jpg]
|
||||
prompt: "Transform this photo into Studio Ghibli anime style, maintain the same composition and character pose, soft watercolor textures, warm pastel colors"
|
||||
```
|
||||
|
||||
### Aspect Ratios
|
||||
|
||||
Supports a rich set of aspect ratios: 1:1, 16:9, 9:16, 3:2, 2:3, 4:3, 3:4, 4:5, 5:4, 21:9, 7:4, 4:7
|
||||
|
||||
---
|
||||
|
||||
## Kling (kling)
|
||||
|
||||
**Best choice for face and character reference generation**.
|
||||
|
||||
### Reference Modes
|
||||
|
||||
| Mode | Use Case |
|
||||
|------|----------|
|
||||
| `subject` | Preserves overall appearance (clothing, pose, style) |
|
||||
| `face` | Preserves facial features only (can change outfit, scene) |
|
||||
|
||||
### Prompt
|
||||
|
||||
```
|
||||
# Face mode: Keep face, change scene
|
||||
image_path: face_reference.jpg
|
||||
reference_type: face
|
||||
prompt: "A woman in a traditional red qipao standing in a lantern-lit ancient Chinese street at night, full body shot, elegant pose, cinematic lighting"
|
||||
|
||||
# Subject mode: Preserve overall appearance
|
||||
image_path: character_reference.jpg
|
||||
reference_type: subject
|
||||
prompt: "The same person standing on a rooftop at sunset, wind blowing through their hair, dramatic sky background, cinematic wide shot"
|
||||
```
|
||||
|
||||
### Batch Generation
|
||||
|
||||
`kling_batch_image_generation` supports independent prompt, reference image, reference mode, and aspect ratio for each image.
|
||||
|
||||
---
|
||||
|
||||
## OpenAI GPT-Image (openai)
|
||||
|
||||
**Best choice for creative illustrations, logos, and transparent backgrounds**.
|
||||
|
||||
### Special Capabilities
|
||||
|
||||
- Transparent background images (ideal for logos, stickers, UI elements)
|
||||
- Creative concept visualization
|
||||
- Abstract art
|
||||
|
||||
### Prompt
|
||||
|
||||
```
|
||||
prompt: "A minimalist logo of a phoenix rising from flames, clean vector style, bold lines, gradient from deep red to bright gold"
|
||||
size: "1024x1024"
|
||||
quality: "high"
|
||||
```
|
||||
|
||||
### Image Editing
|
||||
|
||||
```
|
||||
image_paths: [original.jpg]
|
||||
prompt: "Remove the background and make it transparent, keep only the main subject"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Midjourney (midjourney)
|
||||
|
||||
**Best choice for artistic quality and stylization**.
|
||||
|
||||
### Version Selection
|
||||
|
||||
Append version flag at the end of the prompt:
|
||||
- `--v 7`: V7 (default), realistic and general purpose
|
||||
- `--niji 7`: Niji7, anime/manga style
|
||||
|
||||
### Core Parameters
|
||||
|
||||
| Parameter | Function | Example |
|
||||
|-----------|----------|---------|
|
||||
| `--ar` | Aspect ratio | `--ar 16:9` |
|
||||
| `--sref` | Style reference URL | `--sref https://example.com/style.jpg` |
|
||||
| `--cref` | Character reference URL | `--cref https://example.com/char.jpg` |
|
||||
| `--cw` | Character reference weight (0-100) | `--cw 50` |
|
||||
|
||||
### Prompt Style
|
||||
|
||||
Midjourney favors **concise, evocative descriptions** — no need to be overly verbose:
|
||||
|
||||
```
|
||||
prompt: "ancient samurai meditating under cherry blossoms, moonlit courtyard, atmospheric fog, cinematic lighting, dramatic shadows --ar 16:9 --v 7"
|
||||
```
|
||||
|
||||
### Reference Images
|
||||
|
||||
Provide local reference images via `image_paths`:
|
||||
|
||||
```
|
||||
image_paths: [reference.jpg]
|
||||
prompt: "a warrior in similar armor style standing on a cliff overlooking a vast ocean, epic scale, golden hour --ar 16:9 --v 7"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Seedream (seedream)
|
||||
|
||||
**Supports the most reference images** — version 4.5 supports up to 14 reference images.
|
||||
|
||||
### Model Selection
|
||||
|
||||
| Model | Features |
|
||||
|-------|----------|
|
||||
| doubao-seedream-4-0-250828 | Standard version |
|
||||
| doubao-seedream-4-5-251128 | New version, supports up to 14 reference images |
|
||||
|
||||
### Ideal Use Cases
|
||||
|
||||
- Scenarios requiring many reference images to constrain style
|
||||
- Multi-character consistency (provide multiple character reference images)
|
||||
- Complex scene reconstruction
|
||||
|
||||
### Prompt
|
||||
|
||||
```
|
||||
image_paths: [ref1.jpg, ref2.jpg, ref3.jpg, ...]
|
||||
prompt: "A group photo of these characters together in a modern office setting, natural lighting, professional photography style, everyone is smiling and looking at the camera"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## General Prompting Tips
|
||||
|
||||
### 1. Optimizing for Video First Frames
|
||||
|
||||
When generating images intended as video first frames:
|
||||
|
||||
```
|
||||
✓ Character's mouth slightly open (helps with lip sync)
|
||||
✓ Pose suggests motion (e.g., mid-step, mid-turn)
|
||||
✓ Leave room for motion in the frame (don't push subject to edges)
|
||||
✓ Avoid extreme perspective (video models struggle to maintain it)
|
||||
✗ Don't generate overly static "passport photo" compositions
|
||||
```
|
||||
|
||||
### 2. MV First Frame Batch Generation Strategy
|
||||
|
||||
```
|
||||
1. Determine art style → select primary image generation model
|
||||
2. Use Kling face mode to lock character facial consistency
|
||||
3. Batch generate first frames for each scene with the selected model
|
||||
4. Use Kontext to unify style (if inconsistencies exist)
|
||||
5. After confirmation, send to video model for generation
|
||||
```
|
||||
|
||||
### 3. Style Consistency
|
||||
|
||||
Tips for maintaining unified style across a set of images:
|
||||
- Use the same style keyword suffix
|
||||
- Use Midjourney `--sref` to lock style
|
||||
- Use Kling `face` mode to lock character appearance
|
||||
- Use Kontext with the same style reference image for all images
|
||||
- In batch generation tools (Gemini batch / Kling batch), use the same style description suffix for each image
|
||||
@@ -0,0 +1,125 @@
|
||||
# Kling Video Generation - Prompting Guide
|
||||
|
||||
## Model Overview
|
||||
|
||||
| Attribute | Value |
|
||||
|-----------|-------|
|
||||
| Tool Name | `kling_video_generation` |
|
||||
| Available Models | kling-v1-6, kling-v2-master, kling-v2-5-turbo, kling-v2-6 (default) |
|
||||
| Duration | 5s or 10s |
|
||||
| Modes | std (standard), pro (professional, supports sound) |
|
||||
| Input Modes | T2V, I2V (`first_frame_image_path`) |
|
||||
| Special Features | Pro mode can generate video with sound (`enable_sound: true`) |
|
||||
|
||||
## Core Principles
|
||||
|
||||
### 1. Prompt Style: Director's Instructions
|
||||
|
||||
Kling responds best to **structured, directorial commands**. Organize prompts in this order:
|
||||
|
||||
```
|
||||
[Camera setup] + [Subject description] + [Action directives] + [Environment/Scene] + [Lighting] + [Emotion/Atmosphere] + [Technical parameters]
|
||||
```
|
||||
|
||||
### 2. Pro Mode and Sound
|
||||
|
||||
When sound is needed in the video, you must set:
|
||||
- `mode: "pro"`
|
||||
- `enable_sound: true`
|
||||
|
||||
Sound type is automatically inferred from the visual content — audio cannot be precisely controlled via prompt. Works well for ambient and natural sound effects, not suitable for dialogue or music.
|
||||
|
||||
### 3. Negative Prompt
|
||||
|
||||
Kling supports `negative_prompt` to exclude unwanted elements:
|
||||
|
||||
```
|
||||
negative_prompt: "blurry, distorted faces, extra fingers, deformed hands, low quality, watermark, text overlay"
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Character Generation
|
||||
|
||||
Kling excels in **character consistency and facial quality**:
|
||||
|
||||
- Describe facial features and expressions in detail
|
||||
- Specify clothing details to maintain consistency
|
||||
- Facial expression changes are a Kling strength — use them confidently
|
||||
- In I2V mode, the face quality of the first frame directly determines video quality
|
||||
|
||||
### Motion Control
|
||||
|
||||
- **Simple repetitive actions** work best: walking, turning head, waving, nodding
|
||||
- **Complex full-body movements** (dance, fighting) require lower expectations
|
||||
- Use **temporal adverbs** to control pacing: slowly, gradually, suddenly, gently
|
||||
- Schedule 1 main action within 5s, 2-3 actions within 10s
|
||||
|
||||
### Camera Movement
|
||||
|
||||
Kling understands camera movements with high precision:
|
||||
|
||||
```
|
||||
Static shot → Fixed camera, ideal for character performance
|
||||
Slow pan right → Slow lateral movement, ideal for environment showcase
|
||||
Dolly zoom → Hitchcock zoom, dramatic effect
|
||||
Low angle shot → Looking up, adds grandeur
|
||||
Over-the-shoulder → Over-shoulder shot, conversation scenes
|
||||
Bird's eye view → Overhead view, grand scenes
|
||||
```
|
||||
|
||||
### Style Control
|
||||
|
||||
Kling responds well to these style keywords:
|
||||
|
||||
```
|
||||
cinematic, film noir, anime style, watercolor painting,
|
||||
oil painting, photorealistic, vintage film, documentary,
|
||||
music video style, fashion editorial, slow motion
|
||||
```
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
1. **std mode has no sound**: Sound requires pro mode
|
||||
2. **Don't mix contradictory styles**: e.g., "realistic anime style"
|
||||
3. **Don't describe desired elements in negative_prompt**: Negative prompt is for exclusion only
|
||||
4. **Character count**: More than 2 characters tends to cause confusion — limit to 1-2 people
|
||||
5. **Text generation**: On-screen text is unreliable — avoid requesting text in prompts
|
||||
|
||||
## Example Prompts
|
||||
|
||||
### Character Close-up + Sound (5s, pro, T2V)
|
||||
|
||||
```
|
||||
prompt: "Close-up portrait shot of a young woman sitting by a rainy window. She slowly lifts a cup of coffee to her lips, steam rising from the cup. Rain droplets streak down the glass behind her. Warm interior lighting contrasts with the cool blue exterior. Intimate, contemplative mood. Cinematic shallow depth of field."
|
||||
|
||||
negative_prompt: "blurry, distorted face, extra fingers, low quality, watermark"
|
||||
|
||||
mode: pro, enable_sound: true → Automatically generates rain and indoor ambient sounds
|
||||
```
|
||||
|
||||
### Dynamic Scene (10s, I2V)
|
||||
|
||||
```
|
||||
prompt: "Medium shot, a martial artist in traditional white robes performs slow tai chi movements in a misty bamboo forest at dawn. The camera slowly orbits around him as morning light filters through the bamboo canopy. Leaves gently fall around him. Serene, meditative atmosphere. Cinematic, anamorphic lens flare."
|
||||
|
||||
negative_prompt: "fast motion, blurry, distorted body, modern clothing"
|
||||
```
|
||||
|
||||
### Natural Landscape + Sound (10s, pro, T2V)
|
||||
|
||||
```
|
||||
prompt: "Establishing wide shot of a volcanic coastline at sunset. Waves crash against black lava rocks, sending spray into the golden light. The camera slowly pushes forward toward the horizon. Dramatic clouds streak across the sky. Epic, cinematic scale, National Geographic quality, HDR lighting."
|
||||
|
||||
negative_prompt: "people, text, watermark, low resolution"
|
||||
|
||||
mode: pro, enable_sound: true → Ocean waves and wind sounds
|
||||
```
|
||||
|
||||
### Emotional Performance (5s, I2V)
|
||||
|
||||
```
|
||||
prompt: "Close-up shot, a man receives an unexpected phone call. His expression transitions from surprise to overwhelming joy as tears well up in his eyes. He covers his mouth with one hand. Soft natural light from a nearby window. Shallow depth of field, intimate documentary style."
|
||||
|
||||
negative_prompt: "exaggerated expression, cartoon, anime, blurry"
|
||||
```
|
||||
@@ -0,0 +1,104 @@
|
||||
# Official Hilo Video Generation - Prompting Guide
|
||||
|
||||
## Model Overview
|
||||
|
||||
| Attribute | Value |
|
||||
|-----------|-------|
|
||||
| Tool Name | `official_videos_generation` |
|
||||
| Available Models | Official-Hilo-2.3-Fast, Official-Hilo-2.3, Official-Hilo-02 |
|
||||
| Duration | 6s or 10s |
|
||||
| Resolution | 768P, 1080P |
|
||||
| Input Modes | I2V (first frame, recommended), I2V + last frame, pure T2V |
|
||||
| Special Features | Background sound effects, automatic prompt optimization, lip sync (via `official_generate_video_based_on_image_and_audio`) |
|
||||
|
||||
## Core Principles
|
||||
|
||||
### 1. Always Use I2V Mode
|
||||
|
||||
**Always prioritize using a first frame image (`first_frame_images`) to drive generation**. Pure text-to-video quality is far inferior to image-to-video.
|
||||
|
||||
Workflow: Generate a high-quality first frame with an image model → Then generate video with Hilo.
|
||||
|
||||
### 2. Prompt Structure
|
||||
|
||||
Hilo prompts should read like **a natural language shot description**, not a keyword dump:
|
||||
|
||||
```
|
||||
[Shot type and movement], [Subject and their action], [Environment details], [Lighting and atmosphere], [Style modifiers]
|
||||
```
|
||||
|
||||
### 3. Automatic Prompt Optimization
|
||||
|
||||
`enable_prompt_optimize` is enabled by default — Hilo will automatically expand and optimize your prompt. If you've already written a very precise prompt, you can disable this feature for more precise control.
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Camera Language
|
||||
|
||||
Explicitly specify shot types and camera movements:
|
||||
|
||||
| Shot Type | English Expression |
|
||||
|-----------|-------------------|
|
||||
| Close-up | Close-up shot, extreme close-up |
|
||||
| Medium shot | Medium shot, waist-up shot |
|
||||
| Wide shot | Wide shot, establishing shot |
|
||||
| Long shot | Long shot, aerial view |
|
||||
| Tracking | Tracking shot, following shot |
|
||||
| Push/Pull | Dolly in / dolly out, zoom in / zoom out |
|
||||
| Pan | Pan left / pan right |
|
||||
| Crane | Crane shot, tilt up / tilt down |
|
||||
| Orbit | Orbit shot, 360-degree rotation |
|
||||
| Handheld | Handheld camera, shaky cam |
|
||||
| Static | Static shot, locked-off camera |
|
||||
|
||||
### Action Description
|
||||
|
||||
- Use **progressive tense** for continuous actions: `A woman is walking through...`
|
||||
- Use **action sequences** for changes: `She turns her head slowly, then breaks into a smile`
|
||||
- Control action intensity: Only 1-2 actions within 6s, 2-3 actions within 10s
|
||||
- Avoid overly intense full-body movements (e.g., running, dancing) — they tend to cause distortion
|
||||
|
||||
### I2V + Last Frame (Official-Hilo-02 exclusive)
|
||||
|
||||
`last_frame_images` is only supported by Hilo-02. Ideal for:
|
||||
- Scene transitions (day → night)
|
||||
- Expression changes (calm → smile)
|
||||
- Object transformations (bud → bloom)
|
||||
|
||||
### Background Sound
|
||||
|
||||
Setting `enable_background_sound: true` automatically generates ambient sound effects matching the visuals (wind, water flow, city noise, etc.). Suitable for natural scenes that don't need music.
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
1. **Don't use negative statements**: The model cannot understand "no people" — describe what you want instead
|
||||
2. **Don't overload adjectives**: 3-5 style words are enough; too many will conflict
|
||||
3. **Don't cram too many actions into 6s**: One clear action beats three vague ones
|
||||
4. **Don't skip the first frame**: Pure T2V quality is unreliable — always provide a first frame
|
||||
5. **Text content**: The model cannot reliably generate on-screen text
|
||||
|
||||
## Example Prompts
|
||||
|
||||
### Character Close-up (6s, I2V)
|
||||
|
||||
```
|
||||
Close-up shot, a young woman with long black hair slowly turns her head toward the camera, her expression shifts from contemplative to a gentle smile. Soft golden hour light illuminates her face from the side. Shallow depth of field, cinematic color grading, film grain.
|
||||
```
|
||||
|
||||
### Aerial Landscape (10s, I2V)
|
||||
|
||||
```
|
||||
Aerial drone shot slowly ascending over a misty mountain valley at dawn. Layers of fog drift between the peaks as the first rays of sunlight paint the ridges gold. The camera gradually tilts down to reveal a winding river below. Epic cinematic scale, 4K quality, nature documentary style.
|
||||
```
|
||||
|
||||
### Scene Transition (6s, I2V + last frame, Hilo-02)
|
||||
|
||||
```
|
||||
Smooth transition, the scene gradually shifts from warm afternoon light to cool blue twilight. Cherry blossom petals drift through the air as the lighting changes. The woman's silhouette remains centered in frame. Dreamy, ethereal atmosphere, soft focus background.
|
||||
```
|
||||
|
||||
### Dynamic Action (10s, I2V)
|
||||
|
||||
```
|
||||
Medium tracking shot, a street musician plays acoustic guitar on a rain-soaked city sidewalk at night. Neon reflections shimmer in puddles around his feet. Pedestrians with umbrellas pass in the background as bokeh lights create a dreamy atmosphere. Camera slowly orbits around the musician. Moody, cinematic, urban night photography style.
|
||||
```
|
||||
@@ -0,0 +1,144 @@
|
||||
# Veo3 Video Generation - Prompting Guide
|
||||
|
||||
## Model Overview
|
||||
|
||||
| Attribute | Value |
|
||||
|-----------|-------|
|
||||
| Tool Name | `veo3_video_generation` |
|
||||
| Duration | 8s (fixed) |
|
||||
| Resolution | 720p, 1080p |
|
||||
| Aspect Ratio | 16:9, 9:16 |
|
||||
| Input Modes | T2V, I2V (first/last frame) |
|
||||
| Special Features | **Built-in audio generation** (dialogue, ambient sound, sound effects), first frame + last frame transitions |
|
||||
|
||||
## Core Principles
|
||||
|
||||
### 1. Five-Section Prompt Structure
|
||||
|
||||
Veo3 responds best to the following five-section structure:
|
||||
|
||||
```
|
||||
[Cinematography/Camera] + [Subject description] + [Action/Behavior] + [Scene/Environment] + [Style and Atmosphere]
|
||||
```
|
||||
|
||||
Each section flows naturally into the next, forming a smooth descriptive passage.
|
||||
|
||||
### 2. Audio Is the Core Advantage
|
||||
|
||||
Veo3 **automatically generates audio that matches the visuals**, including:
|
||||
- Dialogue and voices (write dialogue directly in the prompt)
|
||||
- Ambient sound effects (wind, footsteps, city noise)
|
||||
- Emotional music (atmospheric music, background score)
|
||||
|
||||
**Dialogue format**: Describe what characters say in natural language:
|
||||
|
||||
```
|
||||
A woman turns to the camera and says "I've been waiting for this moment."
|
||||
```
|
||||
|
||||
Or use quotation marks to indicate dialogue:
|
||||
|
||||
```
|
||||
Two friends sitting at a cafe, one says "Did you hear about the news?" and the other responds with a surprised expression "No way, tell me everything."
|
||||
```
|
||||
|
||||
### 3. First Frame + Last Frame Transitions
|
||||
|
||||
Veo3 supports specifying both first and last frame images simultaneously — the model will automatically generate a transition animation between the two frames. Ideal for:
|
||||
- Time lapse (day → night)
|
||||
- Seasonal changes (spring → winter)
|
||||
- Aging effects
|
||||
- Scene transitions
|
||||
|
||||
### 4. Fixed 8-Second Duration
|
||||
|
||||
All Veo3 videos are 8 seconds. Plan content complexity accordingly:
|
||||
- 1-2 main actions/events
|
||||
- 1 camera movement
|
||||
- Suitable for a single plot point or atmospheric segment
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Leverage Audio Fully
|
||||
|
||||
Veo3's biggest differentiator is audio. Actively describe sounds in your prompt:
|
||||
|
||||
```
|
||||
✓ "The sound of rain pattering on the window, a piano melody plays softly in the background"
|
||||
✓ "Birds chirping in the morning, leaves rustling in the gentle breeze"
|
||||
✓ "The bustling noise of a Tokyo street market, vendors calling out prices"
|
||||
|
||||
✗ Don't assume the model will automatically generate the sound you want — be explicit
|
||||
```
|
||||
|
||||
### Cinematography Terms
|
||||
|
||||
Veo3 has excellent understanding of professional cinematography terminology:
|
||||
|
||||
```
|
||||
Anamorphic lens → Widescreen anamorphic lens
|
||||
Rack focus → Shifting focus between subjects
|
||||
Dutch angle → Tilted composition
|
||||
Steadicam → Stabilized tracking shot
|
||||
Whip pan → Fast pan
|
||||
Pull focus → Pulling focus
|
||||
Depth of field f/1.4 → Shallow depth of field
|
||||
35mm / 50mm / 85mm → Lens focal lengths
|
||||
```
|
||||
|
||||
### Dialogue Scenes
|
||||
|
||||
Veo3 is the only model that can reliably generate dialogue — take full advantage:
|
||||
|
||||
```
|
||||
A documentary-style interview. A weathered fisherman sits on his boat at golden hour,
|
||||
looking directly at the camera. He speaks with a gravelly voice: "The sea doesn't care
|
||||
about your plans. You either learn her rhythm, or she teaches you the hard way."
|
||||
Waves lap gently against the hull. Shot on 16mm film, warm color grading.
|
||||
```
|
||||
|
||||
### Excluding Unwanted Elements
|
||||
|
||||
Veo3 has no negative_prompt parameter. Exclude elements by **describing the desired state**:
|
||||
|
||||
```
|
||||
✗ "no people in the scene"
|
||||
✓ "an empty, deserted street with no pedestrians visible"
|
||||
|
||||
✗ "no text or watermark"
|
||||
✓ "clean frame, pure cinematic imagery"
|
||||
```
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
1. **Duration is not adjustable**: Fixed at 8s — don't try to specify "10 second clip" in the prompt
|
||||
2. **Don't neglect audio description**: No audio description = random audio that may not match expectations
|
||||
3. **Don't overload**: Don't cram too much plot into 8s
|
||||
4. **Character consistency**: Characters may swap features in multi-person scenes — limit to 1-2 people
|
||||
5. **Text rendering**: On-screen text is unreliable
|
||||
|
||||
## Example Prompts
|
||||
|
||||
### Cinematic Dialogue (8s, T2V)
|
||||
|
||||
```
|
||||
Medium shot, a detective in a dark trench coat stands in a dimly lit alley. Rain pours down around him. He takes a long drag from a cigarette, the ember glowing orange, then exhales slowly and says in a low voice: "She was already gone by the time I got there." Noir lighting with harsh shadows from a single streetlamp. The sound of rain and distant sirens. Shot on 35mm film, desaturated color palette, neo-noir style.
|
||||
```
|
||||
|
||||
### Nature Documentary (8s, I2V first frame)
|
||||
|
||||
```
|
||||
Extreme close-up macro shot of a monarch butterfly emerging from its chrysalis. The wings slowly unfurl, revealing vibrant orange and black patterns still glistening with moisture. Soft morning light illuminates translucent wing membranes. The subtle sound of the membrane stretching and the ambient forest soundscape with distant bird calls. National Geographic quality, shallow depth of field, awe-inspiring nature documentary.
|
||||
```
|
||||
|
||||
### First Frame → Last Frame Transition (8s, first + last frame)
|
||||
|
||||
```
|
||||
Time-lapse transition of a city intersection. The scene smoothly transforms from a bustling daytime rush hour with honking cars and pedestrians crossing, to a quiet midnight scene with empty streets reflecting neon signs. Street lights flicker on as the sky transitions through sunset colors to deep blue. The audio shifts from chaotic city noise to serene nighttime ambiance with distant music from a bar.
|
||||
```
|
||||
|
||||
### Vertical Short Video (8s, 9:16)
|
||||
|
||||
```
|
||||
Vertical format, POV shot of hands opening an old wooden jewelry box. Inside, a golden locket catches the light. Fingers carefully lift the locket and open it, revealing a faded black-and-white photograph inside. A soft gasp is heard. Warm, intimate lighting from a bedside lamp. The gentle creak of the box hinge and a nostalgic music box melody begins to play. Vintage, sentimental, ASMR-quality close-up.
|
||||
```
|
||||
@@ -0,0 +1,143 @@
|
||||
# Wan 2.6 Video Generation - Prompting Guide
|
||||
|
||||
## Model Overview
|
||||
|
||||
| Attribute | Value |
|
||||
|-----------|-------|
|
||||
| Tool Name | `wan_i2v_generation` (single) / `batch_wan_i2v_generation` (batch) |
|
||||
| Duration | 5s, 10s, 15s |
|
||||
| Resolution | 720P, 1080P |
|
||||
| Input Modes | I2V (first frame image required) |
|
||||
| Special Features | Audio-driven video generation (lip sync), multi-shot mode |
|
||||
|
||||
## Core Principles
|
||||
|
||||
### 1. Pure I2V Model
|
||||
|
||||
Wan 2.6 **requires a first frame image** (`image_path`) — pure text-to-video is not supported. The prompt's role is to guide how the first frame comes to life.
|
||||
|
||||
### 2. Basic Prompt Formula
|
||||
|
||||
```
|
||||
[Subject] + [Action] + [Scene/Environment]
|
||||
```
|
||||
|
||||
Advanced version:
|
||||
|
||||
```
|
||||
[Subject] + [Action] + [Scene/Environment] + [Aesthetic control] + [Stylization]
|
||||
```
|
||||
|
||||
### 3. Audio-Driven Generation (Core MV Scenario)
|
||||
|
||||
Provide an audio file via `audio_path`, and Wan will generate video driven by the audio's rhythm and vocal content:
|
||||
- **Lip sync**: First frame of a speaking/singing character + corresponding audio clip
|
||||
- **Rhythm matching**: First frame of dance/movement + music clip
|
||||
- **Atmosphere linking**: Environment scene + background music
|
||||
|
||||
Batch generation (`batch_wan_i2v_generation`) supports up to 3 concurrent jobs — **the core tool for MV production**.
|
||||
|
||||
### 4. Single vs Multi Shot
|
||||
|
||||
- `shot_type: "single"` (default): Single shot — the entire video is one continuous shot
|
||||
- `shot_type: "multi"`: Multi-shot — the model may switch between different angles/framings within the video
|
||||
|
||||
MV scenes typically use single, unless you need to simulate rapid editing.
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Prompt Style: Concise and Direct
|
||||
|
||||
Wan works best with **short, direct prompts** — long descriptions are not needed:
|
||||
|
||||
```
|
||||
✓ "A woman singing emotionally, close-up, tears in her eyes, soft lighting"
|
||||
✗ "In this beautifully composed cinematic masterpiece, we see an incredibly talented young woman who appears to be in her late twenties, performing a deeply emotional song..."
|
||||
```
|
||||
|
||||
### Action Verbs
|
||||
|
||||
Use **clear action verbs** to drive motion:
|
||||
|
||||
| Category | Example Verbs |
|
||||
|----------|---------------|
|
||||
| Character actions | walks, runs, turns, reaches, dances, sings, speaks |
|
||||
| Camera movement | camera pans, camera zooms in, camera orbits, camera tilts up |
|
||||
| Environment motion | wind blows, water flows, clouds drift, leaves fall, fire flickers |
|
||||
| Lighting changes | light fades, sunlight breaks through, shadows shift |
|
||||
|
||||
### Camera Movement Modifiers
|
||||
|
||||
```
|
||||
slow camera push in → Slow push forward
|
||||
smooth tracking shot → Smooth tracking
|
||||
handheld camera movement → Handheld shake
|
||||
static locked-off camera → Completely static
|
||||
gentle camera sway → Subtle swaying (simulates breathing feel)
|
||||
```
|
||||
|
||||
### Duration Planning
|
||||
|
||||
| Duration | Suitable Content | Action Complexity |
|
||||
|----------|-----------------|-------------------|
|
||||
| 5s | Single expression change, simple gesture, static atmosphere | 1 action |
|
||||
| 10s | Complete action sequence, scene atmosphere, simple interaction | 2-3 actions |
|
||||
| 15s | Full narrative segment, multi-step action sequences | 3-4 actions |
|
||||
|
||||
### Audio-Driven Best Practices (MV Scenarios)
|
||||
|
||||
1. **Cut audio clips precisely**: Use `audio_subclip_batch` to pre-split by timestamp
|
||||
2. **Character's mouth slightly open in first frame**: Produces more natural lip sync
|
||||
3. **Audio duration ≤ video duration**: Audio cannot be longer than the video
|
||||
4. **Moderate-speed segments work best**: Very fast rap may not sync properly
|
||||
5. **Unified resolution for batch generation**: Makes subsequent stitching easier
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
1. **First frame image is required**: Missing the first frame will cause an error
|
||||
2. **Don't write overly long prompts**: Under 50 words works best
|
||||
3. **Don't describe what's already in the first frame**: The model already sees it — focus on describing **changes and motion**
|
||||
4. **Don't write contradictory actions with audio**: If the audio is a quiet piano piece, don't write "energetic dancing"
|
||||
5. **Quality may drop in the second half of 15s videos**: Place important content in the first 10s
|
||||
|
||||
## Example Prompts
|
||||
|
||||
### MV Lip Sync (5s, I2V + audio)
|
||||
|
||||
```
|
||||
image_path: [singer close-up first frame]
|
||||
audio_path: [corresponding song clip]
|
||||
prompt: "Close-up, a woman singing passionately, subtle head movements, emotional expression, cinematic lighting"
|
||||
duration: 5
|
||||
```
|
||||
|
||||
### Atmospheric B-Roll (10s, I2V)
|
||||
|
||||
```
|
||||
image_path: [rainy city street first frame]
|
||||
prompt: "Rain falling on empty street, reflections shimmering on wet asphalt, camera slowly moves forward, moody atmospheric lighting, neon signs flickering"
|
||||
duration: 10
|
||||
```
|
||||
|
||||
### Character Action (10s, I2V)
|
||||
|
||||
```
|
||||
image_path: [woman in traditional dress standing first frame]
|
||||
prompt: "Woman in traditional dress turns around gracefully, her long sleeves flowing in the wind, cherry blossoms falling around her, slow motion, ethereal atmosphere"
|
||||
duration: 10
|
||||
```
|
||||
|
||||
### MV Batch Generation Example
|
||||
|
||||
```python
|
||||
# batch_wan_i2v_generation parameter example
|
||||
image_paths: [scene1.jpg, scene2.jpg, scene3.jpg]
|
||||
audio_paths: [clip1.wav, clip2.wav, clip3.wav]
|
||||
prompts: [
|
||||
"Singer performing emotionally, close-up, tears rolling down, soft warm lighting",
|
||||
"Wide shot, figure walking alone on moonlit beach, waves crashing, melancholic atmosphere",
|
||||
"Two hands reaching for each other in slow motion, golden hour backlight, romantic"
|
||||
]
|
||||
durations: [5, 10, 5]
|
||||
resolution: "1080P"
|
||||
```
|
||||
@@ -0,0 +1,125 @@
|
||||
# Character Sheet Workflow
|
||||
|
||||
## Overview
|
||||
|
||||
Character Sheets are a key tool for maintaining character consistency in MV production. By generating standardized character reference images, you ensure uniform character appearance across all subsequent scenes.
|
||||
|
||||
## Character Sheet Types
|
||||
|
||||
### 1. Turnaround Sheet
|
||||
|
||||
Shows the character from multiple angles: front, 3/4 view, side, and back.
|
||||
|
||||
```
|
||||
Prompt template:
|
||||
"Character turnaround sheet of [character description], showing front view, 3/4 view, side view, and back view. [clothing description]. Clean white background, consistent lighting, full body, character design reference sheet, anime/realistic style."
|
||||
```
|
||||
|
||||
### 2. Expression Sheet
|
||||
|
||||
Shows the character in different emotional states.
|
||||
|
||||
```
|
||||
Prompt template:
|
||||
"Expression sheet of [character description], showing 6 different emotions: neutral, happy, sad, angry, surprised, contemplative. Same angle (front-facing), consistent lighting, bust shot, clean layout, character reference sheet."
|
||||
```
|
||||
|
||||
### 3. Pose Sheet
|
||||
|
||||
Shows the character in different action poses.
|
||||
|
||||
```
|
||||
Prompt template:
|
||||
"Pose reference sheet of [character description] in [clothing description], showing 4 dynamic poses: standing, walking, sitting, [specific action]. Full body, clean white background, consistent proportions, action reference sheet."
|
||||
```
|
||||
|
||||
## Workflow
|
||||
|
||||
### Step 1: Define Character Description
|
||||
|
||||
Write a detailed character text description including:
|
||||
- **Body type**: Height, build (slim, athletic, curvy...)
|
||||
- **Facial features**: Face shape, eye color, hairstyle and hair color
|
||||
- **Clothing**: Detailed outfit description
|
||||
- **Aura**: Overall impression/vibe
|
||||
|
||||
Example:
|
||||
```
|
||||
A young Chinese woman in her mid-20s with delicate features. She has long straight
|
||||
black hair reaching her waist, almond-shaped dark brown eyes, and fair skin.
|
||||
She wears a modern hanfu-inspired outfit: a white silk top with subtle floral
|
||||
embroidery and a flowing navy blue pleated skirt. Elegant and ethereal presence.
|
||||
```
|
||||
|
||||
### Step 2: Select Generation Model
|
||||
|
||||
| Style Requirement | Recommended Model | Reason |
|
||||
|-------------------|-------------------|--------|
|
||||
| Realistic characters | Kling (face mode) | Strongest facial consistency |
|
||||
| Anime characters | Midjourney (--niji 7) | Highest anime quality |
|
||||
| Concept design | Gemini / OpenAI | Fast iteration |
|
||||
| Stylized | Kontext | Can convert from photos to style |
|
||||
|
||||
### Step 3: Generate Character Sheets
|
||||
|
||||
1. **Generate the front-facing base image first**:
|
||||
```
|
||||
"Portrait of [character description], front-facing, neutral expression, soft studio lighting,
|
||||
clean background, character reference, high detail"
|
||||
```
|
||||
|
||||
2. **Using the base image as reference, generate turnaround sheet**:
|
||||
```
|
||||
image_paths: [base image]
|
||||
prompt: "Character turnaround sheet based on this character..."
|
||||
```
|
||||
|
||||
3. **Generate expression sheet**:
|
||||
```
|
||||
image_paths: [base image]
|
||||
prompt: "Expression sheet of this same character..."
|
||||
```
|
||||
|
||||
### Step 4: Verify Consistency
|
||||
|
||||
After generation, check:
|
||||
- [ ] Are facial features consistent across all angles?
|
||||
- [ ] Is the hairstyle logical from all angles?
|
||||
- [ ] Are clothing details maintained consistently?
|
||||
- [ ] Are body proportions stable?
|
||||
- [ ] Is the overall style unified?
|
||||
|
||||
Fix inconsistencies using Kontext or Kling.
|
||||
|
||||
### Step 5: Output for Downstream Use
|
||||
|
||||
Character sheet applications:
|
||||
- **Video first frame generation**: Use Kling face mode + scene description
|
||||
- **Storyboard generation**: Use as character reference images
|
||||
- **Lip sync**: Provide front-facing close-up as first frame
|
||||
|
||||
## Photorealistic Character Design
|
||||
|
||||
When "real-person photo" quality is needed for character design:
|
||||
|
||||
```
|
||||
Prompt template:
|
||||
"Professional photography headshot of [character description], [expression],
|
||||
shot on Canon EOS R5, 85mm f/1.4 lens, natural window light,
|
||||
clean neutral background, magazine quality portrait, photorealistic,
|
||||
8K detail, skin texture visible, catchlight in eyes."
|
||||
```
|
||||
|
||||
Key elements:
|
||||
- Specify camera and lens parameters to increase realism
|
||||
- Describe light direction and quality
|
||||
- Mention skin texture and catchlight
|
||||
- Use photography terminology rather than painting terminology
|
||||
|
||||
## MV Character Design Best Practices
|
||||
|
||||
1. **Prepare for lip sync**: Front-facing character close-ups should have mouth slightly open or in a speech-ready state
|
||||
2. **Multiple outfits**: If the MV has scene changes, prepare corresponding outfit versions for each scene
|
||||
3. **Emotional range**: MVs require rich expressions — ensure the expression sheet covers all emotions in the song
|
||||
4. **First-frame friendly**: Character image composition should work as a video first frame (leave room for motion)
|
||||
5. **Save all prompts**: Record the prompt used for each generated image for easy reuse and adjustment
|
||||
Reference in New Issue
Block a user