diff --git a/image-creation/comfyui/SKILL.md b/image-creation/comfyui/SKILL.md new file mode 100644 index 0000000..e5a8a7c --- /dev/null +++ b/image-creation/comfyui/SKILL.md @@ -0,0 +1,612 @@ +--- +name: comfyui +description: "Generate images, video, and audio with ComfyUI — install, launch, manage nodes/models, run workflows with parameter injection. Uses the official comfy-cli for lifecycle and direct REST/WebSocket API for execution." +version: 5.1.0 +author: [kshitijk4poor, alt-glitch, purzbeats] +license: MIT +platforms: [macos, linux, windows] +compatibility: "Requires ComfyUI (local, Comfy Desktop, or Comfy Cloud) and comfy-cli (auto-installed via pipx/uvx by the setup script)." +prerequisites: + commands: ["python3"] +setup: + help: "Run scripts/hardware_check.py FIRST to decide local vs Comfy Cloud; then scripts/comfyui_setup.sh auto-installs locally (or use Cloud API key for platform.comfy.org)." +metadata: + hermes: + tags: + - comfyui + - image-generation + - stable-diffusion + - flux + - sd3 + - wan-video + - hunyuan-video + - creative + - generative-ai + - video-generation + related_skills: [stable-diffusion-image-generation, image_gen] + category: creative +--- + +# ComfyUI + +Generate images, video, audio, and 3D content through ComfyUI using the +official `comfy-cli` for setup/lifecycle and direct REST/WebSocket API +for workflow execution. + +## What's in this skill + +**Reference docs (`references/`):** + +- `official-cli.md` — every `comfy ...` command, with flags +- `rest-api.md` — REST + WebSocket endpoints (local + cloud), payload schemas +- `workflow-format.md` — API-format JSON, common node types, param mapping +- `template-integrity.md` — converting `comfyui-workflow-templates` from + editor format to API format: Reroute bypass, dotted dynamic-input keys + (`values.a`, `resize_type.width`), Cloud quirks (302 redirect, 1 concurrent + free-tier job, 1080p VRAM ceiling), Discord-compatible ffmpeg stitch. + Authored by [@purzbeats](https://github.com/purzbeats). Load this whenever + you're starting from an official template. + +**Scripts (`scripts/`):** + +| Script | Purpose | +|--------|---------| +| `_common.py` | Shared HTTP, cloud routing, node catalogs (don't run directly) | +| `hardware_check.py` | Probe GPU/VRAM/disk → recommend local vs Comfy Cloud | +| `comfyui_setup.sh` | Hardware check + comfy-cli + ComfyUI install + launch + verify | +| `extract_schema.py` | Read a workflow → list controllable params + model deps | +| `check_deps.py` | Check workflow against running server → list missing nodes/models | +| `auto_fix_deps.py` | Run check_deps then `comfy node install` / `comfy model download` | +| `run_workflow.py` | Inject params, submit, monitor, download outputs (HTTP or WS) | +| `run_batch.py` | Submit a workflow N times with sweeps, parallel up to your tier | +| `ws_monitor.py` | Real-time WebSocket viewer for executing jobs (live progress) | +| `health_check.py` | Verification checklist runner — comfy-cli + server + models + smoke test | +| `fetch_logs.py` | Pull traceback / status messages for a given prompt_id | + +**Example workflows (`workflows/`):** SD 1.5, SDXL, Flux Dev, SDXL img2img, +SDXL inpaint, ESRGAN upscale, AnimateDiff video, Wan T2V. See +`workflows/README.md`. + +## When to Use + +- User asks to generate images with Stable Diffusion, SDXL, Flux, SD3, etc. +- User wants to run a specific ComfyUI workflow file +- User wants to chain generative steps (txt2img → upscale → face restore) +- User needs ControlNet, inpainting, img2img, or other advanced pipelines +- User asks to manage ComfyUI queue, check models, or install custom nodes +- User wants video/audio/3D generation via AnimateDiff, Hunyuan, Wan, AudioCraft, etc. + +## Architecture: Two Layers + +``` +┌─────────────────────────────────────────────────────┐ +│ Layer 1: comfy-cli (official lifecycle tool) │ +│ Setup, server lifecycle, custom nodes, models │ +│ → comfy install / launch / stop / node / model │ +└─────────────────────────┬───────────────────────────┘ + │ +┌─────────────────────────▼───────────────────────────┐ +│ Layer 2: REST/WebSocket API + skill scripts │ +│ Workflow execution, param injection, monitoring │ +│ POST /api/prompt, GET /api/view, WS /ws │ +│ → run_workflow.py, run_batch.py, ws_monitor.py │ +└─────────────────────────────────────────────────────┘ +``` + +**Why two layers?** The official CLI is excellent for installation and server +management but has minimal workflow execution support. The REST/WS API fills +that gap — the scripts handle param injection, execution monitoring, and +output download that the CLI doesn't do. + +## Quick Start + +### Detect environment + +```bash +# What's available? +command -v comfy >/dev/null 2>&1 && echo "comfy-cli: installed" +curl -s http://127.0.0.1:8188/system_stats 2>/dev/null && echo "server: running" + +# Can this machine run ComfyUI locally? (GPU/VRAM/disk check) +python3 scripts/hardware_check.py +``` + +If nothing is installed, see **Setup & Onboarding** below — but always run the +hardware check first. + +### One-line health check + +```bash +python3 scripts/health_check.py +# → JSON: comfy_cli on PATH? server reachable? at least one checkpoint? smoke-test passes? +``` + +## Core Workflow + +### Step 1: Get a workflow JSON in API format + +Workflows must be in API format (each node has `class_type`). They come from: + +- ComfyUI web UI → **Workflow → Export (API)** (newer UI) or + the legacy "Save (API Format)" button (older UI) +- This skill's `workflows/` directory (ready-to-run examples) +- Community downloads (civitai, Reddit, Discord) — usually editor format, + must be loaded into ComfyUI then re-exported + +Editor format (top-level `nodes` and `links` arrays) is **not directly +executable**. The scripts detect this and tell you to re-export. + +### Step 2: See what's controllable + +```bash +python3 scripts/extract_schema.py workflow_api.json --summary-only +# → {"parameter_count": 12, "has_negative_prompt": true, "has_seed": true, ...} + +python3 scripts/extract_schema.py workflow_api.json +# → full schema with parameters, model deps, embedding refs +``` + +### Step 3: Run with parameters + +```bash +# Local (defaults to http://127.0.0.1:8188) +python3 scripts/run_workflow.py \ + --workflow workflow_api.json \ + --args '{"prompt": "a beautiful sunset over mountains", "seed": -1, "steps": 30}' \ + --output-dir ./outputs + +# Cloud (export API key once; uses correct /api routing automatically) +export COMFY_CLOUD_API_KEY="comfyui-..." +python3 scripts/run_workflow.py \ + --workflow workflow_api.json \ + --args '{"prompt": "..."}' \ + --host https://cloud.comfy.org \ + --output-dir ./outputs + +# Real-time progress via WebSocket (requires `pip install websocket-client`) +python3 scripts/run_workflow.py \ + --workflow flux_dev.json \ + --args '{"prompt": "..."}' \ + --ws + +# img2img / inpaint: pass --input-image to upload + reference automatically +python3 scripts/run_workflow.py \ + --workflow sdxl_img2img.json \ + --input-image image=./photo.png \ + --args '{"prompt": "make it watercolor", "denoise": 0.6}' + +# Batch / sweep: 8 random seeds, parallel up to cloud tier limit +python3 scripts/run_batch.py \ + --workflow sdxl.json \ + --args '{"prompt": "abstract"}' \ + --count 8 --randomize-seed --parallel 3 \ + --output-dir ./outputs/batch +``` + +`-1` for `seed` (or omitting it with `--randomize-seed`) generates a fresh +random seed per run. + +### Step 4: Present results + +The scripts emit JSON to stdout describing every output file: + +```json +{ + "status": "success", + "prompt_id": "abc-123", + "outputs": [ + {"file": "./outputs/sdxl_00001_.png", "node_id": "9", + "type": "image", "filename": "sdxl_00001_.png"} + ] +} +``` + +## Decision Tree + +| User says | Tool | Command | +|-----------|------|---------| +| **Lifecycle (use comfy-cli)** | | | +| "install ComfyUI" | comfy-cli | `bash scripts/comfyui_setup.sh` | +| "start ComfyUI" | comfy-cli | `comfy launch --background` | +| "stop ComfyUI" | comfy-cli | `comfy stop` | +| "install X node" | comfy-cli | `comfy node install ` | +| "download X model" | comfy-cli | `comfy model download --url --relative-path models/checkpoints` | +| "list installed models" | comfy-cli | `comfy model list` | +| "list installed nodes" | comfy-cli | `comfy node show installed` | +| **Execution (use scripts)** | | | +| "is everything ready?" | script | `health_check.py` (optionally with `--workflow X --smoke-test`) | +| "what can I change in this workflow?" | script | `extract_schema.py W.json` | +| "check if W's deps are met" | script | `check_deps.py W.json` | +| "fix missing deps" | script | `auto_fix_deps.py W.json` | +| "generate an image" | script | `run_workflow.py --workflow W --args '{...}'` | +| "use this image" (img2img) | script | `run_workflow.py --input-image image=./x.png ...` | +| "8 variations with random seeds" | script | `run_batch.py --count 8 --randomize-seed ...` | +| "show me live progress" | script | `ws_monitor.py --prompt-id ` | +| "fetch the error from job X" | script | `fetch_logs.py ` | +| **Direct REST** | | | +| "what's in the queue?" | REST | `curl http://HOST:8188/queue` (local) or `--host https://cloud.comfy.org` | +| "cancel that" | REST | `curl -X POST http://HOST:8188/interrupt` | +| "free GPU memory" | REST | `curl -X POST http://HOST:8188/free` | + +## Setup & Onboarding + +When a user asks to set up ComfyUI, **the FIRST thing to do is ask whether +they want Comfy Cloud (hosted, zero install, API key) or Local (install +ComfyUI on their machine)**. Don't start running install commands or hardware +checks until they've answered. + +**Official docs:** https://docs.comfy.org/installation +**CLI docs:** https://docs.comfy.org/comfy-cli/getting-started +**Cloud docs:** https://docs.comfy.org/get_started/cloud +**Cloud API:** https://docs.comfy.org/development/cloud/overview + +### Step 0: Ask Local vs Cloud (ALWAYS FIRST) + +Suggested script: + +> "Do you want to run ComfyUI locally on your machine, or use Comfy Cloud? +> +> - **Comfy Cloud** — hosted on RTX 6000 Pro GPUs, all common models pre-installed, +> zero setup. Requires an API key (paid subscription required to actually run +> workflows; free tier is read-only). Best if you don't have a capable GPU. +> - **Local** — free, but your machine MUST meet the hardware requirements: +> - NVIDIA GPU with **≥6 GB VRAM** (≥8 GB for SDXL, ≥12 GB for Flux/video), OR +> - AMD GPU with ROCm support (Linux), OR +> - Apple Silicon Mac (M1+) with **≥16 GB unified memory** (≥32 GB recommended). +> - Intel Macs and machines with no GPU will NOT work — use Cloud instead. +> +> Which would you like?" + +Routing: + +- **Cloud** → skip to **Path A**. +- **Local** → run hardware check first, then pick a path from Paths B–E based on the verdict. +- **Unsure** → run the hardware check and let the verdict decide. + +### Step 1: Verify Hardware (ONLY if user chose local) + +```bash +python3 scripts/hardware_check.py --json +# Optional: also probe `torch` for actual CUDA/MPS: +python3 scripts/hardware_check.py --json --check-pytorch +``` + +| Verdict | Meaning | Action | +|------------|---------------------------------------------------------------|--------| +| `ok` | ≥8 GB VRAM (discrete) OR ≥32 GB unified (Apple Silicon) | Local install — use `comfy_cli_flag` from report | +| `marginal` | SD1.5 works; SDXL tight; Flux/video unlikely | Local OK for light workflows, else **Path A (Cloud)** | +| `cloud` | No usable GPU, <6 GB VRAM, <16 GB Apple unified, Intel Mac, Rosetta Python | **Switch to Cloud** unless user explicitly forces local | + +The script also surfaces `wsl: true` (WSL2 with NVIDIA passthrough) and +`rosetta: true` (x86_64 Python on Apple Silicon — must reinstall as ARM64). + +If verdict is `cloud` but the user wants local, do not proceed silently. +Show the `notes` array verbatim and ask whether they want to (a) switch to +Cloud or (b) force a local install (will OOM or be unusably slow on modern models). + +### Choosing an Installation Path + +Use the hardware check first. The table below is the fallback for when the +user has already told you their hardware: + +| Situation | Recommended Path | +|-----------|------------------| +| `verdict: cloud` from hardware check | **Path A: Comfy Cloud** | +| No GPU / want to try without commitment | **Path A: Comfy Cloud** | +| Windows + NVIDIA + non-technical | **Path B: ComfyUI Desktop** | +| Windows + NVIDIA + technical | **Path C: Portable** or **Path D: comfy-cli** | +| Linux + any GPU | **Path D: comfy-cli** (easiest) | +| macOS + Apple Silicon | **Path B: Desktop** or **Path D: comfy-cli** | +| Headless / server / CI / agents | **Path D: comfy-cli** | + +For the fully automated path (hardware check → install → launch → verify): + +```bash +bash scripts/comfyui_setup.sh +# Or with overrides: +bash scripts/comfyui_setup.sh --m-series --port=8190 --workspace=/data/comfy +``` + +It runs `hardware_check.py` internally, refuses to install locally when the +verdict is `cloud` (unless `--force-cloud-override`), picks the right +`comfy-cli` flag, and prefers `pipx`/`uvx` over global `pip` to avoid polluting +system Python. + +--- + +### Path A: Comfy Cloud (No Local Install) + +For users without a capable GPU or who want zero setup. Hosted on RTX 6000 Pro. + +**Docs:** https://docs.comfy.org/get_started/cloud + +1. Sign up at https://comfy.org/cloud +2. Generate an API key at https://platform.comfy.org/login +3. Set the key: + ```bash + export COMFY_CLOUD_API_KEY="comfyui-xxxxxxxxxxxx" + ``` +4. Run workflows: + ```bash + python3 scripts/run_workflow.py \ + --workflow workflows/flux_dev_txt2img.json \ + --args '{"prompt": "..."}' \ + --host https://cloud.comfy.org \ + --output-dir ./outputs + ``` + +**Pricing:** https://www.comfy.org/cloud/pricing +**Concurrent jobs:** Free/Standard 1, Creator 3, Pro 5. Free tier +**cannot run workflows via API** — only browse models. Paid subscription +required for `/api/prompt`, `/api/upload/*`, `/api/view`, etc. + +--- + +### Path B: ComfyUI Desktop (Windows / macOS) + +One-click installer for non-technical users. Currently Beta. + +**Docs:** https://docs.comfy.org/installation/desktop +- **Windows (NVIDIA):** https://download.comfy.org/windows/nsis/x64 +- **macOS (Apple Silicon):** https://comfy.org + +Linux is **not supported** for Desktop — use Path D. + +--- + +### Path C: ComfyUI Portable (Windows Only) + +**Docs:** https://docs.comfy.org/installation/comfyui_portable_windows + +Download from https://github.com/comfyanonymous/ComfyUI/releases, extract, +run `run_nvidia_gpu.bat`. Update via `update/update_comfyui_stable.bat`. + +--- + +### Path D: comfy-cli (All Platforms — Recommended for Agents) + +The official CLI is the best path for headless/automated setups. + +**Docs:** https://docs.comfy.org/comfy-cli/getting-started + +#### Install comfy-cli + +```bash +# Recommended: +pipx install comfy-cli +# Or use uvx without installing: +uvx --from comfy-cli comfy --help +# Or (if pipx/uvx unavailable): +pip install --user comfy-cli +``` + +Disable analytics non-interactively: +```bash +comfy --skip-prompt tracking disable +``` + +#### Install ComfyUI + +```bash +comfy --skip-prompt install --nvidia # NVIDIA (CUDA) +comfy --skip-prompt install --amd # AMD (ROCm, Linux) +comfy --skip-prompt install --m-series # Apple Silicon (MPS) +comfy --skip-prompt install --cpu # CPU only (slow) +comfy --skip-prompt install --nvidia --fast-deps # uv-based dep resolution +``` + +Default location: `~/comfy/ComfyUI` (Linux), `~/Documents/comfy/ComfyUI` +(macOS/Win). Override with `comfy --workspace /custom/path install`. + +#### Launch / verify + +```bash +comfy launch --background # background daemon on :8188 +comfy launch -- --listen 0.0.0.0 --port 8190 # LAN-accessible custom port +curl -s http://127.0.0.1:8188/system_stats # health check +``` + +--- + +### Path E: Manual Install (Advanced / Unsupported Hardware) + +For Ascend NPU, Cambricon MLU, Intel Arc, or other unsupported hardware. + +**Docs:** https://docs.comfy.org/installation/manual_install + +```bash +git clone https://github.com/comfyanonymous/ComfyUI.git +cd ComfyUI +pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu130 +pip install -r requirements.txt +python main.py +``` + +--- + +### Post-Install: Download Models + +```bash +# SDXL (general purpose, ~6.5 GB) +comfy model download \ + --url "https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/resolve/main/sd_xl_base_1.0.safetensors" \ + --relative-path models/checkpoints + +# SD 1.5 (lighter, ~4 GB, good for 6 GB cards) +comfy model download \ + --url "https://huggingface.co/stable-diffusion-v1-5/stable-diffusion-v1-5/resolve/main/v1-5-pruned-emaonly.safetensors" \ + --relative-path models/checkpoints + +# Flux Dev fp8 (smaller variant, ~12 GB) +comfy model download \ + --url "https://huggingface.co/Comfy-Org/flux1-dev/resolve/main/flux1-dev-fp8.safetensors" \ + --relative-path models/checkpoints + +# CivitAI (set token first): +comfy model download \ + --url "https://civitai.com/api/download/models/128713" \ + --relative-path models/checkpoints \ + --set-civitai-api-token "YOUR_TOKEN" +``` + +List installed: `comfy model list`. + +### Post-Install: Install Custom Nodes + +```bash +comfy node install comfyui-impact-pack # popular utility pack +comfy node install comfyui-animatediff-evolved # video generation +comfy node install comfyui-controlnet-aux # ControlNet preprocessors +comfy node install comfyui-essentials # common helpers +comfy node update all +comfy node install-deps --workflow=workflow.json # install everything a workflow needs +``` + +### Post-Install: Verify + +```bash +python3 scripts/health_check.py +# → comfy_cli on PATH? server reachable? checkpoints? smoke test? + +python3 scripts/check_deps.py my_workflow.json +# → are this workflow's nodes/models/embeddings installed? + +python3 scripts/run_workflow.py \ + --workflow workflows/sd15_txt2img.json \ + --args '{"prompt": "test", "steps": 4}' \ + --output-dir ./test-outputs +``` + +## Image Upload (img2img / Inpainting) + +The simplest way is to use `--input-image` with `run_workflow.py`: + +```bash +python3 scripts/run_workflow.py \ + --workflow workflows/sdxl_img2img.json \ + --input-image image=./photo.png \ + --args '{"prompt": "make it cyberpunk", "denoise": 0.6}' +``` + +The flag uploads `photo.png`, then injects its server-side filename into +whatever schema parameter is named `image`. For inpainting, pass both: + +```bash +python3 scripts/run_workflow.py \ + --workflow workflows/sdxl_inpaint.json \ + --input-image image=./photo.png \ + --input-image mask_image=./mask.png \ + --args '{"prompt": "fill with flowers"}' +``` + +Manual upload via REST: +```bash +curl -X POST "http://127.0.0.1:8188/upload/image" \ + -F "image=@photo.png" -F "type=input" -F "overwrite=true" +# Returns: {"name": "photo.png", "subfolder": "", "type": "input"} + +# Cloud equivalent: +curl -X POST "https://cloud.comfy.org/api/upload/image" \ + -H "X-API-Key: $COMFY_CLOUD_API_KEY" \ + -F "image=@photo.png" -F "type=input" -F "overwrite=true" +``` + +## Cloud Specifics + +- **Base URL:** `https://cloud.comfy.org` +- **Auth:** `X-API-Key` header (or `?token=KEY` for WebSocket) +- **API key:** set `$COMFY_CLOUD_API_KEY` once and the scripts pick it up automatically +- **Output download:** `/api/view` returns a 302 to a signed URL; the scripts + follow it and strip `X-API-Key` before fetching from the storage backend + (don't leak the API key to S3/CloudFront). +- **Endpoint differences from local ComfyUI:** + - `/api/object_info`, `/api/queue`, `/api/userdata` — **403 on free tier**; + paid only. + - `/history` is renamed to `/history_v2` on cloud (the scripts route + automatically). + - `/models/` is renamed to `/experiment/models/` on cloud + (the scripts route automatically). + - `clientId` in WebSocket is currently ignored — all connections for a + user receive the same broadcast. Filter by `prompt_id` client-side. + - `subfolder` is accepted on uploads but ignored — cloud has a flat namespace. +- **Concurrent jobs:** Free/Standard: 1, Creator: 3, Pro: 5. Extras queue + automatically. Use `run_batch.py --parallel N` to saturate your tier. + +## Queue & System Management + +```bash +# Local +curl -s http://127.0.0.1:8188/queue | python3 -m json.tool +curl -X POST http://127.0.0.1:8188/queue -d '{"clear": true}' # cancel pending +curl -X POST http://127.0.0.1:8188/interrupt # cancel running +curl -X POST http://127.0.0.1:8188/free \ + -H "Content-Type: application/json" \ + -d '{"unload_models": true, "free_memory": true}' + +# Cloud — same paths under /api/, plus: +python3 scripts/fetch_logs.py --tail-queue --host https://cloud.comfy.org +``` + +## Pitfalls + +1. **API format required** — every script and the `/api/prompt` endpoint expect + API-format workflow JSON. The scripts detect editor format (top-level + `nodes` and `links` arrays) and tell you to re-export via + "Workflow → Export (API)" (newer UI) or "Save (API Format)" (older UI). + +2. **Server must be running** — all execution requires a live server. + `comfy launch --background` starts one. Verify with + `curl http://127.0.0.1:8188/system_stats`. + +3. **Model names are exact** — case-sensitive, includes file extension. + `check_deps.py` does fuzzy matching (with/without extension and folder + prefix), but the workflow itself must use the canonical name. Use + `comfy model list` to discover what's installed. + +4. **Missing custom nodes** — "class_type not found" means a required node + isn't installed. `check_deps.py` reports which package to install; + `auto_fix_deps.py` runs the install for you. + +5. **Working directory** — `comfy-cli` auto-detects the ComfyUI workspace. + If commands fail with "no workspace found", use + `comfy --workspace /path/to/ComfyUI ` or + `comfy set-default /path/to/ComfyUI`. + +6. **Cloud free-tier API limits** — `/api/prompt`, `/api/view`, `/api/upload/*`, + `/api/object_info` all return 403 on free accounts. `health_check.py` and + `check_deps.py` handle this gracefully and surface a clear message. + +7. **Timeout for video/audio workflows** — auto-detected when an output node + is `VHS_VideoCombine`, `SaveVideo`, etc.; the default jumps from 300 s to + 900 s. Override explicitly with `--timeout 1800`. + +8. **Path traversal in output filenames** — server-supplied filenames are + passed through `safe_path_join` to refuse anything escaping `--output-dir`. + Keep this protection on — workflows with custom save nodes can produce + arbitrary paths. + +9. **Workflow JSON is arbitrary code** — custom nodes run Python, so + submitting an unknown workflow has the same trust profile as `eval`. + Inspect workflows from untrusted sources before running. + +10. **Auto-randomized seed** — pass `seed: -1` in `--args` (or use + `--randomize-seed` and omit the seed) to get a fresh seed per run. + The actual seed is logged to stderr. + +11. **`tracking` prompt** — first run of `comfy` may prompt for analytics. + Use `comfy --skip-prompt tracking disable` to skip non-interactively. + `comfyui_setup.sh` does this for you. + +## Verification Checklist + +Use `python3 scripts/health_check.py` to run the whole list at once. Manual: + +- [ ] `hardware_check.py` verdict is `ok` OR the user explicitly chose Comfy Cloud +- [ ] `comfy --version` works (or `uvx --from comfy-cli comfy --help`) +- [ ] `curl http://HOST:PORT/system_stats` returns JSON +- [ ] `comfy model list` shows at least one checkpoint (local) OR + `/api/experiment/models/checkpoints` returns models (cloud) +- [ ] Workflow JSON is in API format +- [ ] `check_deps.py` reports `is_ready: true` (or only `node_check_skipped` + on cloud free tier) +- [ ] Test run with a small workflow completes; outputs land in `--output-dir` diff --git a/image-creation/comfyui/comfyui.zip b/image-creation/comfyui/comfyui.zip new file mode 100644 index 0000000..f35d8b3 Binary files /dev/null and b/image-creation/comfyui/comfyui.zip differ diff --git a/image-creation/comfyui/install.md b/image-creation/comfyui/install.md new file mode 100644 index 0000000..3e4163b --- /dev/null +++ b/image-creation/comfyui/install.md @@ -0,0 +1,2 @@ +使用 ComfyUI 生成图片、视频和音频，并支持安装、启动、管理节点与模型，以及通过参数注入运行工作流。 +使用官方 comfy-cli 负责生命周期管理，并通过直接 REST/WebSocket API 执行任务。 diff --git a/image-creation/comfyui/references/official-cli.md b/image-creation/comfyui/references/official-cli.md new file mode 100644 index 0000000..59a981b --- /dev/null +++ b/image-creation/comfyui/references/official-cli.md @@ -0,0 +1,255 @@ +# comfy-cli Command Reference + +Official CLI from [Comfy-Org/comfy-cli](https://github.com/Comfy-Org/comfy-cli). +Docs: https://docs.comfy.org/comfy-cli/getting-started + +## Installation + +Order of preference: + +```bash +pipx install comfy-cli # recommended (isolated env) +uvx --from comfy-cli comfy --help # zero-install via uv +pip install --user comfy-cli # fallback +``` + +The skill's `comfyui_setup.sh` picks the best available method. + +First run may prompt for analytics. Disable non-interactively: +```bash +comfy --skip-prompt tracking disable +``` + +## Global Options + +| Option | Description | +|--------|-------------| +| `--workspace ` | Target a specific ComfyUI workspace | +| `--recent` | Use most recently used workspace | +| `--here` | Use current directory as workspace | +| `--skip-prompt` | No interactive prompts (use defaults) | +| `-v` / `--version` | Print version | + +Workspace resolution priority: +1. `--workspace` (explicit path) +2. `--recent` (from config) +3. `--here` (cwd) +4. `comfy set-default` path +5. Most recently used +6. `~/comfy/ComfyUI` (Linux) or `~/Documents/comfy/ComfyUI` (macOS/Win) + +## Lifecycle Commands + +### `comfy install` + +Download and install ComfyUI + ComfyUI-Manager. + +```bash +comfy install # interactive GPU selection +comfy install --nvidia +comfy install --amd # ROCm (Linux) +comfy install --m-series # Apple Silicon (MPS) +comfy install --cpu # CPU only (slow) +comfy install --fast-deps # use uv for deps +comfy install --skip-manager # skip ComfyUI-Manager +``` + +| Option | Description | +|--------|-------------| +| `--nvidia` / `--amd` / `--m-series` / `--cpu` | GPU type | +| `--cuda-version` | 11.8, 12.1, 12.4, 12.6, 12.8, 12.9, 13.0 | +| `--rocm-version` | 6.1, 6.2, 6.3, 7.0, 7.1 | +| `--fast-deps` | uv-based dependency resolution | +| `--skip-manager` | Don't install ComfyUI-Manager | +| `--skip-torch-or-directml` | Skip PyTorch install | +| `--version ` | `0.2.0`, `latest`, `nightly` | +| `--commit ` | Install specific commit | +| `--pr "#1234"` | Install from a PR | +| `--restore` | Restore deps for existing install | + +### `comfy launch` + +```bash +comfy launch # foreground :8188 +comfy launch --background # background daemon +comfy launch -- --listen 0.0.0.0 # LAN-accessible +comfy launch -- --port 8190 # custom port +comfy launch -- --cpu # force CPU mode +comfy launch -- --lowvram # 6 GB cards +comfy launch --background -- --listen 0.0.0.0 --port 8190 +``` + +Common extra args after `--`: `--listen`, `--port`, `--cpu`, `--lowvram`, +`--novram`, `--fp16-vae`, `--force-fp32`, `--disable-cuda-malloc`. + +### `comfy stop` + +```bash +comfy stop +``` + +### `comfy run` + +Submit a raw workflow JSON to a running server. **Limited** — no parameter +injection, no structured output download. For agents, use +`scripts/run_workflow.py` instead. + +```bash +comfy run --workflow workflow_api.json +comfy run --workflow workflow_api.json --host 10.0.0.5 --port 8188 +comfy run --workflow workflow_api.json --timeout 300 --wait +``` + +### `comfy which` + +```bash +comfy which # show targeted workspace +comfy --recent which +``` + +### `comfy set-default` + +```bash +comfy set-default /path/to/ComfyUI +comfy set-default /path/to/ComfyUI --launch-extras="--listen 0.0.0.0" +``` + +### `comfy update` + +```bash +comfy update # update ComfyUI core +comfy node update all # update all custom nodes +``` + +--- + +## `comfy node` — Custom Node Management + +All node operations use ComfyUI-Manager (`cm-cli`) under the hood. + +```bash +comfy node show installed # list installed +comfy node show enabled # list enabled +comfy node show all # all available in registry +comfy node simple-show installed # compact list + +comfy node install comfyui-impact-pack +comfy node install --uv-compile # ComfyUI-Manager v4.1+ unified resolver +comfy node uninstall +comfy node update | all +comfy node enable +comfy node disable +comfy node fix # fix broken deps + +comfy node install-deps --workflow=workflow.json +comfy node deps-in-workflow --workflow=w.json --output=deps.json + +comfy node save-snapshot +comfy node restore-snapshot + +comfy node bisect start # binary-search a culprit node +comfy node bisect good +comfy node bisect bad +comfy node bisect reset +``` + +### Dependency Resolution Options + +| Flag | Description | +|------|-------------| +| `--fast-deps` | comfy-cli built-in uv resolver | +| `--uv-compile` | ComfyUI-Manager v4.1+ unified resolver (recommended) | +| `--no-deps` | Skip dep installation | + +Make `uv-compile` default: `comfy manager uv-compile-default true` + +--- + +## `comfy model` — Model Management + +```bash +comfy model list +comfy model list --relative-path models/checkpoints + +comfy model download --url +comfy model download --url --relative-path models/loras +comfy model download --url --filename custom_name.safetensors + +comfy model remove # interactive +comfy model remove --relative-path models/checkpoints --model-names "model.safetensors" +``` + +| Option | Description | +|--------|-------------| +| `--url` | Download URL (CivitAI, HuggingFace, direct) | +| `--relative-path` | Subdirectory under workspace (e.g. `models/checkpoints`) | +| `--filename` | Custom save filename | +| `--set-civitai-api-token` | Persist CivitAI token | +| `--set-hf-api-token` | Persist HuggingFace token | +| `--downloader` | `httpx` (default) or `aria2` | + +Standard model directories: +``` +ComfyUI/models/ +├── checkpoints/ # Full model files +├── loras/ # LoRA adapters +├── vae/ # VAE models +├── controlnet/ # ControlNet models +├── clip/ # CLIP / T5 text encoders +├── clip_vision/ # CLIP vision encoders +├── upscale_models/ # ESRGAN / SwinIR / etc. +├── embeddings/ # Textual inversion embeddings +├── unet/ # Standalone UNet weights +├── diffusion_models/ # Flux / SD3 / Wan diffusion models +├── animatediff_models/ # AnimateDiff motion modules +├── ipadapter/ # IPAdapter weights +└── style_models/ # Style adapters +``` + +--- + +## `comfy manager` — ComfyUI-Manager Settings + +```bash +comfy manager disable # disable Manager completely +comfy manager enable-gui # enable new GUI +comfy manager disable-gui # API-only +comfy manager enable-legacy-gui # legacy GUI +comfy manager uv-compile-default true # make --uv-compile the default +comfy manager clear # clear startup action +``` + +--- + +## `comfy pr-cache` — Frontend PR Cache + +```bash +comfy pr-cache list +comfy pr-cache clean +comfy pr-cache clean 456 +``` + +Cache expires after 7 days; max 10 builds. + +--- + +## Configuration + +| OS | Path | +|----|------| +| Linux | `~/.config/comfy-cli/config.ini` | +| macOS | `~/Library/Application Support/comfy-cli/config.ini` | +| Windows | `~/AppData/Local/comfy-cli/config.ini` | + +Stores: default workspace, recent workspace, background server PID, API +tokens, manager GUI mode, launch extras. + +## Discovery + +Custom-node registry: +- https://registry.comfy.org/ + +Model browsers: +- https://huggingface.co/models +- https://civitai.com (NSFW; requires API token for many) +- https://comfyworkflows.com (community workflows) diff --git a/image-creation/comfyui/references/rest-api.md b/image-creation/comfyui/references/rest-api.md new file mode 100644 index 0000000..64091c9 --- /dev/null +++ b/image-creation/comfyui/references/rest-api.md @@ -0,0 +1,312 @@ +# ComfyUI REST + WebSocket API Reference + +ComfyUI exposes a REST + WebSocket interface for workflow execution and +management. **The same surface is used locally and on Comfy Cloud, with +auth/path differences.** + +## Connection + +| | Local ComfyUI | Comfy Cloud | +|---|---|---| +| Base URL | `http://127.0.0.1:8188` | `https://cloud.comfy.org` | +| API path prefix | none (`/prompt`, `/view`, …) | `/api/...` (`/api/prompt`, `/api/view`, …) | +| Auth | none (or bearer token if configured) | `X-API-Key` header | +| WebSocket | `ws://host:port/ws?clientId={uuid}` | `wss://cloud.comfy.org/ws?clientId={uuid}&token={API_KEY}` | +| `/api/view` response | direct bytes | 302 redirect → signed URL (use `curl -L`) | + +The skill scripts route URLs automatically via `_common.resolve_url()`. + +## Endpoint differences on Comfy Cloud + +The cloud surface diverges from local ComfyUI in several ways. The skill +scripts handle these transparently; document them here so anyone calling +`curl` directly knows. + +| Local path | Cloud path | Notes | +|------------|-----------|-------| +| `/system_stats` | `/api/system_stats` | Cloud version is **public** (no auth required) | +| `/object_info` | `/api/object_info` | **Paid tier only** — free returns 403 | +| `/queue` | `/api/queue` | Paid tier only | +| `/userdata` | `/api/userdata` | Paid tier only | +| `/prompt` (POST) | `/api/prompt` (POST) | Paid tier only | +| `/upload/image` | `/api/upload/image` | Paid tier only; `subfolder` accepted but ignored | +| `/upload/mask` | `/api/upload/mask` | Same as above | +| `/view` | `/api/view` | Paid tier only; **returns 302** to signed URL | +| `/history` | `/api/history_v2` | **Renamed**; old path returns 404 | +| `/history/{id}` | `/api/history_v2/{id}` or `/api/jobs/{id}` | Both work; `/jobs` returns full job | +| `/models` | `/api/experiment/models` | **Renamed** | +| `/models/{folder}` | `/api/experiment/models/{folder}` | **Renamed**; response shape differs (see below) | + +### Cloud model-list response shape + +- **Local:** `["a.safetensors", "b.safetensors", …]` — flat list of strings. +- **Cloud:** `[{"name": "a.safetensors", "pathIndex": 0}, …]` — list of objects. +- **Cloud 404 with `code: "folder_not_found"`** — folder is empty or unknown, + not an "endpoint missing" error. Distinguish by reading the body. + +The skill helper `_common.parse_model_list()` normalizes both. + +## Workflow Execution + +### Submit Workflow + +```bash +# Local +curl -X POST "http://127.0.0.1:8188/prompt" \ + -H "Content-Type: application/json" \ + -d '{"prompt": '"$(cat workflow_api.json)"', "client_id": "'"$(uuidgen)"'"}' + +# Cloud +curl -X POST "https://cloud.comfy.org/api/prompt" \ + -H "X-API-Key: $COMFY_CLOUD_API_KEY" \ + -H "Content-Type: application/json" \ + -d '{"prompt": '"$(cat workflow_api.json)"'}' +``` + +**Response:** +```json +{"prompt_id": "abc-123-def", "number": 1, "node_errors": {}} +``` + +If `node_errors` is non-empty, the workflow has validation errors (missing +nodes, bad inputs). + +### Check Job Status (Cloud) + +```bash +curl -X GET "https://cloud.comfy.org/api/job/{prompt_id}/status" \ + -H "X-API-Key: $COMFY_CLOUD_API_KEY" +``` + +| Status | Description | +| ------------- | ---------------------------------- | +| `pending` | Job is queued and waiting to start | +| `in_progress` | Job is currently executing | +| `completed` | Job finished successfully | +| `failed` | Job encountered an error | +| `cancelled` | Job was cancelled by user | + +### Job detail with outputs (Cloud) + +```bash +curl -X GET "https://cloud.comfy.org/api/jobs/{prompt_id}" \ + -H "X-API-Key: $COMFY_CLOUD_API_KEY" +``` + +Response includes `outputs` keyed by node ID. Cloud uses `video` (singular) +in the output structure; local uses `videos` (plural). The skill scripts +accept both. + +### Get History (Local) + +```bash +curl -s "http://127.0.0.1:8188/history" # all +curl -s "http://127.0.0.1:8188/history/{id}" # one prompt_id +``` + +Local entry shape: +```json +{ + "": { + "prompt": [...], + "outputs": {"": {"images": [...]}}, + "status": { + "status_str": "success" | "error", + "completed": true | false, + "messages": [["execution_start", {...}], ["execution_error", {...}], …] + } + } +} +``` + +**Important:** when reading status, check `status_str == "error"` BEFORE +checking `completed`, because both can be true for failed runs. + +### Download Output + +```bash +# Local (direct bytes) +curl -s "http://127.0.0.1:8188/view?filename=ComfyUI_00001_.png&subfolder=&type=output" \ + -o output.png + +# Cloud (302 → signed URL; -L follows; STRIP X-API-Key for the second hop) +curl -L "https://cloud.comfy.org/api/view?filename=...&type=output" \ + -H "X-API-Key: $COMFY_CLOUD_API_KEY" \ + -o output.png +``` + +The skill's `run_workflow.py` strips `X-API-Key` automatically on the +cross-host redirect, so the signed URL never sees your auth. + +## WebSocket Monitoring + +Connect for real-time execution events. + +```bash +# Local +wscat -c "ws://127.0.0.1:8188/ws?clientId=MY-UUID" + +# Cloud +wscat -c "wss://cloud.comfy.org/ws?clientId=MY-UUID&token=$COMFY_CLOUD_API_KEY" +``` + +**Note:** on Cloud the `clientId` is currently ignored — all messages for a +user are broadcast to every connection. Filter messages client-side by +`data.prompt_id`. + +### JSON Message Types + +| Type | When | Key Fields | +|------|------|------------| +| `status` | Queue change | `status.exec_info.queue_remaining` | +| `notification` | User-friendly status string | `value` | +| `execution_start` | Workflow begins | `prompt_id` | +| `executing` | Node running (or end-of-run if `node` is null on local) | `node`, `prompt_id` | +| `progress` | Sampling steps | `node`, `value`, `max` | +| `progress_state` | Extended progress with per-node metadata | `nodes` (dict) | +| `executed` | Node output ready | `node`, `output` (with `images`/`video`/etc.) | +| `execution_cached` | Nodes skipped because of cache | `nodes` (list of IDs) | +| `execution_success` | All done | `prompt_id` | +| `execution_error` | Failure | `exception_type`, `exception_message`, `traceback`, `node_id` | +| `execution_interrupted` | Cancelled | `prompt_id` | + +### Binary Frames (Preview Images) + +| Type code | Meaning | +|-----------|---------| +| `0x00000001` | `PREVIEW_IMAGE` — `[type:4][image_type:4][data]` (image_type 1=JPEG, 2=PNG) | +| `0x00000003` | `TEXT` — `[type:4][nid_len:4][nid][text]` (UTF-8) | +| `0x00000004` | `PREVIEW_IMAGE_WITH_METADATA` — `[type:4][meta_len:4][json][image_data]` | + +`scripts/ws_monitor.py --previews ` saves preview frames to disk. + +## File Upload + +```bash +# Image +curl -X POST "http://127.0.0.1:8188/upload/image" \ + -F "image=@photo.png" -F "type=input" -F "overwrite=true" +# Returns: {"name": "photo.png", "subfolder": "", "type": "input"} + +# Mask (linked to a previously uploaded image) +curl -X POST "http://127.0.0.1:8188/upload/mask" \ + -F "image=@mask.png" -F "type=input" \ + -F 'original_ref={"filename":"photo.png","subfolder":"","type":"input"}' +``` + +Cloud equivalent: prepend `https://cloud.comfy.org/api` and add `-H "X-API-Key: $COMFY_CLOUD_API_KEY"`. + +## Node & Model Discovery + +```bash +# All node types and their input specs +curl -s "http://127.0.0.1:8188/object_info" | python3 -m json.tool + +# Specific node +curl -s "http://127.0.0.1:8188/object_info/KSampler" + +# Models per folder (local) +curl -s "http://127.0.0.1:8188/models/checkpoints" +curl -s "http://127.0.0.1:8188/models/loras" + +# Models per folder (cloud — note the experimental prefix) +curl -s "https://cloud.comfy.org/api/experiment/models/checkpoints" \ + -H "X-API-Key: $COMFY_CLOUD_API_KEY" +``` + +## Queue Management + +```bash +# View queue +curl -s "http://127.0.0.1:8188/queue" + +# Clear all pending +curl -X POST "http://127.0.0.1:8188/queue" \ + -H "Content-Type: application/json" \ + -d '{"clear": true}' + +# Delete specific items +curl -X POST "http://127.0.0.1:8188/queue" \ + -H "Content-Type: application/json" \ + -d '{"delete": ["prompt_id_1", "prompt_id_2"]}' + +# Cancel currently-running job +curl -X POST "http://127.0.0.1:8188/interrupt" +``` + +## System Management + +```bash +# Stats (VRAM, RAM, GPU, ComfyUI version) +curl -s "http://127.0.0.1:8188/system_stats" + +# Free GPU memory +curl -X POST "http://127.0.0.1:8188/free" \ + -H "Content-Type: application/json" \ + -d '{"unload_models": true, "free_memory": true}' +``` + +## ComfyUI-Manager Endpoints (Optional) + +These require ComfyUI-Manager installed. Useful for installing nodes/models +via the API instead of `comfy-cli`. + +```bash +# Install a custom node from a git URL +curl -X POST "http://127.0.0.1:8188/manager/queue/install" \ + -H "Content-Type: application/json" \ + -d '{"git_url": "https://github.com/user/comfyui-node.git"}' + +# Check install queue status +curl -s "http://127.0.0.1:8188/manager/queue/status" + +# Install model +curl -X POST "http://127.0.0.1:8188/manager/queue/install_model" \ + -H "Content-Type: application/json" \ + -d '{"url": "https://...", "path": "models/checkpoints", "filename": "model.safetensors"}' +``` + +## POST /prompt Payload Format + +```json +{ + "prompt": { + "3": { + "class_type": "KSampler", + "inputs": { + "seed": 42, + "steps": 20, + "cfg": 7.5, + "sampler_name": "euler", + "scheduler": "normal", + "denoise": 1.0, + "model": ["4", 0], + "positive": ["6", 0], + "negative": ["7", 0], + "latent_image": ["5", 0] + } + } + }, + "client_id": "unique-uuid-for-ws-filtering", + "extra_data": { + "api_key_comfy_org": "optional-PARTNER-NODE-key (NOT the cloud auth key)" + } +} +``` + +- `prompt`: workflow graph in API format +- `client_id`: UUID — local server uses it to filter WebSocket events; cloud + ignores it. +- `extra_data.api_key_comfy_org`: ONLY required when the workflow uses + partner nodes (Flux Pro, Ideogram, etc.). Don't conflate with `X-API-Key`. + +## Error Categories (cloud `execution_error` `exception_type`) + +| Type | Meaning | +|------|---------| +| `ValidationError` | Bad workflow / inputs (often nicer to surface from `node_errors`) | +| `ModelDownloadError` | Required model not available | +| `ImageDownloadError` | Failed to fetch input image from URL | +| `OOMError` | Out of GPU memory | +| `InsufficientFundsError` | Account balance too low (partner nodes) | +| `InactiveSubscriptionError` | Subscription not active | diff --git a/image-creation/comfyui/references/template-integrity.md b/image-creation/comfyui/references/template-integrity.md new file mode 100644 index 0000000..050e3e6 --- /dev/null +++ b/image-creation/comfyui/references/template-integrity.md @@ -0,0 +1,243 @@ +# ComfyUI Workflow-Template Integrity + +> **Authored by [@purzbeats](https://github.com/purzbeats)** — adapted from +> [purzbeats/hermes-agent-comfyui-helper](https://github.com/purzbeats/hermes-agent-comfyui-helper). +> Use this reference when converting workflows from the official +> `comfyui-workflow-templates` package (editor format) into API format for +> submission via `/api/prompt`. The conversion has subtle gotchas that cause +> hard-to-diagnose validation errors if you don't follow these rules. + +## Background + +The official ComfyUI template package (`comfyui-workflow-templates`, currently +v0.9.69) is installed inside the ComfyUI venv at a path like: + +``` +/.venv/lib/python3.*/site-packages/comfyui_workflow_templates_*/templates/ +``` + +The exact path depends on how ComfyUI was installed (comfy-cli default, +Comfy Desktop, manual venv, etc.). Find it once with: + +```bash +comfy --workspace run-python -c "import comfyui_workflow_templates, pathlib; print(pathlib.Path(comfyui_workflow_templates.__file__).parent / 'templates')" +``` + +Templates ship in **editor format** — `nodes` / `links` arrays inside +`data['definitions']['subgraphs'][0]`. They must be converted to **API +format** (a `node_id -> {class_type, inputs}` mapping) before submission. + +--- + +## RULE #1: Use templates AS CLOSE TO ORIGINAL AS POSSIBLE + +- **Never strip, simplify, or "minimize" nodes** from a template. +- Full template architecture (dual-pass pipelines, LoRA chains, distilled + sigmas, conditioning paths) is intentional — removing any part breaks quality. +- If an image-dependent path exists but the task is text-to-video, **leave + it wired with the bypass toggle enabled** — don't remove the nodes. +- Only change: prompt text, seed, and dimensions (when explicitly requested). + +## RULE #2: Server validation errors are the source of truth + +When a workflow submission fails, the server response looks like: + +```json +{ + "node_errors": { + "238": { + "errors": [{ + "message": "Required input is missing", + "details": "width", + "extra_info": { "input_name": "resize_type.width" } + }] + } + } +} +``` + +**The `extra_info.input_name` field tells you EXACTLY what JSON key the server +wants. Use it literally.** If it says `"values.a"` or `"resize_type.width"`, +those are the actual key names in the JSON object. Do not "simplify" them to +flat names based on assumptions about what the field "should" be called. + +## RULE #3: Don't rebuild from scratch — patch the failing nodes + +Every regeneration from the template reintroduces the same bugs. Instead: + +1. Submit the workflow once. +2. Read the server error details for exact key names. +3. Use targeted patch/fix calls against the workflow file on disk. +4. Resubmit and check if errors resolved. + +--- + +## Reroute nodes: bypass, don't delete + +Most servers (local, Cloud) don't have a `Reroute` node type. When converting +a template: + +1. Find what feeds into the Reroute by looking at links where + `target_id` = the Reroute node ID. +2. Replace all inputs referencing the Reroute with + `[source_node_id, source_slot]`. +3. Delete the Reroute node from the API mapping. + +**Real example — LTX 2.3 t2v template:** + +- Reroute node 255 receives VAE from `CheckpointLoaderSimple 236` slot 2. +- Three nodes reference Reroute 255 for their VAE input: + `LTXVImgToVideoInplace` (230), `LTXVLatentUpsampler` (253), + `VAEDecodeTiled` (251). +- Fix: replace all occurrences of `vae: ["255", 0]` with `vae: ["236", 2]`. +- `CheckpointLoaderSimple` slot 2 = VAE (not slot 0 = MODEL). + +| | | +|---|---| +| ❌ Wrong | `vae: ["236", 0]` → `MODELV mismatch input_type(VAE)` | +| ✅ Correct | `vae: ["236", 2]` | + +--- + +## Dynamic template nodes: dotted key names are correct + +### ComfyMathExpression (COMFY_AUTOGROW_V3) + +```json +{ + "class_type": "ComfyMathExpression", + "inputs": { + "expression": "a/2", + "values.a": ["257", 0] + } +} +``` + +- `values` is a `COMFY_AUTOGROW_V3` template. +- Input names in links are `values.a`, `values.b`, etc. +- **Keep the dotted format as JSON keys.** +- Do NOT convert to `{"values": {"a": ...}}` or flatten to just `"a"`. + +### ResizeImageMaskNode (COMFY_DYNAMICCOMBO_V3) + +```json +{ + "class_type": "ResizeImageMaskNode", + "inputs": { + "input": ["276", 0], + "scale_method": "lanczos", + "resize_type": "scale dimensions", + "resize_type.width": 1920, + "resize_type.height": 1088, + "resize_type.crop": "center" + } +} +``` + +- `resize_type` is a `COMFY_DYNAMICCOMBO_V3`. +- Mode-specific fields: `resize_type.width`, `resize_type.height`, `resize_type.crop`. +- `scale_method` options: `"nearest-exact"`, `"bilinear"`, `"area"`, `"bicubic"`, `"lanczos"`. +- **Keep the dotted format as JSON keys.** +- Do NOT flatten `resize_type.width` to just `"width"`. + +--- + +## Conversion recipe + +1. Load template from the installed package path. +2. Parse `data['definitions']['subgraphs'][0]`. +3. For each node (skip Reroute): + - Resolve linked inputs from `sg['links']` dict. + - Map `widgets_values` to input field names. + - Keep all dotted key names as-is from the template. +4. Bypass Reroute: trace source, replace references. +5. Change only: prompt text, seed values, and user-requested parameters. +6. Add `SaveVideo` terminal node if template uses only `CreateVideo`. +7. Submit → read errors → patch specific nodes → resubmit. + +## What to NEVER change in a template + +| Element | Why | +|---------|-----| +| Node topology | Graph is designed for the specific model | +| Sigmas values | Tuned for the model/sampler combination | +| LoRA/distilled paths | Required for quality, even if they look unused | +| Model parameters (cfg, steps, shifts) | Model-specific | +| Conditioning chains (zero-out, crop guides) | Required for correct conditioning | +| Pass-through wiring | Don't remove nodes, bypass them | + +--- + +## Cloud compatibility (verified May 2025) + +The full LTX 2.3 T2V template (`video_ltx2_3_t2v.json`) runs **without +modification** on Comfy Cloud. + +**Confirmed working on Cloud (all custom nodes available):** +`ComfyMathExpression`, `ResizeImageMaskNode`, `ResizeImagesByLongerEdge`, +`PrimitiveInt`, `PrimitiveStringMultiline`, `PrimitiveBoolean`, `SaveVideo`, +`LTXVCropGuides`, `LTXVImgToVideoInplace`, `LTXVConcatAVLatent`, +`LTXVSeparateAVLatent`, `LTXVLatentUpsampler`, `LTXVAudioVAELoader`, +`LTXVAudioVAEDecode`, `LTXVEmptyLatentAudio`, `LTXVPreprocess`, +`LTXVConditioning`, `ManualSigmas`, `LTXAVTextEncoderLoader`, plus all core +nodes. + +**Cloud vs Local for LTX 2.3 (768x512):** + +- Cloud: ~39s per video (4x faster). +- Local (RTX 5090): ~160s per video. +- `example.png` placeholder works on Cloud for bypassed image-dependent paths. +- Submission format is **identical** between local and Cloud: + `{"prompt": wf, "extra_data": {}}` to `/api/prompt`. +- Free tier = 1 concurrent job. + +**Cloud submission pitfalls:** + +- `/api/object_info/` returns 404 on free tier — can't query node + schemas remotely, but the workflow runs fine anyway. Always probe + `object_info` locally before building workflows. +- Cloud is ~4x faster — prefer Cloud for batch runs unless local is needed + for debugging. +- Cloud `/api/view` returns **302 redirect to signed GCS URL** — use + `curl -s -L` to follow and download. Python `urllib` fails with 401 + (forwards auth headers to GCS CDN). +- `COMFY_CLOUD_API_KEY` is only in the terminal/bash env, not in the Python + sandbox. Use subprocess or terminal scripts for Cloud API calls. +- Cloud free tier processes jobs **sequentially** (1 at a time). Submit all, + then poll history. +- LTX 2.3 at **1920x1080 OOMs locally** (even RTX 5090) — upscaler pass + exceeds VRAM. Prefer Cloud for 1080p; use 1280x720 locally (~90s/video). + +--- + +## FFmpeg stitch settings (Discord-compatible) + +Generated ComfyUI videos often use `yuv444p` pixel format which does NOT work +on Discord. Re-encode with: + +```bash +ffmpeg -y -i input.mp4 \ + -c:v libx264 -profile:v main -preset medium -crf 13 -pix_fmt yuv420p \ + -c:a aac -b:a 192k \ + output_discord.mp4 +``` + +Key settings: + +- `-pix_fmt yuv420p` — **required for Discord**, ComfyUI outputs `yuv444p` by default. +- `-crf 13` — high quality without massive file size (default 23 is too lossy). +- `-profile:v main` — widely compatible. + +For multi-video crossfade stitching, chain `xfade` (video) and `acrossfade` +(audio): + +```bash +ffmpeg -y -i a.mp4 -i b.mp4 -i c.mp4 \ + -filter_complex "[0:v][1:v]xfade=transition=fade:duration=1:offset=3.04[v1];[v1][2:v]xfade=transition=fade:duration=1:offset=6.08[vout];[0:a][1:a]acrossfade=duration=1:c1=tri:c2=tri[a1];[a1][2:a]acrossfade=duration=1:c1=tri:c2=tri[aout]" \ + -map "[vout]" -map "[aout]" \ + -c:v libx264 -profile:v main -crf 13 -pix_fmt yuv420p \ + -c:a aac -b:a 192k \ + output.mp4 +``` + +Offset for xfade #N = `(N+1) × duration - N × overlap`. diff --git a/image-creation/comfyui/references/workflow-format.md b/image-creation/comfyui/references/workflow-format.md new file mode 100644 index 0000000..e8343de --- /dev/null +++ b/image-creation/comfyui/references/workflow-format.md @@ -0,0 +1,226 @@ +# ComfyUI Workflow JSON Format + +## Two Formats — Only API Format Is Executable + +**API format** is required for `/api/prompt` and every script in this skill. +The web UI also produces an "editor format" used for visual editing, which +**cannot** be submitted directly. + +### API Format + +Top-level keys are string node IDs. Each node has `class_type` and `inputs`: + +```json +{ + "3": { + "class_type": "KSampler", + "inputs": { + "seed": 156680208700286, + "steps": 20, + "cfg": 8, + "sampler_name": "euler", + "scheduler": "normal", + "denoise": 1.0, + "model": ["4", 0], + "positive": ["6", 0], + "negative": ["7", 0], + "latent_image": ["5", 0] + }, + "_meta": {"title": "KSampler"} + }, + "4": { + "class_type": "CheckpointLoaderSimple", + "inputs": {"ckpt_name": "v1-5-pruned-emaonly.safetensors"} + } +} +``` + +**Detection:** every top-level value has `class_type`. The skill's +`_common.is_api_format()` does this check. + +### Editor Format (not directly executable) + +Has `nodes[]` and `links[]` arrays — the visual graph. To convert: open in +ComfyUI's web UI and use **Workflow → Export (API)** (newer UI) or the +"Save (API Format)" button (older UI). + +**Detection:** top-level has `"nodes"` and `"links"` keys. + +## Inputs: Literals vs Links + +```json +"inputs": { + "text": "a cat", // literal — modifiable + "seed": 42, // literal — modifiable + "clip": ["4", 1] // link — wiring; do NOT overwrite +} +``` + +Links are length-2 arrays of `[upstream_node_id, output_slot]`. The skill's +parameter injector refuses to overwrite a link with a literal (logs a +warning and skips). + +## Common Node Types and Their Controllable Parameters + +The full catalog lives in `scripts/_common.py` (`PARAM_PATTERNS` and +`MODEL_LOADERS`). Highlights: + +### Text Prompts + +| Node Class | Key Fields | +|------------|------------| +| `CLIPTextEncode` | `text` | +| `CLIPTextEncodeSDXL` | `text_g`, `text_l`, `width`, `height` | +| `CLIPTextEncodeFlux` | `clip_l`, `t5xxl`, `guidance` | + +To distinguish positive from negative the skill traces `KSampler.negative` +back through Reroute / Primitive nodes to the source CLIPTextEncode. Falls +back to `_meta.title` heuristics ("negative", "neg", "anti"). + +### Sampling + +| Node Class | Key Fields | +|------------|------------| +| `KSampler` | `seed`, `steps`, `cfg`, `sampler_name`, `scheduler`, `denoise` | +| `KSamplerAdvanced` | `noise_seed`, `steps`, `cfg`, `start_at_step`, `end_at_step` | +| `SamplerCustom` | `noise_seed`, `cfg`, `sampler`, `sigmas` | +| `SamplerCustomAdvanced` | `noise_seed` (via RandomNoise input) | +| `RandomNoise` | `noise_seed` | +| `BasicScheduler` | `steps`, `scheduler`, `denoise` | +| `KSamplerSelect` | `sampler_name` | +| `BasicGuider` / `CFGGuider` | `cfg` | +| `ModelSamplingFlux` | `max_shift`, `base_shift`, `width`, `height` | +| `SDTurboScheduler` | `steps`, `denoise` | + +### Latent / Dimensions + +| Node Class | Key Fields | +|------------|------------| +| `EmptyLatentImage` | `width`, `height`, `batch_size` | +| `EmptySD3LatentImage` | `width`, `height`, `batch_size` | +| `EmptyHunyuanLatentVideo` | `width`, `height`, `length`, `batch_size` | +| `EmptyMochiLatentVideo` | `width`, `height`, `length`, `batch_size` | +| `EmptyLTXVLatentVideo` | `width`, `height`, `length`, `batch_size` | + +### Model Loading + +| Node Class | Key Fields | Folder | +|------------|------------|--------| +| `CheckpointLoaderSimple` | `ckpt_name` | `checkpoints` | +| `LoraLoader` | `lora_name`, `strength_model`, `strength_clip` | `loras` | +| `LoraLoaderModelOnly` | `lora_name`, `strength_model` | `loras` | +| `VAELoader` | `vae_name` | `vae` | +| `ControlNetLoader` | `control_net_name` | `controlnet` | +| `CLIPLoader` | `clip_name` | `clip` | +| `DualCLIPLoader` | `clip_name1`, `clip_name2` | `clip` | +| `TripleCLIPLoader` | `clip_name1/2/3` | `clip` | +| `UNETLoader` | `unet_name` | `unet` | +| `DiffusionModelLoader` | `model_name` | `diffusion_models` | +| `UpscaleModelLoader` | `model_name` | `upscale_models` | +| `IPAdapterModelLoader` | `ipadapter_file` | `ipadapter` | +| `ADE_AnimateDiffLoaderWithContext` | `model_name`, `motion_scale` | `animatediff_models` | + +### Image Input/Output + +| Node Class | Key Fields | +|------------|------------| +| `LoadImage` | `image` (server-side filename, after upload) | +| `LoadImageMask` | `image`, `channel` (`red` / `green` / `blue` / `alpha`) | +| `VAEEncode` / `VAEDecode` | (no controllable fields) | +| `VAEEncodeForInpaint` | `grow_mask_by` | +| `SaveImage` | `filename_prefix` | +| `VHS_VideoCombine` | `frame_rate`, `format`, `filename_prefix`, `loop_count`, `pingpong` | + +### ControlNet + +| Node Class | Key Fields | +|------------|------------| +| `ControlNetApply` | `strength` | +| `ControlNetApplyAdvanced` | `strength`, `start_percent`, `end_percent` | + +### IPAdapter (community pack `comfyui_ipadapter_plus`) + +| Node Class | Key Fields | +|------------|------------| +| `IPAdapterAdvanced` | `weight`, `start_at`, `end_at` | +| `IPAdapter` | `weight` | + +### Embeddings (referenced inside prompt strings) + +ComfyUI scans prompt text for `embedding:NAME` syntax. The skill's +`_common.iter_embedding_refs()` extracts these as model dependencies. + +```text +"a beautiful cat, embedding:goodvibes:1.2, embedding:art-style" +``` + +`extract_schema.py` and `check_deps.py` surface these in +`embedding_dependencies` / `missing_embeddings`. + +## Parameter Injection Pattern + +```python +import json, copy + +with open("workflow_api.json") as f: + workflow = json.load(f) + +wf = copy.deepcopy(workflow) +wf["6"]["inputs"]["text"] = "a beautiful sunset" +wf["7"]["inputs"]["text"] = "ugly, blurry" +wf["3"]["inputs"]["seed"] = 42 +wf["3"]["inputs"]["steps"] = 30 +wf["5"]["inputs"]["width"] = 1024 +wf["5"]["inputs"]["height"] = 1024 +``` + +`scripts/extract_schema.py` automates discovering which node IDs/fields +correspond to which user-facing parameters. It returns a `parameters` dict +that `run_workflow.py` reads to inject values from `--args`. + +## Identifying Controllable Parameters (Heuristics) + +For unknown workflows: + +1. **Prompt text** — any `CLIPTextEncode.text`. Use connection tracing back + from `KSampler.positive` / `.negative` to disambiguate (don't trust + meta-title alone). +2. **Seed** — `KSampler.seed` / `KSamplerAdvanced.noise_seed` / `RandomNoise.noise_seed`. +3. **Dimensions** — `Empty*LatentImage.width/height` (must be multiples of 8). +4. **Steps / CFG** — `KSampler.steps`, `KSampler.cfg`. Steps 20–50 typical. + CFG 5–15 typical (Flux uses guidance, not CFG). +5. **Model / checkpoint** — `CheckpointLoaderSimple.ckpt_name`. Filename must + match an installed file *exactly*. +6. **LoRA** — `LoraLoader.lora_name`, `.strength_model`. +7. **Images for img2img / inpaint** — `LoadImage.image`. Server-side filename + after upload. +8. **Denoise** — `KSampler.denoise`. 0.0–1.0; 1.0 = ignore input image, + 0.0 = pass through. Sweet spot for img2img: 0.4–0.7. + +## Output Nodes + +Output is produced by these node types. The skill's `OUTPUT_NODES` set +extends to common community packs. + +| Node | Output Key | Content | +|------|-----------|---------| +| `SaveImage` | `images` | List of `{filename, subfolder, type}` | +| `PreviewImage` | `images` | Temporary preview (not saved) | +| `VHS_VideoCombine` | `gifs` (older) or `videos`/`video` (newer cloud) | Video file refs | +| `SaveAudio` | `audio` | Audio file refs | +| `SaveAnimatedWEBP` / `SaveAnimatedPNG` | `images` | Animated images | +| `Save3D` | `3d` | 3D asset refs | + +After execution, fetch outputs from `/history/{prompt_id}` (local) or +`/api/jobs/{prompt_id}` (cloud) → `outputs` → `{node_id}` → `{key}`. + +## Wrapper Variants + +Some saved JSON files wrap the workflow under a `"prompt"` key (matching +the `/api/prompt` payload shape). The skill's `_common.unwrap_workflow()` +handles this — pass any of: + +- raw API format: `{"3": {...}, "4": {...}}` +- wrapped: `{"prompt": {"3": {...}}, "client_id": "..."}` + +It rejects editor format with a clear error and a re-export instruction. diff --git a/image-creation/comfyui/scripts/_common.py b/image-creation/comfyui/scripts/_common.py new file mode 100644 index 0000000..efe592a --- /dev/null +++ b/image-creation/comfyui/scripts/_common.py @@ -0,0 +1,835 @@ +""" +_common.py — Shared logic for ComfyUI skill scripts. + +Single source of truth for: +- HTTP transport (with retry/backoff, streaming, timeout handling) +- Cloud detection and endpoint mapping (local ComfyUI vs Comfy Cloud) +- Workflow node-type catalogs (param patterns, model loaders, output nodes) +- API-format validation +- Path-traversal-safe file writes +- API-key loading from env / CLI + +Stdlib-only by design (with optional `requests` upgrade if installed). Python 3.10+. +""" + +from __future__ import annotations + +import json +import os +import random +import re +import sys +import time +import uuid +from dataclasses import dataclass +from pathlib import Path +from typing import Any, Iterator +from urllib.parse import urlparse + +# Optional: prefer `requests` if installed (better redirects, streaming, header handling) +try: + import requests # type: ignore[import-not-found] + HAS_REQUESTS = True +except ImportError: # pragma: no cover - exercised via stdlib fallback + HAS_REQUESTS = False + import urllib.error + import urllib.request + + +# ============================================================================= +# Constants & catalogs +# ============================================================================= + +DEFAULT_LOCAL_HOST = "http://127.0.0.1:8188" +DEFAULT_CLOUD_HOST = "https://cloud.comfy.org" +ENV_API_KEY = "COMFY_CLOUD_API_KEY" + +# Connection / retry defaults +DEFAULT_HTTP_TIMEOUT = 60 # seconds — single-attempt request timeout +DEFAULT_RETRIES = 3 # total attempts including the first +RETRY_BASE_DELAY = 1.0 # seconds — exponential backoff base +RETRY_MAX_DELAY = 30.0 # seconds — cap on backoff +RETRY_STATUS_CODES = {408, 429, 500, 502, 503, 504, 522, 524} + +# Streaming download chunk size (bytes) +DOWNLOAD_CHUNK_SIZE = 1 << 16 # 64 KiB + +# Heuristic: workflows with these node types tend to be slow → larger default timeout +SLOW_OUTPUT_NODES = { + "VHS_VideoCombine", "SaveAnimatedWEBP", "SaveAnimatedPNG", + "SaveVideo", "SaveAudio", "SaveAnimateDiffVideo", + "SVD_img2vid_Conditioning", + "WanVideoSampler", "HunyuanVideoSampler", + "CogVideoSampler", "LTXVideoSampler", +} + +# --------------------------------------------------------------------------- +# Output node catalog (extensible — community packs add their own) +# --------------------------------------------------------------------------- +OUTPUT_NODES: set[str] = { + # Built-in + "SaveImage", "PreviewImage", + "SaveAudio", "SaveVideo", "PreviewAudio", "PreviewVideo", + "SaveAnimatedWEBP", "SaveAnimatedPNG", + # Common community packs + "VHS_VideoCombine", # Video Helper Suite + "ImageSave", # Was Node Suite + "Image Save", # Was Node Suite (alt name) + "easy imageSave", # easy-use + "Image Save With Metadata", + "PreviewImage|pysssss", # pysssss preview + "ShowText|pysssss", + "SaveLatent", + "SaveGLB", # 3D + "Save3D", +} + +# --------------------------------------------------------------------------- +# Folder aliases — handle ComfyUI's gradual folder renames +# --------------------------------------------------------------------------- +# When `check_deps.py` queries `/models/` and gets 404 / empty, +# it tries each alias in turn. Critical for Comfy Cloud which has fully +# migrated to the new naming (unet → diffusion_models, clip → text_encoders). +FOLDER_ALIASES: dict[str, list[str]] = { + "unet": ["unet", "diffusion_models"], + "diffusion_models": ["diffusion_models", "unet"], + "clip": ["clip", "text_encoders"], + "text_encoders": ["text_encoders", "clip"], + "controlnet": ["controlnet", "control_net"], +} + + +def folder_aliases_for(folder: str) -> list[str]: + """Return the search order of folder names (primary first).""" + return FOLDER_ALIASES.get(folder, [folder]) + + +# --------------------------------------------------------------------------- +# Model-loader catalog: class_type -> (input field, model folder) +# --------------------------------------------------------------------------- +# A loader can have multiple fields (e.g., DualCLIPLoader has clip_name1 and +# clip_name2). We list them with explicit entries. The folder name is the +# *canonical* one; FOLDER_ALIASES is consulted when querying. +MODEL_LOADERS: dict[str, list[tuple[str, str]]] = { + # Checkpoints + "CheckpointLoaderSimple": [("ckpt_name", "checkpoints")], + "CheckpointLoader": [("ckpt_name", "checkpoints")], + "CheckpointLoader (Simple)": [("ckpt_name", "checkpoints")], + "ImageOnlyCheckpointLoader": [("ckpt_name", "checkpoints")], + "unCLIPCheckpointLoader": [("ckpt_name", "checkpoints")], + # LoRA + "LoraLoader": [("lora_name", "loras")], + "LoraLoaderModelOnly": [("lora_name", "loras")], + "LoraLoaderTagsQuery": [("lora_name", "loras")], + # VAE + "VAELoader": [("vae_name", "vae")], + # ControlNet + "ControlNetLoader": [("control_net_name", "controlnet")], + "DiffControlNetLoader": [("control_net_name", "controlnet")], + "ControlNetLoaderAdvanced": [("control_net_name", "controlnet")], + # CLIP / text encoders (primary "clip" folder; check_deps tries text_encoders too) + "CLIPLoader": [("clip_name", "clip")], + "DualCLIPLoader": [("clip_name1", "clip"), ("clip_name2", "clip")], + "TripleCLIPLoader": [("clip_name1", "clip"), ("clip_name2", "clip"), ("clip_name3", "clip")], + "CLIPVisionLoader": [("clip_name", "clip_vision")], + # UNET / Diffusion model (primary "unet"; check_deps tries diffusion_models too) + "UNETLoader": [("unet_name", "unet")], + "DiffusionModelLoader": [("model_name", "diffusion_models")], + "UNETLoaderGGUF": [("unet_name", "unet")], + # Upscaler + "UpscaleModelLoader": [("model_name", "upscale_models")], + # Style / GLIGEN / Hypernetwork + "StyleModelLoader": [("style_model_name", "style_models")], + "GLIGENLoader": [("gligen_name", "gligen")], + "HypernetworkLoader": [("hypernetwork_name", "hypernetworks")], + # IPAdapter family (community). + # Note: IPAdapterUnifiedLoader's `preset` and IPAdapterInsightFaceLoader's + # `provider` are enums (not file paths), so they're intentionally omitted — + # check_deps would otherwise treat enum values as missing model files. + "IPAdapterModelLoader": [("ipadapter_file", "ipadapter")], + "InstantIDModelLoader": [("instantid_file", "instantid")], + # AnimateDiff / video + "ADE_LoadAnimateDiffModel": [("model_name", "animatediff_models")], + "ADE_AnimateDiffLoaderWithContext": [("model_name", "animatediff_models")], + "ADE_AnimateDiffLoaderGen1": [("model_name", "animatediff_models")], + # Photomaker + "PhotoMakerLoader": [("photomaker_model_name", "photomaker")], + # Sampler / scheduler models + "ModelSamplingFlux": [], # parametric only +} + +# --------------------------------------------------------------------------- +# Param patterns: (class_type, field_name) -> friendly_name +# Order matters — first match wins for naming. Use _meta.title for disambiguation. +# --------------------------------------------------------------------------- +PARAM_PATTERNS: list[tuple[str, str, str]] = [ + # ---- Prompts ---- + ("CLIPTextEncode", "text", "prompt"), + ("CLIPTextEncodeSDXL", "text_g", "prompt"), + ("CLIPTextEncodeSDXL", "text_l", "prompt_l"), + ("CLIPTextEncodeSDXLRefiner", "text", "refiner_prompt"), + ("CLIPTextEncodeFlux", "clip_l", "prompt_l"), + ("CLIPTextEncodeFlux", "t5xxl", "prompt"), + ("CLIPTextEncodeFlux", "guidance", "guidance"), + ("smZ CLIPTextEncode", "text", "prompt"), + ("BNK_CLIPTextEncodeAdvanced", "text", "prompt"), + + # ---- Standard sampling ---- + ("KSampler", "seed", "seed"), + ("KSampler", "steps", "steps"), + ("KSampler", "cfg", "cfg"), + ("KSampler", "sampler_name", "sampler_name"), + ("KSampler", "scheduler", "scheduler"), + ("KSampler", "denoise", "denoise"), + ("KSamplerAdvanced", "noise_seed", "seed"), + ("KSamplerAdvanced", "steps", "steps"), + ("KSamplerAdvanced", "cfg", "cfg"), + ("KSamplerAdvanced", "sampler_name", "sampler_name"), + ("KSamplerAdvanced", "scheduler", "scheduler"), + ("KSamplerAdvanced", "start_at_step", "start_at_step"), + ("KSamplerAdvanced", "end_at_step", "end_at_step"), + + # ---- Modern sampler chain (Flux / SD3 / SDXL refiner via SamplerCustom) ---- + ("RandomNoise", "noise_seed", "seed"), + ("BasicScheduler", "steps", "steps"), + ("BasicScheduler", "scheduler", "scheduler"), + ("BasicScheduler", "denoise", "denoise"), + ("KSamplerSelect", "sampler_name", "sampler_name"), + # NB: BasicGuider has no cfg input (it just bundles model+conditioning). + ("CFGGuider", "cfg", "cfg"), + ("DualCFGGuider", "cfg_conds", "cfg"), + ("DualCFGGuider", "cfg_cond2_negative", "cfg_negative"), + ("ModelSamplingFlux", "max_shift", "max_shift"), + ("ModelSamplingFlux", "base_shift", "base_shift"), + ("ModelSamplingFlux", "width", "model_width"), + ("ModelSamplingFlux", "height", "model_height"), + ("ModelSamplingSD3", "shift", "shift"), + ("ModelSamplingDiscrete", "sampling", "sampling"), + ("SDTurboScheduler", "steps", "steps"), + ("SDTurboScheduler", "denoise", "denoise"), + ("SamplerCustom", "noise_seed", "seed"), + ("SamplerCustom", "cfg", "cfg"), + # NB: SamplerCustomAdvanced takes a NOISE input (from RandomNoise) — no seed field directly. + + # ---- Dimensions / latent ---- + ("EmptyLatentImage", "width", "width"), + ("EmptyLatentImage", "height", "height"), + ("EmptyLatentImage", "batch_size", "batch_size"), + ("EmptySD3LatentImage", "width", "width"), + ("EmptySD3LatentImage", "height", "height"), + ("EmptySD3LatentImage", "batch_size", "batch_size"), + ("EmptyHunyuanLatentVideo", "width", "width"), + ("EmptyHunyuanLatentVideo", "height", "height"), + ("EmptyHunyuanLatentVideo", "length", "length"), + ("EmptyHunyuanLatentVideo", "batch_size", "batch_size"), + ("EmptyMochiLatentVideo", "width", "width"), + ("EmptyMochiLatentVideo", "height", "height"), + ("EmptyMochiLatentVideo", "length", "length"), + ("EmptyLTXVLatentVideo", "width", "width"), + ("EmptyLTXVLatentVideo", "height", "height"), + ("EmptyLTXVLatentVideo", "length", "length"), + ("LatentUpscale", "width", "upscale_width"), + ("LatentUpscale", "height", "upscale_height"), + ("LatentUpscaleBy", "scale_by", "scale_by"), + ("ImageScale", "width", "width"), + ("ImageScale", "height", "height"), + + # ---- Image input ---- + ("LoadImage", "image", "image"), + ("LoadImageMask", "image", "mask_image"), + ("LoadImageOutput", "image", "image"), + ("VHS_LoadVideo", "video", "video"), + ("VHS_LoadAudio", "audio", "audio"), + + # ---- Model selection (sometimes useful to swap per run) ---- + ("CheckpointLoaderSimple", "ckpt_name", "ckpt_name"), + ("CheckpointLoader", "ckpt_name", "ckpt_name"), + ("ImageOnlyCheckpointLoader", "ckpt_name", "ckpt_name"), + ("VAELoader", "vae_name", "vae_name"), + ("UNETLoader", "unet_name", "unet_name"), + ("DiffusionModelLoader", "model_name", "diffusion_model_name"), + ("UpscaleModelLoader", "model_name", "upscale_model_name"), + ("CLIPLoader", "clip_name", "clip_name"), + ("DualCLIPLoader", "clip_name1", "clip_name1"), + ("DualCLIPLoader", "clip_name2", "clip_name2"), + ("ControlNetLoader", "control_net_name", "controlnet_name"), + + # ---- LoRA ---- + ("LoraLoader", "lora_name", "lora_name"), + ("LoraLoader", "strength_model", "lora_strength"), + ("LoraLoader", "strength_clip", "lora_strength_clip"), + ("LoraLoaderModelOnly", "lora_name", "lora_name"), + ("LoraLoaderModelOnly", "strength_model", "lora_strength"), + + # ---- ControlNet ---- + ("ControlNetApply", "strength", "controlnet_strength"), + ("ControlNetApplyAdvanced", "strength", "controlnet_strength"), + ("ControlNetApplyAdvanced", "start_percent", "controlnet_start"), + ("ControlNetApplyAdvanced", "end_percent", "controlnet_end"), + + # ---- IPAdapter ---- + ("IPAdapterAdvanced", "weight", "ipadapter_weight"), + ("IPAdapterAdvanced", "start_at", "ipadapter_start"), + ("IPAdapterAdvanced", "end_at", "ipadapter_end"), + ("IPAdapter", "weight", "ipadapter_weight"), + + # ---- Upscale ---- + ("ImageUpscaleWithModel", "upscale_method", "upscale_method"), + + # ---- AnimateDiff ---- + ("ADE_AnimateDiffLoaderWithContext", "motion_scale", "motion_scale"), + ("ADE_AnimateDiffLoaderGen1", "motion_scale", "motion_scale"), + + # ---- Video / Save ---- + ("VHS_VideoCombine", "frame_rate", "frame_rate"), + ("VHS_VideoCombine", "format", "video_format"), + ("VHS_VideoCombine", "filename_prefix", "filename_prefix"), + ("SaveImage", "filename_prefix", "filename_prefix"), + + # ---- Hunyuan / Wan / LTX video ---- + ("HunyuanVideoSampler", "seed", "seed"), + ("HunyuanVideoSampler", "steps", "steps"), + ("HunyuanVideoSampler", "cfg", "cfg"), + ("WanVideoSampler", "seed", "seed"), + ("WanVideoSampler", "steps", "steps"), + ("WanVideoSampler", "cfg", "cfg"), + ("LTXVScheduler", "max_shift", "max_shift"), + ("LTXVScheduler", "base_shift", "base_shift"), + + # ---- rgthree primitives (often used as user-facing inputs) ---- + ("Seed (rgthree)", "seed", "seed"), + ("Image Comparer (rgthree)", "image_a", "image"), + ("Power Lora Loader (rgthree)", "PowerLoraLoaderHeaderWidget", "_lora_header"), + + # ---- Easy-use / utility primitives ---- + ("PrimitiveNode", "value", "primitive_value"), + ("easy seed", "seed", "seed"), + ("easy positive", "positive", "prompt"), + ("easy negative", "negative", "negative_prompt"), + ("easy fullLoader", "ckpt_name", "ckpt_name"), + ("easy fullLoader", "vae_name", "vae_name"), + ("easy fullLoader", "lora_name", "lora_name"), + ("easy fullLoader", "positive", "prompt"), + ("easy fullLoader", "negative", "negative_prompt"), +] + +# Prompt-like fields whose value should be scanned for embedding references +PROMPT_FIELDS = {"text", "text_g", "text_l", "t5xxl", "clip_l", "positive", "negative"} + +# Pattern matches: embedding:name, embedding:name.pt, embedding:name:1.2, (embedding:name:1.2) +# Word-boundary at start avoids matching things like "no_embedding:foo". +EMBEDDING_REGEX = re.compile( + r"(?:^|[\s,(\[])embedding\s*:\s*([A-Za-z0-9_\-\./\\]+?)(?:\.(?:pt|safetensors|bin))?(?=[\s:,)\(\]]|$)", + re.IGNORECASE, +) + + +# ============================================================================= +# Cloud detection & endpoint routing +# ============================================================================= + +CLOUD_DOMAIN_SUFFIXES = (".comfy.org",) +CLOUD_DOMAIN_EXACT = {"cloud.comfy.org"} + + +def is_cloud_host(host: str) -> bool: + """True if the host points at Comfy Cloud (or staging/preview subdomain).""" + parsed = urlparse(host if "://" in host else f"http://{host}") + hostname = (parsed.hostname or "").lower() + if hostname in CLOUD_DOMAIN_EXACT: + return True + return any(hostname.endswith(s) for s in CLOUD_DOMAIN_SUFFIXES) + + +def build_cloud_aware_url(base: str, path: str, *, force_cloud: bool | None = None) -> str: + """Build a URL that adds /api prefix when targeting Comfy Cloud. + + Local ComfyUI accepts both `/foo` and `/api/foo` for many endpoints. + Cloud requires `/api/foo`. + + `path` should be a path component (e.g. "/prompt") or full path with query + (e.g. "/view?filename=x"). + """ + base = base.rstrip("/") + cloud = is_cloud_host(base) if force_cloud is None else force_cloud + if not path.startswith("/"): + path = "/" + path + if cloud and not path.startswith("/api/"): + path = "/api" + path + return base + path + + +def cloud_endpoint(path: str) -> str: + """Map a cloud endpoint path to its current canonical form. + + Handles known renames documented in the Comfy Cloud API: + /history -> /history_v2 + /models/ -> /experiment/models/ + /models -> /experiment/models + """ + if path.startswith("/history") and not path.startswith("/history_v2"): + return "/history_v2" + path[len("/history"):] + if path.startswith("/models/"): + return "/experiment/models/" + path[len("/models/"):] + if path == "/models": + return "/experiment/models" + return path + + +def resolve_url(base: str, path: str, *, is_cloud: bool | None = None) -> str: + """Top-level URL resolver. Applies cloud rename + /api prefix as needed.""" + cloud = is_cloud_host(base) if is_cloud is None else is_cloud + if cloud: + path = cloud_endpoint(path) + return build_cloud_aware_url(base, path, force_cloud=cloud) + + +# ============================================================================= +# API key resolution +# ============================================================================= + +def resolve_api_key(explicit: str | None) -> str | None: + """Look up API key from CLI flag → env var. Strips whitespace and quotes.""" + val = explicit if explicit else os.environ.get(ENV_API_KEY) + if val is None: + return None + val = val.strip().strip("'\"") + return val or None + + +# ============================================================================= +# HTTP transport +# ============================================================================= + +@dataclass +class HTTPResponse: + status: int + headers: dict[str, str] + body: bytes + url: str # final URL after redirects + + def text(self, encoding: str = "utf-8") -> str: + return self.body.decode(encoding, errors="replace") + + def json(self) -> Any: + return json.loads(self.body.decode("utf-8", errors="replace")) + + +def _sleep_backoff(attempt: int, base: float = RETRY_BASE_DELAY, cap: float = RETRY_MAX_DELAY) -> None: + """Sleep with full-jitter exponential backoff.""" + delay = min(cap, base * (2 ** attempt)) + delay = random.uniform(0, delay) + time.sleep(delay) + + +def http_request( + method: str, + url: str, + *, + headers: dict[str, str] | None = None, + json_body: Any = None, + data: bytes | None = None, + files: dict | None = None, + form: dict | None = None, + timeout: float = DEFAULT_HTTP_TIMEOUT, + follow_redirects: bool = True, + retries: int = DEFAULT_RETRIES, + stream: bool = False, + sink: Path | None = None, +) -> HTTPResponse: + """Single entry point for all HTTP traffic. + + Behavior: + - Retries on connection errors and on HTTP statuses in RETRY_STATUS_CODES, + with exponential backoff + jitter. + - For cross-host redirects, drops Authorization-style headers (so signed + URLs don't leak the API key to S3/CloudFront). + - When `stream=True` and `sink` is a Path, streams the response body to + disk in 64 KiB chunks instead of buffering. + + Either `json_body`, `data`, or `files`+`form` may be supplied (mutually exclusive). + """ + if headers is None: + headers = {} + headers = dict(headers) # copy + headers.setdefault("User-Agent", "hermes-comfyui-skill/5.0") + + if files or form is not None: + # Multipart upload — needs `requests`. The stdlib fallback lacks + # multipart encoding helpers; raise a clear error. + if not HAS_REQUESTS: + raise RuntimeError( + "Multipart upload requires the `requests` package. " + "Install with: pip install requests" + ) + + last_exc: Exception | None = None + for attempt in range(retries): + try: + resp = _http_once( + method=method, url=url, headers=headers, + json_body=json_body, data=data, files=files, form=form, + timeout=timeout, follow_redirects=follow_redirects, + stream=stream, sink=sink, + ) + if resp.status in RETRY_STATUS_CODES and attempt + 1 < retries: + _sleep_backoff(attempt) + continue + return resp + except (TimeoutError, ConnectionError, OSError) as e: + last_exc = e + if attempt + 1 < retries: + _sleep_backoff(attempt) + continue + raise + + # Should not reach here unless retries was 0 + if last_exc: + raise last_exc + raise RuntimeError("http_request: retries exhausted with no response") + + +_SENSITIVE_HEADERS = ("x-api-key", "authorization", "cookie") + + +if HAS_REQUESTS: + class _StripSensitiveOnRedirectSession(requests.Session): + """Session that drops sensitive headers on cross-host redirects. + + `requests` already strips `Authorization` cross-host (rebuild_auth), + but it does NOT strip custom headers like `X-API-Key`. We override + `rebuild_auth` to additionally strip every header in + `_SENSITIVE_HEADERS` when the destination is a different host — + critical when ComfyUI Cloud's `/api/view` redirects to a signed S3 URL. + """ + + def rebuild_auth(self, prepared_request, response): # type: ignore[override] + super().rebuild_auth(prepared_request, response) + try: + old_url = response.request.url + new_url = prepared_request.url + old_host = (urlparse(old_url).hostname or "").lower() + new_host = (urlparse(new_url).hostname or "").lower() + if old_host and new_host and old_host != new_host: + headers = prepared_request.headers + for key in list(headers.keys()): + if key.lower() in _SENSITIVE_HEADERS: + del headers[key] + except Exception: + # Defensive: never let header stripping break a redirect. + pass + + +def _http_once( + *, method: str, url: str, headers: dict[str, str], + json_body: Any, data: bytes | None, files: dict | None, form: dict | None, + timeout: float, follow_redirects: bool, + stream: bool, sink: Path | None, +) -> HTTPResponse: + """One HTTP attempt. No retry.""" + if HAS_REQUESTS: + kwargs: dict[str, Any] = { + "method": method, "url": url, "headers": headers, + "timeout": timeout, "allow_redirects": follow_redirects, + } + if json_body is not None: + kwargs["json"] = json_body + elif data is not None: + kwargs["data"] = data + elif files is not None or form is not None: + kwargs["files"] = files + kwargs["data"] = form + if stream: + kwargs["stream"] = True + + # Use the subclass that strips sensitive headers cross-host + with _StripSensitiveOnRedirectSession() as s: + try: + r = s.request(**kwargs) + if stream and sink is not None: + sink.parent.mkdir(parents=True, exist_ok=True) + with sink.open("wb") as f: + for chunk in r.iter_content(DOWNLOAD_CHUNK_SIZE): + if chunk: + f.write(chunk) + body = b"" # already drained + else: + body = r.content + return HTTPResponse( + status=r.status_code, + headers={k: v for k, v in r.headers.items()}, + body=body, + url=r.url, + ) + except requests.exceptions.RequestException as e: + # Convert to TimeoutError / ConnectionError so the retry loop + # picks them up uniformly with the stdlib path. + if isinstance(e, requests.exceptions.Timeout): + raise TimeoutError(str(e)) from e + raise ConnectionError(str(e)) from e + + # ---------- stdlib fallback ---------- + if json_body is not None: + body_bytes = json.dumps(json_body).encode("utf-8") + headers.setdefault("Content-Type", "application/json") + else: + body_bytes = data + req = urllib.request.Request(url, data=body_bytes, headers=headers, method=method) + + # urllib follows redirects by default. We need to: + # 1) intercept cross-host redirects and drop X-API-Key + # 2) optionally NOT follow redirects when follow_redirects=False + class _RedirectHandler(urllib.request.HTTPRedirectHandler): + def __init__(self, original_host: str, follow: bool): + self.original_host = original_host + self.follow = follow + + def redirect_request(self, req2, fp, code, msg, hdrs, newurl): + if not self.follow: + return None + new_host = (urlparse(newurl).hostname or "").lower() + if new_host != self.original_host: + # Build a new request with cleaned headers + clean_headers = { + k: v for k, v in req2.header_items() + if k.lower() not in {"x-api-key", "authorization", "cookie"} + } + new_req = urllib.request.Request(newurl, headers=clean_headers, method="GET") + return new_req + return super().redirect_request(req2, fp, code, msg, hdrs, newurl) + + original_host = (urlparse(url).hostname or "").lower() + opener = urllib.request.build_opener(_RedirectHandler(original_host, follow_redirects)) + + try: + resp = opener.open(req, timeout=timeout) + except urllib.error.HTTPError as e: + return HTTPResponse( + status=e.code, + headers=dict(e.headers) if e.headers else {}, + body=e.read() or b"", + url=getattr(e, "url", url), + ) + + final_url = resp.geturl() + final_status = resp.status + final_headers = dict(resp.headers) + + if stream and sink is not None: + sink.parent.mkdir(parents=True, exist_ok=True) + with sink.open("wb") as f: + while True: + chunk = resp.read(DOWNLOAD_CHUNK_SIZE) + if not chunk: + break + f.write(chunk) + return HTTPResponse(status=final_status, headers=final_headers, body=b"", url=final_url) + + return HTTPResponse(status=final_status, headers=final_headers, body=resp.read(), url=final_url) + + +def http_get(url: str, **kwargs: Any) -> HTTPResponse: + return http_request("GET", url, **kwargs) + + +def http_post(url: str, **kwargs: Any) -> HTTPResponse: + return http_request("POST", url, **kwargs) + + +# ============================================================================= +# Workflow validation & helpers +# ============================================================================= + +def is_api_format(workflow: Any) -> bool: + """API format = top-level dict where each value has `class_type`.""" + if not isinstance(workflow, dict): + return False + if "nodes" in workflow and "links" in workflow: + return False + for v in workflow.values(): + if isinstance(v, dict) and "class_type" in v: + return True + return False + + +def unwrap_workflow(payload: Any) -> dict: + """Unwrap common wrapper variants. Returns API-format workflow or raises ValueError.""" + if isinstance(payload, dict) and is_api_format(payload): + return payload + # Some files wrap workflow under "prompt" key (e.g. saved /prompt payloads) + if isinstance(payload, dict) and "prompt" in payload and is_api_format(payload["prompt"]): + return payload["prompt"] + # Editor format + if isinstance(payload, dict) and "nodes" in payload and "links" in payload: + raise ValueError( + "Workflow is in editor format (has top-level 'nodes' and 'links' arrays). " + "Re-export from ComfyUI using 'Workflow → Export (API)' (newer UI) " + "or 'Save (API Format)' (older UI)." + ) + raise ValueError( + "Workflow is not in API format. Each top-level entry must have a 'class_type' field." + ) + + +def is_link(value: Any) -> bool: + """True if `value` is a [node_id, output_index] connection (length-2 list).""" + return ( + isinstance(value, list) + and len(value) == 2 + and isinstance(value[0], str) + and isinstance(value[1], int) + ) + + +def iter_nodes(workflow: dict) -> Iterator[tuple[str, dict]]: + """Yield (node_id, node) for each valid API-format node.""" + for node_id, node in workflow.items(): + if isinstance(node, dict) and "class_type" in node: + yield node_id, node + + +def iter_model_deps(workflow: dict) -> Iterator[dict]: + """Yield {node_id, class_type, field, value, folder} for each model dependency.""" + for node_id, node in iter_nodes(workflow): + cls = node["class_type"] + if cls not in MODEL_LOADERS: + continue + inputs = node.get("inputs", {}) or {} + for field_name, folder in MODEL_LOADERS[cls]: + val = inputs.get(field_name) + if val and isinstance(val, str) and not is_link(val): + yield { + "node_id": node_id, + "class_type": cls, + "field": field_name, + "value": val, + "folder": folder, + } + + +def iter_embedding_refs(workflow: dict) -> Iterator[tuple[str, str]]: + """Yield (node_id, embedding_name) for every embedding mention in prompts.""" + for node_id, node in iter_nodes(workflow): + inputs = node.get("inputs", {}) or {} + for field_name, val in inputs.items(): + if field_name not in PROMPT_FIELDS: + continue + if not isinstance(val, str): + continue + for m in EMBEDDING_REGEX.finditer(val): + yield node_id, m.group(1) + + +# ============================================================================= +# Path safety +# ============================================================================= + +def safe_path_join(base: Path, *parts: str) -> Path: + """Join paths, raising if the result escapes `base`. + + Server-supplied filenames may contain `../` etc. This guards against + path-traversal attacks when downloading outputs. + """ + base_resolved = base.resolve() + candidate = base.joinpath(*parts).resolve() + try: + candidate.relative_to(base_resolved) + except ValueError as e: + raise ValueError( + f"Refusing path traversal: {candidate} is outside {base_resolved}" + ) from e + return candidate + + +def media_type_from_filename(filename: str) -> str: + ext = Path(filename).suffix.lower() + if ext in {".mp4", ".webm", ".avi", ".mov", ".mkv", ".gif", ".webp"}: + return "video" + if ext in {".wav", ".mp3", ".flac", ".ogg", ".m4a"}: + return "audio" + if ext in {".glb", ".obj", ".ply", ".gltf"}: + return "3d" + if ext in {".json", ".txt", ".md"}: + return "text" + return "image" + + +def looks_like_video_workflow(workflow: dict) -> bool: + """Used to bump default timeout for video workflows.""" + for _, node in iter_nodes(workflow): + if node["class_type"] in SLOW_OUTPUT_NODES: + return True + if node["class_type"].lower().startswith(("animatediff", "ade_", "wanvideo", "hunyuanvideo", "ltxvideo", "cogvideo")): + return True + return False + + +# ============================================================================= +# Seed handling +# ============================================================================= + +# ComfyUI's max seed range. Many UIs treat `-1` as "randomize on submit". +SEED_MAX = 2**63 - 1 +SEED_MIN = 0 + + +def coerce_seed(value: Any) -> int: + """Convert -1 or None to a fresh random seed; otherwise return int(value). + + Accepts numeric -1 OR string "-1" (both treated as "randomize"). Other + parse failures raise TypeError/ValueError for the caller to surface. + """ + if value is None: + return random.randint(SEED_MIN, SEED_MAX) + # Stringly-typed -1 from CLI / JSON should also randomize + if isinstance(value, str) and value.strip() == "-1": + return random.randint(SEED_MIN, SEED_MAX) + if value == -1: + return random.randint(SEED_MIN, SEED_MAX) + return int(value) + + +# ============================================================================= +# Cloud model-list normalization +# ============================================================================= + +def parse_model_list(payload: Any) -> set[str]: + """Normalize model-list responses from local ComfyUI vs Comfy Cloud. + + Local: `["a.safetensors", "b.safetensors"]` + Cloud: `[{"name": "a.safetensors", "pathIndex": 0}, ...]` + """ + if not isinstance(payload, list): + return set() + out: set[str] = set() + for item in payload: + if isinstance(item, str): + out.add(item) + elif isinstance(item, dict): + name = item.get("name") or item.get("filename") or item.get("path") + if isinstance(name, str): + out.add(name) + return out + + +# ============================================================================= +# Misc utilities +# ============================================================================= + +def new_client_id() -> str: + return str(uuid.uuid4()) + + +def fmt_kv(d: dict) -> str: + """Pretty key=value for log lines.""" + return " ".join(f"{k}={v!r}" for k, v in d.items()) + + +def emit_json(obj: Any, *, indent: int = 2) -> None: + """Print JSON to stdout. Centralised so behavior can be tweaked (e.g., --raw).""" + print(json.dumps(obj, indent=indent, default=str)) + + +def log(msg: str) -> None: + """stderr log with consistent prefix (so JSON stdout stays clean).""" + print(f"[comfyui-skill] {msg}", file=sys.stderr) diff --git a/image-creation/comfyui/scripts/auto_fix_deps.py b/image-creation/comfyui/scripts/auto_fix_deps.py new file mode 100644 index 0000000..788bf8e --- /dev/null +++ b/image-creation/comfyui/scripts/auto_fix_deps.py @@ -0,0 +1,225 @@ +#!/usr/bin/env python3 +""" +auto_fix_deps.py — Run check_deps.py, then attempt to install whatever is missing. + +For local servers: + - Missing custom nodes → `comfy node install ` + - Missing models → `comfy model download` (only if a URL is supplied via + --model-source-file or detected via well-known names) + +For cloud: prints what would be needed but cannot install (cloud preinstalls +custom nodes and most models server-side; if something genuinely isn't there, +ask Comfy support). + +This is conservative: it never installs without an explicit URL for models +(downloading the wrong model is hard to undo). Custom nodes from the registry +are auto-installed by name. + +Usage: + python3 auto_fix_deps.py workflow_api.json + python3 auto_fix_deps.py workflow_api.json --models-from-file urls.json + python3 auto_fix_deps.py workflow_api.json --dry-run +""" + +from __future__ import annotations + +import argparse +import json +import shutil +import subprocess +import sys +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from _common import ( # noqa: E402 + DEFAULT_LOCAL_HOST, ENV_API_KEY, emit_json, log, resolve_api_key, +) +from check_deps import check_deps # noqa: E402 +from _common import unwrap_workflow # noqa: E402 + + +def comfy_cli_available() -> str | None: + """Return command prefix for comfy-cli, or None.""" + if shutil.which("comfy"): + return "comfy" + if shutil.which("uvx"): + return "uvx --from comfy-cli comfy" + return None + + +def run_cmd(cmd: list[str], *, dry_run: bool = False) -> tuple[int, str]: + if dry_run: + return 0, "[dry-run]" + log(f"$ {' '.join(cmd)}") + proc = subprocess.run(cmd, capture_output=True, text=True, check=False) + out = (proc.stdout or "") + (proc.stderr or "") + return proc.returncode, out + + +def install_node(package: str, *, dry_run: bool = False, comfy_cmd: str = "comfy") -> bool: + cmd = comfy_cmd.split() + ["--skip-prompt", "node", "install", package] + code, _ = run_cmd(cmd, dry_run=dry_run) + return code == 0 + + +def install_model(url: str, folder: str, filename: str | None = None, + *, dry_run: bool = False, comfy_cmd: str = "comfy", + hf_token: str | None = None, civitai_token: str | None = None) -> bool: + cmd = comfy_cmd.split() + [ + "--skip-prompt", "model", "download", + "--url", url, + "--relative-path", f"models/{folder}", + ] + if filename: + cmd.extend(["--filename", filename]) + if hf_token: + cmd.extend(["--set-hf-api-token", hf_token]) + if civitai_token: + cmd.extend(["--set-civitai-api-token", civitai_token]) + code, _ = run_cmd(cmd, dry_run=dry_run) + return code == 0 + + +def main(argv: list[str] | None = None) -> int: + p = argparse.ArgumentParser(description="Run check_deps and install whatever is missing") + p.add_argument("workflow") + p.add_argument("--host", default=DEFAULT_LOCAL_HOST) + p.add_argument("--api-key", help=f"or set ${ENV_API_KEY}") + p.add_argument("--models-from-file", + help="JSON file mapping {model_filename: download_url} for models that need install") + p.add_argument("--hf-token", help="HuggingFace token for downloads") + p.add_argument("--civitai-token", help="CivitAI token for downloads") + p.add_argument("--dry-run", action="store_true", + help="Show what would be installed without doing it") + p.add_argument("--no-restart", action="store_true", + help="Don't suggest restarting the server after node install") + args = p.parse_args(argv) + + api_key = resolve_api_key(args.api_key) + + wf_path = Path(args.workflow).expanduser() + if not wf_path.exists(): + emit_json({"error": f"Workflow not found: {args.workflow}"}) + return 1 + try: + with wf_path.open() as f: + workflow = unwrap_workflow(json.load(f)) + except (ValueError, json.JSONDecodeError) as e: + emit_json({"error": str(e)}) + return 1 + + report = check_deps(workflow, host=args.host, api_key=api_key) + + if report["is_ready"]: + emit_json({"status": "ready", "report": report}) + return 0 + + if report["is_cloud"]: + emit_json({ + "status": "cannot_fix_cloud", + "reason": "Comfy Cloud preinstalls nodes; if something is genuinely missing, contact support.", + "report": report, + }) + return 1 + + comfy_cmd = comfy_cli_available() + if not comfy_cmd: + emit_json({ + "status": "cannot_fix", + "reason": "comfy-cli not on PATH; install with `pip install comfy-cli` or `pipx install comfy-cli`", + "report": report, + }) + return 1 + + actions: list[dict] = [] + failures: list[dict] = [] + + # ---- Install missing custom nodes ---- + seen_packages: set[str] = set() + for entry in report["missing_nodes"]: + cmd = entry.get("fix_command", "") + if cmd.startswith("comfy node install "): + package = cmd.split(" ")[-1] + if package in seen_packages: + continue + seen_packages.add(package) + ok = install_node(package, dry_run=args.dry_run, comfy_cmd=comfy_cmd) + (actions if ok else failures).append({ + "kind": "node", "package": package, "node_class": entry["class_type"], + "ok": ok, + }) + else: + failures.append({ + "kind": "node", "node_class": entry["class_type"], + "ok": False, "reason": "No registry mapping known. " + entry.get("fix_hint", ""), + }) + + # ---- Install missing models (only when URL provided) ---- + sources: dict[str, str] = {} + if args.models_from_file: + try: + sources = json.loads(Path(args.models_from_file).read_text()) + except (OSError, json.JSONDecodeError) as e: + log(f"Could not read --models-from-file: {e}") + + for entry in report["missing_models"]: + filename = entry["value"] + url = sources.get(filename) + if not url: + failures.append({ + "kind": "model", "filename": filename, "folder": entry["folder"], + "ok": False, "reason": "No URL provided in --models-from-file. " + "Refusing to guess.", + }) + continue + ok = install_model( + url, entry["folder"], filename, + dry_run=args.dry_run, comfy_cmd=comfy_cmd, + hf_token=args.hf_token, civitai_token=args.civitai_token, + ) + (actions if ok else failures).append({ + "kind": "model", "filename": filename, "folder": entry["folder"], + "url": url, "ok": ok, + }) + + # ---- Embeddings ---- + for entry in report["missing_embeddings"]: + emb_name = entry["embedding_name"] + # Try common extensions in user-supplied source map + url = (sources.get(f"{emb_name}.pt") + or sources.get(f"{emb_name}.safetensors") + or sources.get(emb_name)) + if not url: + failures.append({ + "kind": "embedding", "name": emb_name, + "ok": False, "reason": "No URL provided in --models-from-file.", + }) + continue + target_filename = ( + f"{emb_name}.safetensors" if url.endswith(".safetensors") + else f"{emb_name}.pt" + ) + ok = install_model( + url, "embeddings", target_filename, + dry_run=args.dry_run, comfy_cmd=comfy_cmd, + hf_token=args.hf_token, civitai_token=args.civitai_token, + ) + (actions if ok else failures).append({ + "kind": "embedding", "name": emb_name, "url": url, "ok": ok, + }) + + needs_restart = any(a["kind"] == "node" and a.get("ok") for a in actions) + + emit_json({ + "status": "fixed" if not failures else "partial", + "actions_taken": actions, + "failures": failures, + "needs_server_restart": needs_restart and not args.no_restart, + "restart_hint": "comfy stop && comfy launch --background", + "dry_run": args.dry_run, + }) + return 0 if not failures else 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/image-creation/comfyui/scripts/check_deps.py b/image-creation/comfyui/scripts/check_deps.py new file mode 100644 index 0000000..607e2c0 --- /dev/null +++ b/image-creation/comfyui/scripts/check_deps.py @@ -0,0 +1,437 @@ +#!/usr/bin/env python3 +""" +check_deps.py — Verify a ComfyUI workflow's dependencies (custom nodes, models, +embeddings) against a running server. + +Improvements over v1: + - Cloud-aware endpoint mapping (handles `/api/experiment/models/{folder}` and + `/api/object_info` variants verified against live cloud API) + - Distinguishes 200-empty (genuinely no models in folder) vs 404 + (folder doesn't exist) vs 403 (auth/tier issue) — no silent passes + - Outputs concrete remediation commands (e.g. `comfy node install `) + when nodes are missing + - Detects embedding references inside prompt strings as model deps + - Skips check on cloud free tier `/api/object_info` (403) without false alarm + - Accepts API key from CLI flag OR $COMFY_CLOUD_API_KEY env var + +Usage: + python3 check_deps.py workflow_api.json + python3 check_deps.py workflow_api.json --host 127.0.0.1 --port 8188 + python3 check_deps.py workflow_api.json --host https://cloud.comfy.org + +Stdlib-only. Python 3.10+. +""" + +from __future__ import annotations + +import argparse +import json +import sys +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from _common import ( # noqa: E402 + DEFAULT_LOCAL_HOST, ENV_API_KEY, + emit_json, folder_aliases_for, http_get, is_cloud_host, + iter_embedding_refs, iter_model_deps, iter_nodes, parse_model_list, + resolve_api_key, resolve_url, unwrap_workflow, +) + + +# Known node → custom-node-package map. When a workflow needs a node we don't +# recognize, suggesting the right `comfy node install ...` makes the difference +# between a working agent and a stuck one. +NODE_TO_PACKAGE: dict[str, str] = { + # rgthree (Reroute is JS-only and doesn't appear in /object_info) + "Power Lora Loader (rgthree)": "rgthree-comfy", + "Image Comparer (rgthree)": "rgthree-comfy", + "Seed (rgthree)": "rgthree-comfy", + "Display Any (rgthree)": "rgthree-comfy", + "Display Int (rgthree)": "rgthree-comfy", + # Impact pack + "FaceDetailer": "comfyui-impact-pack", + "DetailerForEach": "comfyui-impact-pack", + "BboxDetectorSEGS": "comfyui-impact-pack", + "SAMLoader": "comfyui-impact-pack", + "ImpactWildcardProcessor": "comfyui-impact-pack", + # Impact subpack (separate package) + "UltralyticsDetectorProvider": "comfyui-impact-subpack", + # Was Node Suite + "Image Save": "was-node-suite-comfyui", + "Number Counter": "was-node-suite-comfyui", + "Text String": "was-node-suite-comfyui", + # easy-use + "easy fullLoader": "comfyui-easy-use", + "easy positive": "comfyui-easy-use", + "easy negative": "comfyui-easy-use", + "easy seed": "comfyui-easy-use", + "easy imageSave": "comfyui-easy-use", + # Video Helper Suite + "VHS_VideoCombine": "comfyui-videohelpersuite", + "VHS_LoadVideo": "comfyui-videohelpersuite", + "VHS_LoadAudio": "comfyui-videohelpersuite", + # AnimateDiff + "ADE_AnimateDiffLoaderWithContext": "comfyui-animatediff-evolved", + "ADE_AnimateDiffLoaderGen1": "comfyui-animatediff-evolved", + "ADE_LoadAnimateDiffModel": "comfyui-animatediff-evolved", + # ControlNet aux preprocessors (full class names) + "CannyEdgePreprocessor": "comfyui_controlnet_aux", + "DWPreprocessor": "comfyui_controlnet_aux", + "OpenposePreprocessor": "comfyui_controlnet_aux", + "DepthAnythingPreprocessor": "comfyui_controlnet_aux", + "Zoe_DepthAnythingPreprocessor": "comfyui_controlnet_aux", + "AnimalPosePreprocessor": "comfyui_controlnet_aux", + # IPAdapter Plus + "IPAdapterAdvanced": "comfyui_ipadapter_plus", + "IPAdapterUnifiedLoader": "comfyui_ipadapter_plus", + "IPAdapterModelLoader": "comfyui_ipadapter_plus", + "IPAdapterInsightFaceLoader": "comfyui_ipadapter_plus", + # InstantID + "InstantIDModelLoader": "comfyui_instantid", + "ApplyInstantID": "comfyui_instantid", + # Comfy essentials (note: registry slug uses underscore, not hyphen) + "GetImageSize+": "comfyui_essentials", + "ImageBatchMultiple+": "comfyui_essentials", + # pysssss + "ShowText|pysssss": "comfyui-custom-scripts", + "PreviewImage|pysssss": "comfyui-custom-scripts", + # SUPIR + "SUPIR_Upscale": "comfyui-supir", + "SUPIR_first_stage": "comfyui-supir", + # GGUF (case-sensitive registry slug) + "UNETLoaderGGUF": "ComfyUI-GGUF", + "DualCLIPLoaderGGUF": "ComfyUI-GGUF", + # Florence2 + "Florence2Run": "comfyui-florence2", + # WAS + "Image Filter Adjustments": "was-node-suite-comfyui", + # Photomaker (case-sensitive) + "PhotoMakerLoader": "ComfyUI-PhotoMaker-Plus", + # Wan video (case-sensitive) + "WanVideoSampler": "ComfyUI-WanVideoWrapper", + "WanVideoModelLoader": "ComfyUI-WanVideoWrapper", +} + +# Nodes whose package isn't on the comfy registry — need git-URL install via +# ComfyUI-Manager. We surface a helpful hint instead of an unrunnable command. +NODE_TO_GIT_URL: dict[str, str] = { + "HunyuanVideoSampler": "https://github.com/kijai/ComfyUI-HunyuanVideoWrapper", + "HunyuanVideoModelLoader": "https://github.com/kijai/ComfyUI-HunyuanVideoWrapper", +} + + +def fetch_object_info(url: str, headers: dict) -> tuple[set[str] | None, dict | None]: + """Returns (installed_node_set, error_info). Error info is a dict if we + couldn't query (e.g. cloud free tier), else None. + """ + r = http_get(url, headers=headers, retries=2, timeout=30) + if r.status == 200: + try: + data = r.json() + if isinstance(data, dict): + return set(data.keys()), None + except Exception: + pass + return None, {"http_status": 200, "reason": "non-dict response"} + if r.status == 403: + try: + body = r.json() + except Exception: + body = {"raw": r.text()[:200]} + return None, {"http_status": 403, "reason": "forbidden", "body": body} + if r.status == 404: + return None, {"http_status": 404, "reason": "endpoint not found"} + return None, {"http_status": r.status, "reason": "unexpected", "body": r.text()[:200]} + + +def _fetch_one_folder( + base: str, folder: str, headers: dict, *, is_cloud: bool, +) -> tuple[set[str] | None, dict | None]: + """Single-folder fetch, no aliasing. Returns (installed_set, error_info).""" + url = resolve_url(base, f"/models/{folder}", is_cloud=is_cloud) + r = http_get(url, headers=headers, retries=2, timeout=30) + if r.status == 200: + try: + return parse_model_list(r.json()), None + except Exception: + return set(), {"http_status": 200, "reason": "non-list response"} + if r.status == 404: + body_text = r.text() + try: + body = r.json() + except Exception: + body = {"raw": body_text[:200]} + code = body.get("code") if isinstance(body, dict) else None + if code == "folder_not_found": + # Folder is genuinely empty/missing on server — not the same as + # "endpoint missing". Return empty set with informational error. + return set(), {"http_status": 404, "reason": "folder_empty_or_unknown", "body": body} + return None, {"http_status": 404, "reason": "endpoint not found", "body": body} + if r.status == 403: + try: + body = r.json() + except Exception: + body = {} + return None, {"http_status": 403, "reason": "forbidden", "body": body} + return None, {"http_status": r.status, "reason": "unexpected"} + + +def fetch_models_for_folder( + base: str, folder: str, headers: dict, *, is_cloud: bool, +) -> tuple[set[str] | None, dict | None]: + """Fetch installed models for a folder, trying aliases. + + Folder renames over time (e.g. unet → diffusion_models, clip → text_encoders) + mean a workflow asking for a model in `unet` may need to look in + `diffusion_models`. We union models from every reachable alias. + + Returns (combined_set | None, last_error | None). + """ + aliases = folder_aliases_for(folder) + combined: set[str] = set() + any_success = False + last_err: dict | None = None + for alias in aliases: + models, err = _fetch_one_folder(base, alias, headers, is_cloud=is_cloud) + if models is not None: + combined.update(models) + any_success = True + last_err = None + else: + last_err = err + if not any_success: + return None, last_err + return combined, None + + +def fetch_embeddings(base: str, headers: dict, *, is_cloud: bool) -> tuple[set[str] | None, dict | None]: + """Local ComfyUI exposes /embeddings; cloud uses /experiment/models/embeddings.""" + if is_cloud: + return fetch_models_for_folder(base, "embeddings", headers, is_cloud=True) + # Local: dedicated /embeddings returns a flat list of names + r = http_get(resolve_url(base, "/embeddings", is_cloud=False), headers=headers, retries=2) + if r.status == 200: + try: + data = r.json() + if isinstance(data, list): + # Strip extensions from the registered names since prompt syntax + # usually omits them ("embedding:goodvibes" vs "goodvibes.pt") + names = set() + for n in data: + if isinstance(n, str): + names.add(n) + # Also store stem for fuzzy matching + names.add(Path(n).stem) + return names, None + except Exception: + pass + return None, {"http_status": r.status, "reason": "unexpected"} + + +def normalize_for_match(name: str) -> set[str]: + """Generate matching variants of a model name (with/without extension, slashes, etc.)""" + s = {name} + s.add(Path(name).stem) + s.add(Path(name).name) + # ComfyUI sometimes strips/keeps the leading folder + if "/" in name or "\\" in name: + flat = name.replace("\\", "/").split("/")[-1] + s.add(flat) + s.add(Path(flat).stem) + return {x for x in s if x} + + +def model_present(needed: str, installed: set[str]) -> bool: + if not installed: + return False + needed_variants = normalize_for_match(needed) + installed_norm: set[str] = set() + for inst in installed: + installed_norm.update(normalize_for_match(inst)) + return bool(needed_variants & installed_norm) + + +def suggest_install_command(node_class: str) -> str | None: + pkg = NODE_TO_PACKAGE.get(node_class) + if pkg: + return f"comfy node install {pkg}" + return None + + +def suggest_git_url(node_class: str) -> str | None: + """For nodes not on the registry, return a git URL the user can hand to + ComfyUI-Manager's `/manager/queue/install` endpoint.""" + return NODE_TO_GIT_URL.get(node_class) + + +def check_deps( + workflow: dict, host: str, *, api_key: str | None = None, +) -> dict: + headers: dict[str, str] = {} + if api_key: + headers["X-API-Key"] = api_key + + is_cloud = is_cloud_host(host) + base = host.rstrip("/") + + # ---- 1. Required nodes ---- + required_nodes: set[str] = set() + for _, node in iter_nodes(workflow): + required_nodes.add(node["class_type"]) + + object_info_url = resolve_url(base, "/object_info", is_cloud=is_cloud) + installed_nodes, obj_err = fetch_object_info(object_info_url, headers) + + missing_nodes: list[dict] = [] + node_check_skipped = False + if installed_nodes is None: + # Couldn't query (e.g. cloud free tier). Don't false-alarm; mark skipped. + node_check_skipped = True + else: + for cls in sorted(required_nodes): + if cls not in installed_nodes: + entry = {"class_type": cls} + cmd = suggest_install_command(cls) + git_url = suggest_git_url(cls) + if cmd: + entry["fix_command"] = cmd + elif git_url: + entry["fix_git_url"] = git_url + entry["fix_hint"] = ( + f"Not on registry. Install via Manager with this git URL: {git_url}" + ) + else: + entry["fix_hint"] = ( + "Search https://registry.comfy.org or " + "use ComfyUI-Manager UI to find the package providing this node." + ) + missing_nodes.append(entry) + + # ---- 2. Required models ---- + model_cache: dict[str, tuple[set[str] | None, dict | None]] = {} + missing_models: list[dict] = [] + folder_errors: dict[str, dict] = {} + + for dep in iter_model_deps(workflow): + folder = dep["folder"] + if folder not in model_cache: + model_cache[folder] = fetch_models_for_folder( + base, folder, headers, is_cloud=is_cloud, + ) + installed, err = model_cache[folder] + if installed is None: + # Couldn't enumerate this folder — record once + folder_errors.setdefault(folder, err or {}) + # Don't flag as missing (we don't know); the folder_errors block surfaces this + continue + if not model_present(dep["value"], installed): + entry = dict(dep) + entry["fix_hint"] = ( + f"comfy model download --url --relative-path models/{folder} " + f"--filename {dep['value']!r}" + ) + missing_models.append(entry) + + # ---- 3. Embedding refs in prompts ---- + emb_installed, emb_err = fetch_embeddings(base, headers, is_cloud=is_cloud) + missing_embeddings: list[dict] = [] + seen_emb: set[tuple[str, str]] = set() + for nid, emb_name in iter_embedding_refs(workflow): + if (nid, emb_name) in seen_emb: + continue + seen_emb.add((nid, emb_name)) + if emb_installed is None: + # Couldn't enumerate — skip silently here, surface the error in the + # folder_errors block + continue + if not model_present(emb_name, emb_installed): + missing_embeddings.append({ + "node_id": nid, + "embedding_name": emb_name, + "folder": "embeddings", + "fix_hint": ( + f"Download {emb_name}.pt or .safetensors and place in " + f"models/embeddings/, or `comfy model download --url " + f"--relative-path models/embeddings`" + ), + }) + + if emb_err and emb_installed is None: + folder_errors.setdefault("embeddings", emb_err) + + is_ready = ( + not node_check_skipped + and not missing_nodes + and not missing_models + and not missing_embeddings + ) + + return { + "is_ready": is_ready, + "node_check_skipped": node_check_skipped, + "node_check_skip_reason": obj_err if node_check_skipped else None, + "missing_nodes": missing_nodes, + "missing_models": missing_models, + "missing_embeddings": missing_embeddings, + "folder_errors": folder_errors, + # 0 is a legitimate count (e.g. empty server). Use None only when not queried. + "installed_node_count": len(installed_nodes) if installed_nodes is not None else None, + "required_node_count": len(required_nodes), + "required_nodes": sorted(required_nodes), + "host": base, + "is_cloud": is_cloud, + } + + +def main(argv: list[str] | None = None) -> int: + p = argparse.ArgumentParser(description="Check ComfyUI workflow dependencies against a running server") + p.add_argument("workflow", help="Path to workflow API JSON file") + p.add_argument("--host", default=DEFAULT_LOCAL_HOST, help="ComfyUI server URL") + p.add_argument("--port", type=int, help="Server port (overrides --host port)") + p.add_argument("--api-key", help=f"API key for cloud (or set ${ENV_API_KEY} env var)") + p.add_argument("--strict", action="store_true", + help="Exit non-zero if node check is skipped (e.g. on cloud free tier)") + args = p.parse_args(argv) + + host = args.host + if args.port is not None: + # Strip any port from host and append --port + from urllib.parse import urlparse, urlunparse + parsed = urlparse(host if "://" in host else f"http://{host}") + new_netloc = f"{parsed.hostname}:{args.port}" + host = urlunparse(parsed._replace(netloc=new_netloc)) + + api_key = resolve_api_key(args.api_key) + + wf_path = Path(args.workflow).expanduser() + if not wf_path.exists(): + emit_json({"error": f"Workflow file not found: {args.workflow}"}) + return 1 + try: + with wf_path.open() as f: + payload = json.load(f) + workflow = unwrap_workflow(payload) + except ValueError as e: + emit_json({"error": str(e)}) + return 1 + except json.JSONDecodeError as e: + emit_json({"error": f"Invalid JSON: {e}"}) + return 1 + + try: + result = check_deps(workflow, host=host, api_key=api_key) + except Exception as e: + emit_json({"error": f"Dep check failed: {e}", "host": host}) + return 1 + + emit_json(result) + + if not result["is_ready"]: + return 1 + if args.strict and result["node_check_skipped"]: + return 1 + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/image-creation/comfyui/scripts/comfyui_setup.sh b/image-creation/comfyui/scripts/comfyui_setup.sh new file mode 100644 index 0000000..dd03698 --- /dev/null +++ b/image-creation/comfyui/scripts/comfyui_setup.sh @@ -0,0 +1,286 @@ +#!/usr/bin/env bash +# ComfyUI Setup — Install, launch, and verify using the official comfy-cli. +# +# Improvements over v1: +# - Prefers `pipx` / `uvx` over global `pip install` (avoids polluting system Python) +# - Idempotent: detects already-running server and skips re-launch +# - Configurable port via --port=N (default 8188) +# - Configurable workspace via --workspace=PATH +# - Persistent log file in /tmp/comfyui_setup..log for debugging +# - SIGINT trap cleans up partial state +# - Refuses local install when hardware_check.py verdict is "cloud" +# - Forwards extra flags to comfy-cli (e.g. --cuda-version=12.4) +# +# Usage: +# bash scripts/comfyui_setup.sh +# (auto-detects GPU; uses recommendation from hardware_check.py) +# bash scripts/comfyui_setup.sh --nvidia +# bash scripts/comfyui_setup.sh --m-series --port=8190 +# bash scripts/comfyui_setup.sh --amd --workspace=/data/comfy +# +# Flags: +# --nvidia | --amd | --m-series | --cpu GPU selection (skips hw check) +# --port=N HTTP port (default 8188) +# --workspace=PATH ComfyUI install location +# --skip-launch Install only, don't start server +# --force-cloud-override Install locally even if hw says cloud +# -- Pass remaining args to `comfy install` + +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +HARDWARE_CHECK="$SCRIPT_DIR/hardware_check.py" +LOG_FILE="/tmp/comfyui_setup.$$.log" +PORT=8188 +WORKSPACE="" +GPU_FLAG="" +SKIP_LAUNCH=0 +FORCE_CLOUD_OVERRIDE=0 +EXTRA_INSTALL_ARGS=() + +cleanup() { + local exit_code=$? + if [ $exit_code -ne 0 ]; then + echo "==> Setup exited with status $exit_code. Log: $LOG_FILE" >&2 + fi + exit $exit_code +} +trap cleanup EXIT INT TERM + +log() { echo "==> $*" | tee -a "$LOG_FILE" >&2; } +err() { echo "ERROR: $*" | tee -a "$LOG_FILE" >&2; } + +# --- Argument parsing --- +PASSTHROUGH=0 +for arg in "$@"; do + if [ "$PASSTHROUGH" -eq 1 ]; then + EXTRA_INSTALL_ARGS+=("$arg") + continue + fi + case "$arg" in + --nvidia|--amd|--m-series|--cpu) + GPU_FLAG="$arg" + ;; + --port=*) + PORT="${arg#*=}" + ;; + --workspace=*) + WORKSPACE="${arg#*=}" + ;; + --skip-launch) + SKIP_LAUNCH=1 + ;; + --force-cloud-override) + FORCE_CLOUD_OVERRIDE=1 + ;; + --) + PASSTHROUGH=1 + ;; + --help|-h) + # Print the leading comment block, stripping the `# ` prefix. + # Stops at the first blank line which separates docs from code. + awk ' + NR == 1 { next } # skip shebang + /^[^#]/ { exit } # stop at first non-comment line + /^$/ { exit } # ...or first blank line + { sub(/^# ?/, ""); print } + ' "$0" + exit 0 + ;; + *) + err "Unknown argument: $arg" + exit 64 + ;; + esac +done + +log "Logging to $LOG_FILE" + +# --- Step 0: Hardware check (skipped if user gave an explicit GPU flag) --- +if [ -z "$GPU_FLAG" ]; then + if [ ! -f "$HARDWARE_CHECK" ]; then + log "hardware_check.py not found — defaulting to --nvidia" + GPU_FLAG="--nvidia" + else + log "Running hardware check…" + set +e + HW_JSON="$(python3 "$HARDWARE_CHECK" --json 2>>"$LOG_FILE")" + HW_EXIT=$? + set -e + + if [ -z "$HW_JSON" ]; then + err "hardware_check.py produced no output (exit $HW_EXIT). Pass an explicit flag." + exit 1 + fi + echo "$HW_JSON" | tee -a "$LOG_FILE" >&2 + + VERDICT="$(echo "$HW_JSON" | python3 -c 'import sys,json; print(json.load(sys.stdin).get("verdict",""))')" + FLAG="$(echo "$HW_JSON" | python3 -c 'import sys,json; print(json.load(sys.stdin).get("comfy_cli_flag") or "")')" + + if [ "$VERDICT" = "cloud" ] && [ "$FORCE_CLOUD_OVERRIDE" -ne 1 ]; then + log "" + log "Hardware check: this machine is not suitable for local ComfyUI." + log "Recommended: Comfy Cloud — https://platform.comfy.org" + log "" + log "To override and force a local install, re-run with --force-cloud-override" + log "or pass an explicit GPU flag (--nvidia|--amd|--m-series|--cpu)." + exit 2 + fi + + if [ "$VERDICT" = "marginal" ]; then + log "Hardware check: verdict is MARGINAL." + log " SD1.5 should work; SDXL/Flux may be slow or OOM." + log " Consider Comfy Cloud for heavier workflows: https://platform.comfy.org" + fi + + if [ -z "$FLAG" ]; then + log "hardware_check could not pick a comfy-cli flag. Defaulting to --nvidia." + log "(For Intel Arc or unsupported hardware, use the manual install path.)" + GPU_FLAG="--nvidia" + else + GPU_FLAG="$FLAG" + fi + fi +fi + +log "GPU flag: $GPU_FLAG" +log "Port: $PORT" +[ -n "$WORKSPACE" ] && log "Workspace: $WORKSPACE" +[ "${#EXTRA_INSTALL_ARGS[@]}" -gt 0 ] && log "Extra install args: ${EXTRA_INSTALL_ARGS[*]}" + +# --- Step 1: Install comfy-cli (prefer pipx / uvx over global pip) --- +COMFY_BIN="" +if command -v comfy >/dev/null 2>&1; then + COMFY_BIN="comfy" + log "comfy-cli already on PATH: $(comfy -v 2>/dev/null || echo 'unknown version')" +elif command -v uvx >/dev/null 2>&1; then + log "Using uvx (no install needed)" + COMFY_BIN="uvx --from comfy-cli comfy" +elif command -v pipx >/dev/null 2>&1; then + log "Installing comfy-cli via pipx…" + pipx install comfy-cli >>"$LOG_FILE" 2>&1 + COMFY_BIN="comfy" + # pipx adds shims to ~/.local/bin which may need to be on PATH + if ! command -v comfy >/dev/null 2>&1; then + if [ -x "$HOME/.local/bin/comfy" ]; then + export PATH="$HOME/.local/bin:$PATH" + COMFY_BIN="$HOME/.local/bin/comfy" + fi + fi +else + log "Neither pipx nor uvx found. Falling back to pip install --user…" + log " (Recommend installing pipx: https://pipx.pypa.io)" + if ! pip install --user comfy-cli >>"$LOG_FILE" 2>&1; then + # macOS: PEP 668 externally-managed-environment may block --user + log "pip install --user failed. Retrying with --break-system-packages…" + pip install --user --break-system-packages comfy-cli >>"$LOG_FILE" 2>&1 || { + err "Could not install comfy-cli. Install pipx or uv first." + exit 1 + } + fi + # Resolve the actual `comfy` script — pip --user puts it in: + # Linux: ~/.local/bin/comfy + # macOS: ~/Library/Python//bin/comfy OR ~/.local/bin/comfy + COMFY_BIN="" + for candidate in "$HOME/.local/bin/comfy" \ + "$HOME/Library/Python/3.13/bin/comfy" \ + "$HOME/Library/Python/3.12/bin/comfy" \ + "$HOME/Library/Python/3.11/bin/comfy" \ + "$HOME/Library/Python/3.10/bin/comfy"; do + if [ -x "$candidate" ]; then + COMFY_BIN="$candidate" + export PATH="$(dirname "$candidate"):$PATH" + break + fi + done + if [ -z "$COMFY_BIN" ]; then + if command -v comfy >/dev/null 2>&1; then + COMFY_BIN="comfy" + else + err "Installed comfy-cli but couldn't find the 'comfy' script." + err "Add the right Python user-bin directory to PATH and retry." + exit 1 + fi + fi +fi + +# --- Step 2: Disable analytics tracking (avoid interactive prompt) --- +log "Disabling analytics tracking…" +$COMFY_BIN --skip-prompt tracking disable >>"$LOG_FILE" 2>&1 || true + +# --- Step 3: Install ComfyUI --- +WORKSPACE_ARG=() +if [ -n "$WORKSPACE" ]; then + WORKSPACE_ARG=(--workspace "$WORKSPACE") +fi + +if $COMFY_BIN "${WORKSPACE_ARG[@]}" which 2>/dev/null | grep -q "ComfyUI"; then + EXISTING_WS="$($COMFY_BIN "${WORKSPACE_ARG[@]}" which 2>/dev/null || true)" + log "ComfyUI already installed at: $EXISTING_WS" +else + log "Installing ComfyUI ($GPU_FLAG)…" + if ! $COMFY_BIN "${WORKSPACE_ARG[@]}" --skip-prompt install "$GPU_FLAG" "${EXTRA_INSTALL_ARGS[@]}" >>"$LOG_FILE" 2>&1; then + err "Install failed. Tail of log:" + tail -20 "$LOG_FILE" >&2 + exit 1 + fi +fi + +if [ "$SKIP_LAUNCH" -eq 1 ]; then + log "Setup complete (--skip-launch). Run \`$COMFY_BIN launch --background -- --port $PORT\` when ready." + exit 0 +fi + +# --- Step 4: Detect already-running server --- +if curl -fsS "http://127.0.0.1:$PORT/system_stats" >/dev/null 2>&1; then + log "Server already running on port $PORT — skipping launch." + log "Stop with \`$COMFY_BIN stop\` if you want a fresh start." + curl -fsS "http://127.0.0.1:$PORT/system_stats" | python3 -m json.tool 2>/dev/null || true + log "Done." + exit 0 +fi + +# --- Step 5: Launch --- +log "Launching ComfyUI in background on port $PORT…" +LAUNCH_EXTRAS=("--" "--port" "$PORT") +if ! $COMFY_BIN "${WORKSPACE_ARG[@]}" launch --background "${LAUNCH_EXTRAS[@]}" >>"$LOG_FILE" 2>&1; then + err "Background launch failed. Tail of log:" + tail -20 "$LOG_FILE" >&2 + err "Try foreground launch to see real-time errors: $COMFY_BIN launch -- --port $PORT" + exit 1 +fi + +# --- Step 6: Wait for server --- +log "Waiting for server…" +MAX_WAIT=60 +ELAPSED=0 +while [ $ELAPSED -lt $MAX_WAIT ]; do + if curl -fsS "http://127.0.0.1:$PORT/system_stats" >/dev/null 2>&1; then + log "Server is running!" + curl -fsS "http://127.0.0.1:$PORT/system_stats" | python3 -m json.tool 2>/dev/null || true + break + fi + sleep 2 + ELAPSED=$((ELAPSED + 2)) +done + +if [ $ELAPSED -ge $MAX_WAIT ]; then + err "Server did not start within ${MAX_WAIT}s." + err "Inspect log: $LOG_FILE" + err "Or run foreground: $COMFY_BIN launch -- --port $PORT" + exit 1 +fi + +log "" +log "Setup complete!" +log " Server: http://127.0.0.1:$PORT" +log " Web UI: http://127.0.0.1:$PORT (open in browser)" +log " Stop: $COMFY_BIN stop" +log " Log: $LOG_FILE (kept until shell closes)" +log "" +log "Next steps:" +log " - Download a model: $COMFY_BIN model download --url --relative-path models/checkpoints" +log " - Run a workflow: python3 $SCRIPT_DIR/run_workflow.py --workflow --args '{...}'" + +# Disable trap on success path +trap - EXIT diff --git a/image-creation/comfyui/scripts/extract_schema.py b/image-creation/comfyui/scripts/extract_schema.py new file mode 100644 index 0000000..0eab65b --- /dev/null +++ b/image-creation/comfyui/scripts/extract_schema.py @@ -0,0 +1,315 @@ +#!/usr/bin/env python3 +""" +extract_schema.py — Analyze a ComfyUI API-format workflow and extract +controllable parameters. + +Improvements over v1: + - Catalogs live in `_common.py`, shared with `check_deps.py` + - Coverage expanded for Flux / SD3 / Wan / Hunyuan / LTX / IPAdapter / rgthree + - Symmetric duplicate-name resolution: ALL duplicates get a node-id suffix + (instead of "first wins, second renamed"), so callers see consistent names + - Negative prompt detected by tracing `KSampler.negative` connections back to + the source CLIPTextEncode (more reliable than meta-title heuristic) + - Embedding references in prompt text are extracted as model dependencies + - Detects Primitive nodes that drive other nodes' inputs (and surfaces them + as the user-facing parameter) + - Reroutes are followed when tracing connections + +Usage: + python3 extract_schema.py workflow_api.json + python3 extract_schema.py workflow_api.json --output schema.json + +Stdlib-only. Python 3.10+. +""" + +from __future__ import annotations + +import argparse +import json +import sys +from pathlib import Path +from typing import Any + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from _common import ( # noqa: E402 + OUTPUT_NODES, PARAM_PATTERNS, PROMPT_FIELDS, + is_link, iter_embedding_refs, iter_model_deps, iter_nodes, unwrap_workflow, +) + + +# Sampler nodes whose `positive` / `negative` connections we trace +SAMPLER_NODE_FAMILY = { + "KSampler", "KSamplerAdvanced", + "SamplerCustom", "SamplerCustomAdvanced", + "BasicGuider", "CFGGuider", "DualCFGGuider", +} + + +def infer_type(value: Any) -> str: + if isinstance(value, bool): + return "bool" + if isinstance(value, int): + return "int" + if isinstance(value, float): + return "float" + if isinstance(value, str): + return "string" + if isinstance(value, list): + return "link" + if isinstance(value, dict): + return "object" + return "unknown" + + +def trace_to_node(workflow: dict, link: list, *, max_hops: int = 8) -> str | None: + """Follow a [node_id, slot] link, hopping through Reroute / Primitive nodes + if needed, to find the *upstream* node id that holds the actual value/input. + + Bounded by both `max_hops` AND a visited-set to prevent infinite loops on + pathological graphs. + """ + if not is_link(link): + return None + nid: str | None = link[0] + visited: set[str] = set() + for _ in range(max_hops): + if nid is None or nid in visited: + return nid + visited.add(nid) + node = workflow.get(nid) + if not isinstance(node, dict): + return None + cls = node.get("class_type", "") + # Reroute / Primitive / passthrough wrappers + if cls in {"Reroute", "PrimitiveNode", "Note", "easy showAnything"}: + inputs = node.get("inputs", {}) or {} + # Find first link-shaped input and follow it + next_link = next((v for v in inputs.values() if is_link(v)), None) + if next_link is None: + return nid + nid = next_link[0] + continue + return nid + return nid + + +def find_negative_prompt_node(workflow: dict) -> str | None: + """Trace `negative` input of a sampler back to the source text encoder.""" + for nid, node in iter_nodes(workflow): + if node["class_type"] not in SAMPLER_NODE_FAMILY: + continue + inputs = node.get("inputs", {}) or {} + neg = inputs.get("negative") + if not is_link(neg): + continue + src = trace_to_node(workflow, neg) + if src and isinstance(workflow.get(src), dict): + cls = workflow[src].get("class_type", "") + if cls.startswith("CLIPTextEncode") or cls in {"smZ CLIPTextEncode", "BNK_CLIPTextEncodeAdvanced"}: + return src + return None + + +def find_positive_prompt_node(workflow: dict) -> str | None: + for nid, node in iter_nodes(workflow): + if node["class_type"] not in SAMPLER_NODE_FAMILY: + continue + inputs = node.get("inputs", {}) or {} + pos = inputs.get("positive") + if not is_link(pos): + continue + src = trace_to_node(workflow, pos) + if src and isinstance(workflow.get(src), dict): + cls = workflow[src].get("class_type", "") + if cls.startswith("CLIPTextEncode") or cls in {"smZ CLIPTextEncode", "BNK_CLIPTextEncodeAdvanced"}: + return src + return None + + +def extract_schema(workflow: dict) -> dict: + """Extract controllable parameters from a workflow. + + Returns: + { + "parameters": { friendly_name: {node_id, field, type, value, ...} }, + "output_nodes": [node_id, ...], + "model_dependencies": [{node_id, class_type, field, value, folder}], + "embedding_dependencies": [{node_id, embedding_name, found_in_field, value_excerpt}], + "summary": {...} + } + """ + output_nodes: list[str] = [] + + # First pass: identify positive / negative prompt nodes via connection tracing + pos_node = find_positive_prompt_node(workflow) + neg_node = find_negative_prompt_node(workflow) + + # ----- collect raw parameter candidates ----- + # Each candidate = (friendly_name, node_id, field, value) + # We resolve duplicate friendly_names AFTER the loop so dedup is symmetric. + raw_params: list[dict] = [] + + for node_id, node in iter_nodes(workflow): + cls = node["class_type"] + inputs = node.get("inputs", {}) or {} + + if cls in OUTPUT_NODES: + output_nodes.append(node_id) + + # Match this node against PARAM_PATTERNS + for p_class, p_field, friendly in PARAM_PATTERNS: + if cls != p_class: + continue + if p_field not in inputs: + continue + value = inputs[p_field] + t = infer_type(value) + if t == "link": + continue # connections aren't directly controllable + + actual_name = friendly + + # Disambiguate prompt vs negative_prompt by connection tracing + if friendly == "prompt": + if node_id == neg_node and pos_node != neg_node: + actual_name = "negative_prompt" + elif node_id == pos_node: + actual_name = "prompt" + else: + # Fallback: use _meta.title hints if present + meta_title = (node.get("_meta") or {}).get("title", "").lower() + if any(t_ in meta_title for t_ in ("negative", "neg", "-prompt", "anti")): + actual_name = "negative_prompt" + + raw_params.append({ + "name_hint": actual_name, + "node_id": node_id, + "field": p_field, + "type": t, + "value": value, + "class_type": cls, + }) + + # ----- symmetric duplicate-name resolution ----- + # Group by name_hint. If a hint appears once, keep it. If multiple, suffix + # ALL with their node_id. Always-stable, always-uniquely-addressable. + by_name: dict[str, list[dict]] = {} + for r in raw_params: + by_name.setdefault(r["name_hint"], []).append(r) + + parameters: dict[str, dict] = {} + for name, entries in by_name.items(): + if len(entries) == 1: + r = entries[0] + parameters[name] = { + "node_id": r["node_id"], "field": r["field"], + "type": r["type"], "value": r["value"], + "class_type": r["class_type"], + } + else: + # Sort by node_id (string-natural) for stability + entries.sort(key=lambda x: (str(x["node_id"]).zfill(8), x["field"])) + for r in entries: + full_name = f"{name}_{r['node_id']}" + parameters[full_name] = { + "node_id": r["node_id"], "field": r["field"], + "type": r["type"], "value": r["value"], + "class_type": r["class_type"], + "alias_of": name, + } + + # ----- model dependencies ----- + model_deps = list(iter_model_deps(workflow)) + + # ----- embedding dependencies (in prompt text) ----- + embedding_deps: list[dict] = [] + seen_emb: set[tuple[str, str]] = set() + for nid, emb_name in iter_embedding_refs(workflow): + key = (nid, emb_name) + if key in seen_emb: + continue + seen_emb.add(key) + # Find which field had the reference, for context + node = workflow.get(nid, {}) + inputs = node.get("inputs", {}) or {} + found_field = None + excerpt = None + for fname, fval in inputs.items(): + if isinstance(fval, str) and fname in PROMPT_FIELDS and emb_name in fval: + found_field = fname + excerpt = fval[:120] + break + embedding_deps.append({ + "node_id": nid, + "embedding_name": emb_name, + "field": found_field, + "value_excerpt": excerpt, + "folder": "embeddings", + }) + + # ----- summary ----- + summary = { + "parameter_count": len(parameters), + "output_node_count": len(output_nodes), + "model_dep_count": len(model_deps), + "embedding_dep_count": len(embedding_deps), + "has_negative_prompt": "negative_prompt" in parameters, + "has_seed": "seed" in parameters or any(p.startswith("seed_") for p in parameters), + "is_video_workflow": any( + workflow.get(n, {}).get("class_type", "") in { + "VHS_VideoCombine", "SaveVideo", "SaveAnimatedWEBP", "SaveAnimatedPNG", + } for n in output_nodes + ), + } + + return { + "parameters": parameters, + "output_nodes": output_nodes, + "model_dependencies": model_deps, + "embedding_dependencies": embedding_deps, + "summary": summary, + } + + +def main(argv: list[str] | None = None) -> int: + p = argparse.ArgumentParser(description="Extract controllable parameters from a ComfyUI workflow") + p.add_argument("workflow", help="Path to workflow API JSON file") + p.add_argument("--output", "-o", help="Output file (default: stdout)") + p.add_argument("--summary-only", action="store_true", + help="Only print the summary block") + args = p.parse_args(argv) + + wf_path = Path(args.workflow).expanduser() + if not wf_path.exists(): + print(f"Error: {wf_path} not found", file=sys.stderr) + return 1 + + try: + with wf_path.open() as f: + payload = json.load(f) + workflow = unwrap_workflow(payload) + except ValueError as e: + print(f"Error: {e}", file=sys.stderr) + return 1 + except json.JSONDecodeError as e: + print(f"Error: invalid JSON — {e}", file=sys.stderr) + return 1 + + schema = extract_schema(workflow) + + if args.summary_only: + out = json.dumps(schema["summary"], indent=2) + else: + out = json.dumps(schema, indent=2, default=str) + + if args.output: + Path(args.output).write_text(out) + print(f"Schema written to {args.output}", file=sys.stderr) + else: + print(out) + + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/image-creation/comfyui/scripts/fetch_logs.py b/image-creation/comfyui/scripts/fetch_logs.py new file mode 100644 index 0000000..e885a03 --- /dev/null +++ b/image-creation/comfyui/scripts/fetch_logs.py @@ -0,0 +1,157 @@ +#!/usr/bin/env python3 +""" +fetch_logs.py — Retrieve workflow execution diagnostics from a ComfyUI server. + +When a workflow errors, the server's /history (local) or /jobs (cloud) entry +contains the full Python traceback. This script makes it easy to fetch by +prompt_id, with sensible formatting. + +Usage: + python3 fetch_logs.py + python3 fetch_logs.py --host https://cloud.comfy.org + python3 fetch_logs.py --tail-queue # show currently queued/running jobs +""" + +from __future__ import annotations + +import argparse +import sys +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from _common import ( # noqa: E402 + DEFAULT_LOCAL_HOST, ENV_API_KEY, emit_json, http_get, is_cloud_host, + resolve_api_key, resolve_url, +) + + +def fetch_history_entry(host: str, headers: dict, prompt_id: str, *, is_cloud: bool) -> dict: + if is_cloud: + # Try /jobs/{id} first + url = resolve_url(host, f"/jobs/{prompt_id}", is_cloud=True) + r = http_get(url, headers=headers, retries=2, timeout=30) + if r.status == 200: + try: + return {"ok": True, "entry": r.json(), "source": "/api/jobs"} + except Exception: + pass + # Fallback to history_v2 + url = resolve_url(host, f"/history/{prompt_id}", is_cloud=True) + r = http_get(url, headers=headers, retries=2, timeout=30) + try: + data = r.json() + except Exception: + data = None + if r.status == 200 and data: + return {"ok": True, "entry": data, "source": "/api/history_v2"} + return {"ok": False, "http_status": r.status, "body": r.text()[:500]} + + url = resolve_url(host, f"/history/{prompt_id}", is_cloud=False) + r = http_get(url, headers=headers, retries=2, timeout=30) + if r.status != 200: + return {"ok": False, "http_status": r.status, "body": r.text()[:500]} + try: + data = r.json() + except Exception: + return {"ok": False, "reason": "non-JSON response"} + if not isinstance(data, dict) or prompt_id not in data: + return {"ok": False, "reason": "prompt_id not found in history", + "history_keys": list(data.keys())[:5] if isinstance(data, dict) else []} + return {"ok": True, "entry": data[prompt_id], "source": "/history"} + + +def fetch_queue(host: str, headers: dict) -> dict: + url = resolve_url(host, "/queue") + r = http_get(url, headers=headers, retries=2, timeout=15) + try: + data = r.json() + except Exception: + data = {"raw": r.text()[:500]} + return {"http_status": r.status, "data": data} + + +def extract_diagnostics(entry: dict) -> dict: + """Pull out the parts a human cares about: status, errors, traceback, timing.""" + diag: dict = {} + status = entry.get("status") or {} + diag["status_str"] = status.get("status_str") + diag["completed"] = status.get("completed") + + messages = status.get("messages") or [] + diag["execution_log"] = [] + for msg in messages: + if isinstance(msg, list) and len(msg) >= 2: + mtype, mdata = msg[0], msg[1] + diag["execution_log"].append({"type": mtype, "data": mdata}) + else: + diag["execution_log"].append(msg) + + # Look for execution_error inside messages + errors = [] + for msg in messages: + if isinstance(msg, list) and len(msg) >= 2 and msg[0] == "execution_error": + errors.append(msg[1]) + if errors: + diag["errors"] = errors + + # Cloud's /jobs response shape: top-level outputs / status / etc. + if "outputs" in entry: + out = entry["outputs"] or {} + if isinstance(out, dict): + diag["output_node_ids"] = list(out.keys()) + # Count file refs across all output buckets (images / video / etc.) + total = 0 + for node_output in out.values(): + if not isinstance(node_output, dict): + continue + for v in node_output.values(): + if isinstance(v, list): + total += len(v) + diag["output_count"] = total + else: + diag["output_node_ids"] = [] + diag["output_count"] = 0 + return diag + + +def main(argv: list[str] | None = None) -> int: + p = argparse.ArgumentParser(description="Fetch workflow execution diagnostics") + p.add_argument("prompt_id", nargs="?", help="prompt_id to look up") + p.add_argument("--host", default=DEFAULT_LOCAL_HOST) + p.add_argument("--api-key", help=f"or set ${ENV_API_KEY}") + p.add_argument("--raw", action="store_true", + help="Print the full history entry instead of the digest") + p.add_argument("--tail-queue", action="store_true", + help="Show currently running/pending jobs instead") + args = p.parse_args(argv) + + api_key = resolve_api_key(args.api_key) + headers = {"X-API-Key": api_key} if api_key else {} + is_cloud = is_cloud_host(args.host) + + if args.tail_queue: + emit_json(fetch_queue(args.host, headers)) + return 0 + + if not args.prompt_id: + print("Error: prompt_id is required (or use --tail-queue)", file=sys.stderr) + return 1 + + res = fetch_history_entry(args.host, headers, args.prompt_id, is_cloud=is_cloud) + if not res.get("ok"): + emit_json(res) + return 1 + + if args.raw: + emit_json(res) + return 0 + + diag = extract_diagnostics(res["entry"]) + diag["source"] = res.get("source") + diag["prompt_id"] = args.prompt_id + emit_json(diag) + return 0 if diag.get("status_str") not in {"error",} else 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/image-creation/comfyui/scripts/hardware_check.py b/image-creation/comfyui/scripts/hardware_check.py new file mode 100644 index 0000000..083d018 --- /dev/null +++ b/image-creation/comfyui/scripts/hardware_check.py @@ -0,0 +1,497 @@ +#!/usr/bin/env python3 +"""hardware_check.py — Detect whether this machine can realistically run ComfyUI locally. + +Improvements over v1: + - Multi-GPU detection: scans all NVIDIA / AMD GPUs, picks the best one (most VRAM) + - Apple Silicon: detects Rosetta-via-x86_64 false negative; warns instead of misclassifying + - Apple generation: defaults to None (unknown) instead of mis-tagging as M1 + - WSL2 detection: identifies WSL2 + nvidia-smi situation explicitly + - ROCm: prefers `rocm-smi --json` for new ROCm 6.x output + - Disk space check: warns if /home or workspace volume has < 25 GB free + - PyTorch verification (optional): tries to import torch and check device availability + - Windows: prefers PowerShell `Get-CimInstance` over deprecated `wmic` + - More accurate VRAM thresholds and verdict reasons + +Emits a structured JSON report. Exit codes match `verdict`: + 0 → ok + 1 → marginal + 2 → cloud + +Usage: + python3 hardware_check.py [--json] [--check-pytorch] +""" + +from __future__ import annotations + +import json +import os +import platform +import re +import shutil +import subprocess +import sys +from typing import Any + + +# Thresholds (GiB). +MIN_VRAM_GB_USABLE = 6 +OK_VRAM_GB = 8 +GREAT_VRAM_GB = 12 +MIN_MAC_RAM_GB = 16 +OK_MAC_RAM_GB = 32 +MIN_FREE_DISK_GB = 25 # ComfyUI core ~5 GB + one model ~5–24 GB + +_COMFY_CLI_FLAG = { + "nvidia": "--nvidia", + "amd": "--amd", + "apple-silicon": "--m-series", + "intel": None, + "comfy-cloud": None, + "cpu": "--cpu", +} + + +def _run(cmd: list[str], timeout: int = 8) -> str: + try: + out = subprocess.run( + cmd, capture_output=True, text=True, timeout=timeout, check=False + ) + return (out.stdout or "") + (out.stderr or "") + except (FileNotFoundError, subprocess.TimeoutExpired, OSError): + return "" + + +def is_wsl() -> bool: + """Return True when running under Windows Subsystem for Linux.""" + if platform.system() != "Linux": + return False + if "microsoft" in platform.release().lower() or "wsl" in platform.release().lower(): + return True + try: + with open("/proc/version", "r") as fh: + return "microsoft" in fh.read().lower() + except OSError: + return False + + +def is_rosetta() -> bool: + """Return True when Python is running translated under Rosetta on Apple Silicon.""" + if platform.system() != "Darwin": + return False + if platform.machine() == "arm64": + return False + # x86_64 on Darwin — could be Intel Mac or Rosetta. Probe sysctl. + out = _run(["sysctl", "-in", "sysctl.proc_translated"]).strip() + return out == "1" + + +def detect_nvidia() -> dict | None: + """Detect NVIDIA GPUs. Returns the GPU with the most VRAM, plus list of all.""" + if not shutil.which("nvidia-smi"): + return None + out = _run([ + "nvidia-smi", + "--query-gpu=index,name,memory.total,driver_version", + "--format=csv,noheader,nounits", + ]) + if not out.strip(): + return None + gpus = [] + for line in out.strip().splitlines(): + parts = [p.strip() for p in line.split(",")] + if len(parts) < 3: + continue + try: + idx = int(parts[0]) + name = parts[1] + vram_mb = int(parts[2]) + except ValueError: + continue + driver = parts[3] if len(parts) > 3 else "" + gpus.append({ + "vendor": "nvidia", + "index": idx, + "name": name, + "vram_gb": round(vram_mb / 1024, 1), + "driver": driver, + }) + if not gpus: + return None + # Pick GPU with most VRAM + best = max(gpus, key=lambda g: g["vram_gb"]) + if len(gpus) > 1: + best["all_gpus"] = gpus + return best + + +def detect_rocm() -> dict | None: + if not shutil.which("rocm-smi"): + return None + # Prefer JSON output (new ROCm 6.x) + out = _run(["rocm-smi", "--showproductname", "--showmeminfo", "vram", "--json"]) + if out.strip().startswith("{"): + try: + data = json.loads(out) + cards = [] + for card_id, info in data.items(): + if not card_id.startswith("card"): + continue + name = (info.get("Card series") or info.get("Card model") + or info.get("Marketing Name") or "AMD GPU") + vram_b = info.get("VRAM Total Memory (B)") or info.get("vram_total_memory_b") or 0 + try: + vram_b = int(vram_b) + except (ValueError, TypeError): + vram_b = 0 + cards.append({ + "vendor": "amd", + "name": str(name).strip(), + "vram_gb": round(vram_b / (1024**3), 1), + "driver": "rocm", + }) + if cards: + best = max(cards, key=lambda c: c["vram_gb"]) + if len(cards) > 1: + best["all_gpus"] = cards + return best + except json.JSONDecodeError: + pass + # Fall back to text parsing + out = _run(["rocm-smi", "--showproductname", "--showmeminfo", "vram"]) + if not out.strip(): + return None + name_m = re.search(r"Card (?:series|model|Marketing Name):\s*(.+)", out) + vram_m = re.search(r"VRAM Total Memory \(B\):\s*(\d+)", out) + vram_gb = round(int(vram_m.group(1)) / (1024**3), 1) if vram_m else 0.0 + return { + "vendor": "amd", + "name": name_m.group(1).strip() if name_m else "AMD GPU", + "vram_gb": vram_gb, + "driver": "rocm", + } + + +def detect_apple_silicon() -> dict | None: + if platform.system() != "Darwin": + return None + if platform.machine() != "arm64": + return None + chip = _run(["sysctl", "-n", "machdep.cpu.brand_string"]).strip() + m = re.search(r"Apple M(\d+)", chip) + generation = int(m.group(1)) if m else None + mem_bytes = 0 + try: + mem_bytes = int(_run(["sysctl", "-n", "hw.memsize"]).strip() or 0) + except ValueError: + pass + ram_gb = round(mem_bytes / (1024**3), 1) if mem_bytes else 0.0 + + # Detect chip variant ("Pro", "Max", "Ultra") — affects performance even at same gen + variant = None + for v in ("Ultra", "Max", "Pro"): + if v in chip: + variant = v + break + + return { + "vendor": "apple", + "name": chip or "Apple Silicon", + "generation": generation, + "variant": variant, + "unified_memory_gb": ram_gb, + } + + +def detect_intel_arc() -> dict | None: + if platform.system() not in {"Linux", "Windows"}: + return None + if shutil.which("clinfo"): + out = _run(["clinfo", "--list"]) + if "Intel" in out and ("Arc" in out or "Xe" in out): + return {"vendor": "intel", "name": "Intel Arc/Xe", "vram_gb": 0.0} + # Windows: try Get-CimInstance + if platform.system() == "Windows" and shutil.which("powershell"): + out = _run(["powershell", "-NoProfile", + "Get-CimInstance Win32_VideoController | Select-Object Name | Format-List"]) + if "Intel" in out and ("Arc" in out or "Iris Xe" in out): + return {"vendor": "intel", "name": "Intel Arc/Iris Xe", "vram_gb": 0.0} + return None + + +def total_system_ram_gb() -> float: + sysname = platform.system() + if sysname == "Darwin": + try: + return round(int(_run(["sysctl", "-n", "hw.memsize"]).strip() or 0) / (1024**3), 1) + except ValueError: + return 0.0 + if sysname == "Linux": + try: + with open("/proc/meminfo", "r") as fh: + for line in fh: + if line.startswith("MemTotal:"): + kb = int(line.split()[1]) + return round(kb / (1024**2), 1) + except OSError: + return 0.0 + if sysname == "Windows": + if shutil.which("powershell"): + out = _run([ + "powershell", "-NoProfile", + "(Get-CimInstance Win32_ComputerSystem).TotalPhysicalMemory", + ]) + m = re.search(r"(\d{8,})", out) + if m: + return round(int(m.group(1)) / (1024**3), 1) + # Fall back to wmic for older Windows + out = _run(["wmic", "ComputerSystem", "get", "TotalPhysicalMemory"]) + m = re.search(r"(\d{6,})", out) + if m: + return round(int(m.group(1)) / (1024**3), 1) + return 0.0 + + +def total_free_disk_gb(path: str = ".") -> float: + try: + usage = shutil.disk_usage(path) + return round(usage.free / (1024**3), 1) + except OSError: + return 0.0 + + +def check_pytorch_cuda() -> dict | None: + """Optional PyTorch availability check. Only run when --check-pytorch is set.""" + try: + import torch # type: ignore[import-not-found] + except Exception as e: + return {"available": False, "reason": f"torch not importable: {e}"} + info: dict[str, Any] = { + "available": True, + "torch_version": torch.__version__, + } + try: + info["cuda_available"] = bool(torch.cuda.is_available()) + if info["cuda_available"]: + info["cuda_device_count"] = torch.cuda.device_count() + info["cuda_device_0"] = torch.cuda.get_device_name(0) + except Exception: + info["cuda_available"] = False + try: + info["mps_available"] = bool(torch.backends.mps.is_available()) + except Exception: + info["mps_available"] = False + return info + + +def classify(gpu: dict | None, ram_gb: float, free_disk_gb: float, *, wsl: bool, rosetta: bool) -> tuple[str, str, list[str]]: + notes: list[str] = [] + + if rosetta: + notes.append( + "Detected Python running under Rosetta on Apple Silicon. " + "ComfyUI MPS support requires native ARM64 Python — install via " + "`brew install python` or arm64 Miniforge, then re-run." + ) + return "cloud", "comfy-cloud", notes + + if wsl and gpu and gpu["vendor"] == "nvidia": + notes.append("Detected WSL2 + NVIDIA — confirm `nvidia-smi` works in your WSL distro before installing.") + + if free_disk_gb and free_disk_gb < MIN_FREE_DISK_GB: + notes.append( + f"Free disk space ({free_disk_gb} GB) is below the {MIN_FREE_DISK_GB} GB recommended minimum. " + "ComfyUI core (~5 GB) plus one SDXL model (~6.5 GB) needs space; Flux Dev needs ~24 GB." + ) + + # Host RAM matters even for discrete-GPU systems: ComfyUI swaps model + # weights through CPU RAM when shuffling between text encoders / VAE / UNet. + # Apple's unified-memory check is handled below so don't double-warn. + if ram_gb and ram_gb < 8 and gpu and gpu.get("vendor") != "apple": + notes.append( + f"System RAM ({ram_gb} GB) is low. ComfyUI swaps model weights through " + "host RAM; <8 GB causes severe slowdowns. 16+ GB recommended." + ) + + if gpu is None: + notes.append( + "No supported accelerator found (NVIDIA CUDA / AMD ROCm / Apple Silicon / Intel Arc)." + ) + notes.append( + "CPU-only ComfyUI works but is unusably slow for modern models — use Comfy Cloud." + ) + return "cloud", "comfy-cloud", notes + + if gpu["vendor"] == "apple": + gen = gpu.get("generation") + variant = gpu.get("variant") + mem = gpu.get("unified_memory_gb", 0.0) + gen_str = f"M{gen}" if gen else "Apple Silicon" + if variant: + gen_str += f" {variant}" + if mem < MIN_MAC_RAM_GB: + notes.append( + f"{gen_str} with {mem} GB unified memory — below the {MIN_MAC_RAM_GB} GB practical minimum." + ) + notes.append("SD1.5 may work; SDXL/Flux will swap or OOM. Recommend Comfy Cloud.") + return "cloud", "comfy-cloud", notes + if mem < OK_MAC_RAM_GB: + notes.append( + f"{gen_str} with {mem} GB — SDXL works but slow. Flux/video likely too tight." + ) + return "marginal", "apple-silicon", notes + notes.append(f"{gen_str} with {mem} GB unified memory — good for SDXL/Flux.") + return "ok", "apple-silicon", notes + + if gpu["vendor"] == "intel": + notes.append("Intel Arc detected — ComfyUI IPEX support is experimental; Comfy Cloud is more reliable.") + return "marginal", "intel", notes + + # Discrete NVIDIA / AMD + vram = gpu.get("vram_gb", 0.0) + name = gpu["name"] + if vram < MIN_VRAM_GB_USABLE: + notes.append( + f"{name} has only {vram} GB VRAM — below the {MIN_VRAM_GB_USABLE} GB practical minimum." + ) + notes.append("Most modern models won't load. Recommend Comfy Cloud.") + return "cloud", "comfy-cloud", notes + if vram < OK_VRAM_GB: + notes.append( + f"{name} ({vram} GB VRAM) — SD1.5 works, SDXL tight, Flux/video unlikely." + ) + return "marginal", gpu["vendor"], notes + if vram < GREAT_VRAM_GB: + notes.append(f"{name} ({vram} GB VRAM) — SDXL comfortable, Flux possible with optimizations.") + return "ok", gpu["vendor"], notes + notes.append(f"{name} ({vram} GB VRAM) — can run everything including Flux/video.") + return "ok", gpu["vendor"], notes + + +def build_report(*, check_pytorch: bool = False) -> dict: + sysname = platform.system() + arch = platform.machine() + ram_gb = total_system_ram_gb() + free_disk_gb = total_free_disk_gb(os.path.expanduser("~")) + + rosetta = is_rosetta() + wsl = is_wsl() + + gpu = ( + detect_nvidia() + or detect_rocm() + or detect_apple_silicon() + or detect_intel_arc() + ) + + # Intel Mac: arm64 detect failed AND no other GPU paths + if gpu is None and sysname == "Darwin" and arch != "arm64" and not rosetta: + notes = [ + "Intel Mac detected — no MPS backend available.", + "ComfyUI will fall back to CPU which is unusably slow. Use Comfy Cloud.", + ] + report = { + "os": sysname, + "arch": arch, + "system_ram_gb": ram_gb, + "free_disk_gb": free_disk_gb, + "wsl": False, + "rosetta": False, + "gpu": None, + "verdict": "cloud", + "recommended_install_path": "comfy-cloud", + "comfy_cli_flag": None, + "notes": notes, + "install_urls": _install_urls(), + } + if check_pytorch: + report["pytorch"] = check_pytorch_cuda() + return report + + verdict, install_path, notes = classify( + gpu, ram_gb, free_disk_gb, wsl=wsl, rosetta=rosetta, + ) + + report = { + "os": sysname, + "arch": arch, + "system_ram_gb": ram_gb, + "free_disk_gb": free_disk_gb, + "wsl": wsl, + "rosetta": rosetta, + "gpu": gpu, + "verdict": verdict, + "recommended_install_path": install_path, + "comfy_cli_flag": _COMFY_CLI_FLAG.get(install_path), + "notes": notes, + "install_urls": _install_urls(), + } + if check_pytorch: + report["pytorch"] = check_pytorch_cuda() + return report + + +def _install_urls() -> dict: + return { + "desktop": "https://docs.comfy.org/installation/desktop", + "manual": "https://docs.comfy.org/installation/manual_install", + "comfy_cli": "https://docs.comfy.org/comfy-cli/getting-started", + "cloud": "https://platform.comfy.org", + } + + +def main(argv: list[str] | None = None) -> int: + import argparse + p = argparse.ArgumentParser(description="Check whether this machine can run ComfyUI locally.") + p.add_argument("--json", action="store_true", help="Emit machine-readable JSON only") + p.add_argument("--check-pytorch", action="store_true", + help="Also probe `torch` for CUDA/MPS availability (slower)") + args = p.parse_args(argv) + + report = build_report(check_pytorch=args.check_pytorch) + + if args.json: + print(json.dumps(report, indent=2)) + else: + print(f"OS: {report['os']} ({report['arch']})") + if report.get("wsl"): + print("Env: WSL2") + if report.get("rosetta"): + print("Env: Rosetta (x86_64 Python on Apple Silicon)") + print(f"RAM: {report['system_ram_gb']} GB") + print(f"Free disk: {report['free_disk_gb']} GB (~/)") + if report["gpu"]: + g = report["gpu"] + if g["vendor"] == "apple": + print(f"GPU: {g['name']} — {g.get('unified_memory_gb', 0)} GB unified memory") + else: + print(f"GPU: {g['name']} — {g.get('vram_gb', 0)} GB VRAM") + if g.get("all_gpus") and len(g["all_gpus"]) > 1: + print(f" ({len(g['all_gpus'])} GPUs total; using best by VRAM)") + else: + print("GPU: (none detected)") + print(f"Verdict: {report['verdict']} → {report['recommended_install_path']}") + if report["comfy_cli_flag"]: + print(f" run: comfy --skip-prompt install {report['comfy_cli_flag']}") + if report.get("pytorch"): + pt = report["pytorch"] + if pt.get("available"): + line = f"PyTorch: {pt.get('torch_version')}" + if pt.get("cuda_available"): + line += f" + CUDA ({pt.get('cuda_device_0', '?')})" + if pt.get("mps_available"): + line += " + MPS" + print(line) + else: + print(f"PyTorch: not available — {pt.get('reason')}") + for n in report["notes"]: + print(f" • {n}") + + if report["verdict"] == "ok": + return 0 + if report["verdict"] == "marginal": + return 1 + return 2 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/image-creation/comfyui/scripts/health_check.py b/image-creation/comfyui/scripts/health_check.py new file mode 100644 index 0000000..63c5025 --- /dev/null +++ b/image-creation/comfyui/scripts/health_check.py @@ -0,0 +1,223 @@ +#!/usr/bin/env python3 +""" +health_check.py — One-stop verification that the ComfyUI environment is ready. + +Runs through the verification checklist: + 1. comfy-cli on PATH + 2. server reachable (/system_stats) + 3. at least one checkpoint installed + 4. (optional) a specific workflow's deps are met + 5. (optional) actually submit a tiny test workflow and verify round-trip + +Usage: + python3 health_check.py + python3 health_check.py --host https://cloud.comfy.org + python3 health_check.py --workflow my.json + python3 health_check.py --smoke-test # actually submit a tiny workflow +""" + +from __future__ import annotations + +import argparse +import json +import shutil +import sys +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from _common import ( # noqa: E402 + DEFAULT_LOCAL_HOST, ENV_API_KEY, emit_json, http_get, parse_model_list, + resolve_api_key, resolve_url, unwrap_workflow, +) + + +def comfy_cli_status() -> dict: + if shutil.which("comfy"): + return {"available": True, "method": "comfy", "path": shutil.which("comfy")} + if shutil.which("uvx"): + return {"available": True, "method": "uvx", + "hint": "Invoke as `uvx --from comfy-cli comfy ...`"} + return { + "available": False, + "hint": "Install with: pipx install comfy-cli (or `pip install comfy-cli`)", + } + + +def server_status(host: str, headers: dict) -> dict: + url = resolve_url(host, "/system_stats") + try: + r = http_get(url, headers=headers, retries=2, timeout=10) + if r.status == 200: + try: + stats = r.json() or {} + except Exception: + stats = {} + return {"reachable": True, "url": url, "stats": stats} + return {"reachable": False, "url": url, "http_status": r.status, "body": r.text()[:200]} + except Exception as e: + return {"reachable": False, "url": url, "error": str(e)} + + +def checkpoint_status(host: str, headers: dict) -> dict: + url = resolve_url(host, "/models/checkpoints") + try: + r = http_get(url, headers=headers, retries=2, timeout=15) + except Exception as e: + return {"queryable": False, "error": str(e)} + if r.status != 200: + return {"queryable": False, "http_status": r.status, "url": url, "body": r.text()[:200]} + try: + models = parse_model_list(r.json()) + except Exception: + models = set() + return {"queryable": True, "count": len(models), + "first_few": sorted(models)[:5]} + + +SMOKE_WORKFLOW = { + # Minimal SD1.5 workflow that doesn't depend on rare nodes. + # 256x256 + 1 step is the smallest config that doesn't trigger SDXL/Flux + # validation errors while still executing fast. + "3": { + "class_type": "KSampler", + "inputs": { + "seed": 1, "steps": 1, "cfg": 7.0, + "sampler_name": "euler", "scheduler": "normal", "denoise": 1.0, + "model": ["4", 0], "positive": ["6", 0], "negative": ["7", 0], + "latent_image": ["5", 0], + }, + }, + "4": {"class_type": "CheckpointLoaderSimple", + "inputs": {"ckpt_name": "REPLACE_ME"}}, + "5": {"class_type": "EmptyLatentImage", + "inputs": {"width": 256, "height": 256, "batch_size": 1}}, + "6": {"class_type": "CLIPTextEncode", + "inputs": {"text": "test", "clip": ["4", 1]}}, + "7": {"class_type": "CLIPTextEncode", + "inputs": {"text": "", "clip": ["4", 1]}}, + "9": {"class_type": "SaveImage", + "inputs": {"filename_prefix": "smoke", "images": ["3", 0]}}, +} + + +def smoke_test(host: str, headers: dict, ckpt_name: str | None) -> dict: + """Submit a tiny workflow and verify the server accepts it. + + Cancels the job immediately after acceptance so we don't burn GPU + time / cloud minutes on a smoke test. + """ + if not ckpt_name: + return {"ran": False, "reason": "no checkpoint available"} + wf = json.loads(json.dumps(SMOKE_WORKFLOW)) + wf["4"]["inputs"]["ckpt_name"] = ckpt_name + + # Lazy import to avoid circular issues + from run_workflow import ComfyRunner + api_key = headers.get("X-API-Key") + runner = ComfyRunner(host=host, api_key=api_key) + sub = runner.submit(wf) + if "_http_error" in sub: + return {"ran": True, "submitted": False, + "http_status": sub["_http_error"], "body": sub.get("body")} + pid = sub.get("prompt_id") + if not pid: + return {"ran": True, "submitted": False, "response": sub} + + # Cancel so we don't actually waste compute on the smoke test. + cancelled = False + try: + cancelled = runner.cancel(pid) + except Exception: + pass + + return { + "ran": True, "submitted": True, "prompt_id": pid, + "cancelled_after_submit": cancelled, + "note": "Submission accepted; cancelled to avoid running the full pipeline.", + } + + +def main(argv: list[str] | None = None) -> int: + p = argparse.ArgumentParser(description="One-stop ComfyUI health check") + p.add_argument("--host", default=DEFAULT_LOCAL_HOST) + p.add_argument("--api-key", help=f"or set ${ENV_API_KEY}") + p.add_argument("--workflow", help="Optional: also run check_deps on this workflow") + p.add_argument("--smoke-test", action="store_true", + help="Submit a tiny test workflow and verify round-trip") + p.add_argument("--strict", action="store_true", + help="Exit non-zero on any non-pass condition (including warnings)") + args = p.parse_args(argv) + + api_key = resolve_api_key(args.api_key) + headers = {"X-API-Key": api_key} if api_key else {} + + cli = comfy_cli_status() + server = server_status(args.host, headers) + ckpts = checkpoint_status(args.host, headers) if server.get("reachable") else None + + # ---- workflow check ---- + workflow_check: dict | None = None + if args.workflow: + wf_path = Path(args.workflow).expanduser() + if not wf_path.exists(): + workflow_check = {"error": "workflow file not found"} + else: + try: + with wf_path.open() as f: + workflow = unwrap_workflow(json.load(f)) + from check_deps import check_deps + workflow_check = check_deps(workflow, host=args.host, api_key=api_key) + except (ValueError, json.JSONDecodeError) as e: + workflow_check = {"error": str(e)} + + smoke = None + if args.smoke_test and server.get("reachable"): + first_ckpt = ckpts["first_few"][0] if ckpts and ckpts.get("first_few") else None + smoke = smoke_test(args.host, headers, first_ckpt) + + # ---- verdict ---- + verdict = "pass" + reasons: list[str] = [] + if not server.get("reachable"): + verdict = "fail" + reasons.append("server unreachable") + if ckpts and ckpts.get("queryable") and ckpts.get("count", 0) == 0: + verdict = "warn" if verdict == "pass" else verdict + reasons.append("no checkpoints installed") + if workflow_check and workflow_check.get("error"): + verdict = "fail" + reasons.append(f"workflow check failed: {workflow_check['error']}") + elif workflow_check and not workflow_check.get("is_ready"): + if workflow_check.get("node_check_skipped"): + reasons.append("node check skipped (cloud free tier)") + else: + verdict = "fail" + reasons.append("workflow has missing deps") + if smoke and smoke.get("ran") and not smoke.get("submitted"): + verdict = "fail" + reasons.append("smoke-test submission failed") + if not cli.get("available"): + verdict = "warn" if verdict == "pass" else verdict + reasons.append("comfy-cli not on PATH (lifecycle commands won't work)") + + report = { + "verdict": verdict, + "reasons": reasons, + "host": args.host, + "comfy_cli": cli, + "server": server, + "checkpoints": ckpts, + "workflow_check": workflow_check, + "smoke_test": smoke, + } + emit_json(report) + + if verdict == "pass": + return 0 + if verdict == "warn": + return 1 if args.strict else 0 + return 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/image-creation/comfyui/scripts/run_batch.py b/image-creation/comfyui/scripts/run_batch.py new file mode 100644 index 0000000..7f5b159 --- /dev/null +++ b/image-creation/comfyui/scripts/run_batch.py @@ -0,0 +1,243 @@ +#!/usr/bin/env python3 +""" +run_batch.py — Run a workflow many times, varying parameters per run. + +Two modes: + 1. --count N --randomize-seed + Submit N runs, each with a fresh random seed. Use for quick variations. + 2. --sweep '{"seed": [1,2,3], "steps": [20,30]}' + Cartesian product of values. With cloud subscription, runs in parallel + up to your tier's concurrent-job limit. + +Both modes write each run's outputs into output-dir/run_NNN/. + +Examples: + python3 run_batch.py --workflow flux_dev.json \ + --args '{"prompt": "a cat"}' \ + --count 8 --randomize-seed \ + --output-dir ./outputs/cat-batch + + python3 run_batch.py --workflow sdxl.json \ + --args '{"prompt": "abstract"}' \ + --sweep '{"seed": [1,2,3], "steps": [20, 40]}' \ + --output-dir ./outputs/sweep +""" + +from __future__ import annotations + +import argparse +import itertools +import json +import sys +from concurrent.futures import ThreadPoolExecutor, as_completed +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from _common import ( # noqa: E402 + DEFAULT_LOCAL_HOST, ENV_API_KEY, coerce_seed, emit_json, log, + looks_like_video_workflow, resolve_api_key, unwrap_workflow, +) +from run_workflow import ( # noqa: E402 + ComfyRunner, download_outputs, inject_params, +) +from extract_schema import extract_schema # noqa: E402 + + +def expand_sweep(sweep: dict, base_args: dict, count: int, randomize_seed: bool) -> list[dict]: + """Generate a list of args dicts for each run.""" + if sweep: + # Cartesian product + keys = list(sweep.keys()) + values = [sweep[k] if isinstance(sweep[k], list) else [sweep[k]] for k in keys] + runs = [] + for combo in itertools.product(*values): + ar = dict(base_args) + for k, v in zip(keys, combo): + ar[k] = v + runs.append(ar) + return runs + # Count mode + runs = [] + for _ in range(count): + ar = dict(base_args) + if randomize_seed: + ar["seed"] = coerce_seed(None) + runs.append(ar) + return runs + + +def execute_one( + runner: ComfyRunner, workflow: dict, schema: dict, args: dict, + *, output_dir: Path, timeout: int, ws: bool, +) -> dict: + wf, warnings = inject_params(workflow, schema, args) + sub = runner.submit(wf) + if "_http_error" in sub: + return {"status": "error", "error": "submission HTTP error", + "details": sub.get("body"), "args": args} + pid = sub.get("prompt_id") + if not pid: + return {"status": "error", "error": "no prompt_id", "response": sub, "args": args} + if sub.get("node_errors"): + return {"status": "error", "error": "validation failed", + "node_errors": sub["node_errors"], "args": args} + + if ws: + result = runner.monitor_ws(pid, timeout=timeout) + else: + result = runner.poll_status(pid, timeout=timeout) + + if result["status"] != "success": + return { + "status": result["status"], + "prompt_id": pid, + "details": result.get("data"), + "args": args, + } + + outputs = result.get("outputs") or runner.get_outputs(pid) + downloaded = download_outputs(runner, outputs, output_dir, preserve_subfolder=False) + return { + "status": "success", + "prompt_id": pid, + "args": args, + "outputs": downloaded, + "warnings": warnings, + } + + +def main(argv: list[str] | None = None) -> int: + p = argparse.ArgumentParser( + description="Submit a workflow many times with varying parameters.", + ) + p.add_argument("--workflow", required=True) + p.add_argument("--args", default="{}", help="Base parameters JSON") + p.add_argument("--count", type=int, default=0, + help="Number of runs (use with --randomize-seed)") + p.add_argument("--sweep", default="", + help='JSON dict of param→list of values. Cartesian product. ' + 'e.g. \'{"seed":[1,2,3],"cfg":[5,8]}\'') + p.add_argument("--randomize-seed", action="store_true", + help="In --count mode, vary seed per run") + p.add_argument("--host", default=DEFAULT_LOCAL_HOST) + p.add_argument("--api-key", help=f"or set ${ENV_API_KEY}") + p.add_argument("--partner-key") + p.add_argument("--parallel", type=int, default=1, + help="Concurrent submissions (cloud: up to your tier limit). " + "Default 1 (sequential)") + p.add_argument("--output-dir", default="./outputs/batch") + p.add_argument("--timeout", type=int, default=0) + p.add_argument("--ws", action="store_true") + p.add_argument("--continue-on-error", action="store_true", + help="Don't stop the batch when a run fails") + args = p.parse_args(argv) + + if args.count <= 0 and not args.sweep: + emit_json({"error": "Specify --count N or --sweep '{...}'"}) + return 1 + + base_args = json.loads(args.args) if args.args.strip() else {} + sweep = json.loads(args.sweep) if args.sweep.strip() else {} + + # Validate sweep shape + if sweep: + if not isinstance(sweep, dict): + emit_json({"error": "--sweep must be a JSON object {param: [values]}"}) + return 1 + empty = [k for k, v in sweep.items() if isinstance(v, list) and len(v) == 0] + if empty: + emit_json({"error": f"--sweep parameters have empty value lists: {empty}"}) + return 1 + # If user passed BOTH --sweep and --count/--randomize-seed, --sweep wins + if args.count or args.randomize_seed: + log("--sweep set; ignoring --count / --randomize-seed (sweep defines the runs)") + + wf_path = Path(args.workflow).expanduser() + if not wf_path.exists(): + emit_json({"error": f"Workflow not found: {args.workflow}"}) + return 1 + try: + with wf_path.open() as f: + workflow = unwrap_workflow(json.load(f)) + except (ValueError, json.JSONDecodeError) as e: + emit_json({"error": str(e)}) + return 1 + + schema = extract_schema(workflow) + runs = expand_sweep(sweep, base_args, args.count, args.randomize_seed) + log(f"Planned {len(runs)} run(s)") + + api_key = resolve_api_key(args.api_key) + runner = ComfyRunner(host=args.host, api_key=api_key, partner_key=args.partner_key) + + ok, info = runner.check_server() + if not ok: + emit_json({"error": "Cannot reach server", "details": info, "host": args.host}) + return 1 + + timeout = args.timeout + if timeout <= 0: + timeout = 900 if looks_like_video_workflow(workflow) else 300 + + base_dir = Path(args.output_dir).expanduser() + base_dir.mkdir(parents=True, exist_ok=True) + + results: list[dict] = [] + failures = 0 + + if args.parallel > 1: + with ThreadPoolExecutor(max_workers=args.parallel) as ex: + future_to_idx = {} + for i, ar in enumerate(runs): + run_dir = base_dir / f"run_{i:04d}" + fut = ex.submit( + execute_one, runner, workflow, schema, ar, + output_dir=run_dir, timeout=timeout, ws=args.ws, + ) + future_to_idx[fut] = i + for fut in as_completed(future_to_idx): + i = future_to_idx[fut] + try: + r = fut.result() + except Exception as e: + r = {"status": "error", "error": str(e), "args": runs[i]} + r["index"] = i + results.append(r) + if r["status"] != "success": + failures += 1 + log(f" run {i} → {r['status']}: {r.get('error','?')}") + if not args.continue_on_error: + log(" --continue-on-error not set; aborting batch") + break + else: + log(f" run {i} → success: {len(r.get('outputs', []))} files") + else: + for i, ar in enumerate(runs): + run_dir = base_dir / f"run_{i:04d}" + r = execute_one(runner, workflow, schema, ar, + output_dir=run_dir, timeout=timeout, ws=args.ws) + r["index"] = i + results.append(r) + if r["status"] != "success": + failures += 1 + log(f" run {i} → {r['status']}: {r.get('error','?')}") + if not args.continue_on_error: + log(" --continue-on-error not set; aborting batch") + break + else: + log(f" run {i} → success: {len(r.get('outputs', []))} files") + + results.sort(key=lambda x: x.get("index", 0)) + emit_json({ + "status": "success" if failures == 0 else "partial", + "total": len(runs), + "completed": sum(1 for r in results if r["status"] == "success"), + "failed": failures, + "output_dir": str(base_dir), + "results": results, + }) + return 0 if failures == 0 else 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/image-creation/comfyui/scripts/run_workflow.py b/image-creation/comfyui/scripts/run_workflow.py new file mode 100644 index 0000000..05afb1e --- /dev/null +++ b/image-creation/comfyui/scripts/run_workflow.py @@ -0,0 +1,796 @@ +#!/usr/bin/env python3 +""" +run_workflow.py — Inject parameters into a ComfyUI workflow, submit it, monitor +execution, and download outputs. + +Improvements over v1: + - Cloud-aware URL routing (handles /api prefix and /history_v2 / /experiment/models renames) + - API key from CLI flag OR $COMFY_CLOUD_API_KEY env var + - WebSocket progress monitoring (--ws), with HTTP polling fallback + - Streaming download (no whole-file buffering — handles GB-size video outputs) + - Path-traversal-safe output writes + - Subfolder-aware download paths (no silent overwrites) + - Retry with exponential backoff on transient errors + - Status-error correctly classified before "completed: true" + - Image upload helper (--input-image NAME=PATH) + - Auto-randomize seed when value is -1 or omitted on a randomize-seed flag + - Auto-extends timeout heuristically for video workflows + - Editor-format detection with helpful error + - Doesn't pollute extra_data.api_key_comfy_org with the cloud auth key + unless --partner-key is provided (correct semantic per cloud docs) + +Usage: + # Local server + python3 run_workflow.py --workflow workflow_api.json \ + --args '{"prompt": "a cat", "seed": 42}' \ + --output-dir ./outputs + + # Cloud server (API key from env var) + export COMFY_CLOUD_API_KEY="comfyui-xxxxxxx" + python3 run_workflow.py --workflow workflow_api.json \ + --args '{"prompt": "a cat"}' \ + --host https://cloud.comfy.org \ + --output-dir ./outputs + + # With image input (auto-uploads, then references) + python3 run_workflow.py --workflow img2img.json \ + --input-image image=./photo.png \ + --args '{"prompt": "make it cyberpunk"}' + + # WebSocket real-time progress + python3 run_workflow.py --workflow flux_dev.json \ + --args '{"prompt": "..."}' \ + --ws + +Stdlib-only by default (Python 3.10+). Will use `requests`/`websocket-client` +if installed for nicer behavior. +""" + +from __future__ import annotations + +import argparse +import copy +import json +import sys +import time +from pathlib import Path +from typing import Any +from urllib.parse import urlencode, urlparse + +# Local import — _common.py sits next to this script. +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from _common import ( # noqa: E402 + DEFAULT_LOCAL_HOST, ENV_API_KEY, + coerce_seed, emit_json, http_get, http_post, http_request, + is_cloud_host, is_link, log, looks_like_video_workflow, + media_type_from_filename, new_client_id, resolve_api_key, resolve_url, + safe_path_join, unwrap_workflow, +) + + +# ============================================================================= +# Runner +# ============================================================================= + +class WorkflowRunError(Exception): + """Raised when a workflow run fails (validation, execution, timeout).""" + + def __init__(self, status: str, message: str, **details: Any): + super().__init__(message) + self.status = status + self.message = message + self.details = details + + def to_dict(self) -> dict: + d = {"status": self.status, "error": self.message} + d.update(self.details) + return d + + +class ComfyRunner: + def __init__( + self, + host: str = DEFAULT_LOCAL_HOST, + api_key: str | None = None, + client_id: str | None = None, + partner_key: str | None = None, + ): + self.host = host.rstrip("/") + self.api_key = api_key + self.partner_key = partner_key + self.is_cloud = is_cloud_host(self.host) + self.client_id = client_id or new_client_id() + + @property + def headers(self) -> dict[str, str]: + h: dict[str, str] = {} + if self.api_key: + h["X-API-Key"] = self.api_key + return h + + def _url(self, path: str) -> str: + return resolve_url(self.host, path, is_cloud=self.is_cloud) + + # ---------- server health ---------- + def check_server(self) -> tuple[bool, dict | None]: + try: + r = http_get(self._url("/system_stats"), headers=self.headers, retries=2) + if r.status == 200: + try: + return True, r.json() + except Exception: + return True, None + return False, {"http_status": r.status, "body": r.text()[:500]} + except Exception as e: + return False, {"error": str(e)} + + # ---------- upload ---------- + def upload_image(self, path: Path, *, image_type: str = "input", overwrite: bool = True, + endpoint: str = "/upload/image", extra_form: dict | None = None) -> dict: + """Upload an image file via multipart. Returns server-side ref dict.""" + if not path.exists(): + raise FileNotFoundError(f"input image not found: {path}") + # Stream the file via a handle to avoid OOM on huge inputs (16MP+ photos). + with path.open("rb") as fh: + files = {"image": (path.name, fh)} + form = {"type": image_type} + if overwrite: + form["overwrite"] = "true" + if extra_form: + form.update({k: str(v) for k, v in extra_form.items()}) + r = http_request( + "POST", self._url(endpoint), + headers=self.headers, files=files, form=form, + timeout=300, retries=2, + ) + if r.status != 200: + raise WorkflowRunError( + "upload_failed", + f"Upload of {path.name} failed: HTTP {r.status}", + body=r.text()[:500], + ) + try: + return r.json() + except Exception: + return {"name": path.name} + + def upload_mask(self, path: Path, original_ref: dict) -> dict: + """Upload an inpaint mask, linked to a previously uploaded source image. + + `original_ref` should be the dict returned by `upload_image()` for the + source image (or `{"filename": ..., "subfolder": ..., "type": "input"}`). + """ + return self.upload_image( + path, + endpoint="/upload/mask", + extra_form={ + "subfolder": "clipspace", + "original_ref": json.dumps(original_ref), + }, + ) + + # ---------- submit ---------- + def submit(self, workflow: dict) -> dict: + payload: dict[str, Any] = {"prompt": workflow, "client_id": self.client_id} + if self.partner_key: + payload["extra_data"] = {"api_key_comfy_org": self.partner_key} + + r = http_post(self._url("/prompt"), headers=self.headers, json_body=payload, timeout=120) + try: + body = r.json() + except Exception: + body = {"raw": r.text()[:500]} + if r.status != 200: + return {"_http_error": r.status, "body": body} + return body + + # ---------- HTTP polling ---------- + def poll_status(self, prompt_id: str, *, timeout: float = 300.0, + initial_interval: float = 1.5, max_interval: float = 8.0) -> dict: + start = time.time() + interval = initial_interval + + while time.time() - start < timeout: + if self.is_cloud: + r = http_get( + self._url(f"/job/{prompt_id}/status"), + headers=self.headers, retries=2, timeout=30, + ) + if r.status == 200: + try: + data = r.json() + except Exception: + data = {} + s = data.get("status") + if s == "completed": + return {"status": "success", "data": data} + if s in {"failed",}: + return {"status": "error", "data": data} + if s == "cancelled": + return {"status": "cancelled", "data": data} + # pending / in_progress → continue + elif r.status == 404: + # Cloud sometimes 404s briefly between submit and dispatcher pickup + pass + else: + # transient error — retry loop covers it + pass + else: + # Local: /history/{id} grows once execution completes + r = http_get( + self._url(f"/history/{prompt_id}"), + headers=self.headers, retries=2, timeout=30, + ) + if r.status == 200: + try: + data = r.json() or {} + except Exception: + data = {} + entry = data.get(prompt_id) + if isinstance(entry, dict): + st = entry.get("status") or {} + # IMPORTANT: check error first — `completed: true` can coexist with errors + status_str = st.get("status_str") + if status_str == "error": + return {"status": "error", "data": entry} + if st.get("completed", False): + return {"status": "success", "outputs": entry.get("outputs", {})} + # not in history yet → continue polling + + time.sleep(interval) + interval = min(max_interval, interval * 1.4) + + return {"status": "timeout", "elapsed": time.time() - start} + + # ---------- WebSocket monitoring ---------- + def monitor_ws(self, prompt_id: str, *, timeout: float = 300.0, + on_progress: Any = None) -> dict: + """Connect to /ws and listen until execution_success / execution_error. + + Falls back to HTTP polling if `websocket-client` is not installed. + Returns same shape as poll_status. + """ + try: + import websocket # type: ignore[import-not-found] + except ImportError: + log("websocket-client not installed; falling back to HTTP polling") + return self.poll_status(prompt_id, timeout=timeout) + + # Build WS URL. Preserve any base-path components the user gave us + # (e.g. http://example.com/comfyui → ws://example.com/comfyui/ws). + parsed = urlparse(self.host) + scheme = "wss" if parsed.scheme == "https" else "ws" + netloc = parsed.netloc + base_path = parsed.path.rstrip("/") + ws_url = f"{scheme}://{netloc}{base_path}/ws?clientId={self.client_id}" + if self.is_cloud and self.api_key: + ws_url += f"&token={self.api_key}" + + outputs: dict[str, Any] = {} + error_payload: dict[str, Any] | None = None + success = False + seen_executed = False + + ws = websocket.create_connection(ws_url, timeout=timeout) + try: + ws.settimeout(timeout) + deadline = time.time() + timeout + while time.time() < deadline: + msg = ws.recv() + if isinstance(msg, bytes): + # Binary preview frame — ignore for now; ws_monitor.py prints them + continue + try: + payload = json.loads(msg) + except Exception: + continue + mtype = payload.get("type", "") + mdata = payload.get("data", {}) or {} + + # Filter to our job (cloud broadcasts; local filters via client_id) + pid = mdata.get("prompt_id") + if pid is not None and pid != prompt_id: + continue + + if mtype == "progress": + if callable(on_progress): + on_progress({ + "type": "progress", + "value": mdata.get("value"), + "max": mdata.get("max"), + "node": mdata.get("node"), + }) + elif mtype == "progress_state": + if callable(on_progress): + on_progress({"type": "progress_state", "nodes": mdata.get("nodes", {})}) + elif mtype == "executing": + node = mdata.get("node") + if callable(on_progress): + on_progress({"type": "executing", "node": node}) + # When `node` is None on a local server, that signals end-of-run + if node is None and not self.is_cloud and seen_executed: + success = True + break + elif mtype == "executed": + seen_executed = True + nid = mdata.get("node") + out = mdata.get("output") or {} + if nid: + outputs[nid] = out + elif mtype == "notification": + if callable(on_progress): + on_progress({"type": "notification", "message": mdata.get("value", "")}) + elif mtype == "execution_success": + success = True + break + elif mtype == "execution_error": + error_payload = mdata + break + elif mtype == "execution_interrupted": + error_payload = {"interrupted": True, **mdata} + break + finally: + try: + ws.close() + except Exception: + pass + + if error_payload is not None: + return {"status": "error", "data": error_payload} + if success: + return {"status": "success", "outputs": outputs} + return {"status": "timeout", "elapsed": timeout} + + # ---------- outputs ---------- + def get_outputs(self, prompt_id: str) -> dict: + if self.is_cloud: + # Try /jobs/{id} first (returns full job with outputs); fall back to /history_v2 + r = http_get(self._url(f"/jobs/{prompt_id}"), headers=self.headers, retries=2) + if r.status == 200: + try: + return (r.json() or {}).get("outputs", {}) or {} + except Exception: + pass + # Fallback + r = http_get(self._url(f"/history/{prompt_id}"), headers=self.headers, retries=2) + if r.status == 200: + try: + body = r.json() or {} + except Exception: + body = {} + if isinstance(body, dict) and prompt_id in body: + return body[prompt_id].get("outputs", {}) or {} + if isinstance(body, dict) and "outputs" in body: + return body["outputs"] or {} + return {} + # Local + r = http_get(self._url(f"/history/{prompt_id}"), headers=self.headers, retries=2) + if r.status != 200: + return {} + try: + body = r.json() or {} + except Exception: + return {} + entry = body.get(prompt_id) or {} + return entry.get("outputs", {}) or {} + + def download_output( + self, *, filename: str, subfolder: str, file_type: str, + output_dir: Path, preserve_subfolder: bool = True, overwrite: bool = False, + ) -> Path: + """Stream a single output to disk. Path-traversal-safe.""" + params = {"filename": filename, "subfolder": subfolder, "type": file_type} + url = self._url("/view") + "?" + urlencode(params) + + # Compute target path safely. If preserve_subfolder, include subfolder in the + # local path; otherwise put the file in output_dir flat. + target_parts: list[str] = [] + if preserve_subfolder and subfolder: + target_parts.extend(p for p in subfolder.split("/") if p and p not in {".", ".."}) + target_parts.append(filename) + out_path = safe_path_join(output_dir, *target_parts) + + if out_path.exists() and not overwrite: + stem, suffix = out_path.stem, out_path.suffix + i = 1 + while True: + candidate = out_path.with_name(f"{stem}_{i}{suffix}") + if not candidate.exists(): + out_path = candidate + break + i += 1 + + out_path.parent.mkdir(parents=True, exist_ok=True) + + # Stream download. Two-step for cloud: get the 302, then fetch signed URL + # so we don't accidentally send X-API-Key to the storage backend. + # The HTTP transport already strips X-API-Key on cross-host redirect + # via _strip_api_key_on_redirect, so a single follow_redirects=True call + # is safe AND simpler. + r = http_request( + "GET", url, headers=self.headers, + timeout=600, retries=3, follow_redirects=True, + stream=True, sink=out_path, + ) + if r.status != 200: + try: + if out_path.exists(): + out_path.unlink() + except Exception: + pass + raise WorkflowRunError( + "download_failed", + f"Download of {filename} failed: HTTP {r.status}", + url=url, + ) + return out_path + + # ---------- queue / cancel ---------- + def cancel(self, prompt_id: str | None = None) -> bool: + if prompt_id: + r = http_post( + self._url("/queue"), headers=self.headers, + json_body={"delete": [prompt_id]}, retries=1, + ) + return r.status == 200 + # Interrupt currently running + r = http_post(self._url("/interrupt"), headers=self.headers, retries=1) + return r.status == 200 + + +# ============================================================================= +# Schema / parameter injection +# ============================================================================= + +def _inline_schema(workflow: dict) -> dict: + """Generate schema using the sibling extract_schema module.""" + from extract_schema import extract_schema # noqa: WPS433 + return extract_schema(workflow) + + +def load_schema(schema_path: str | None, workflow: dict) -> dict: + if schema_path: + with open(schema_path) as f: + return json.load(f) + return _inline_schema(workflow) + + +def inject_params( + workflow: dict, schema: dict, args: dict, + *, randomize_seed_if_unset: bool = False, +) -> tuple[dict, list[str]]: + """Inject user args into the workflow. Returns (new_workflow, warnings).""" + wf = copy.deepcopy(workflow) + params = schema.get("parameters", {}) or {} + warnings: list[str] = [] + + # Auto-randomize seed when it's -1 in args, or when randomize_seed_if_unset + # and user didn't pass a seed. + if "seed" in params: + if "seed" in args and args["seed"] in {None, -1, "-1"}: + args = dict(args) + args["seed"] = coerce_seed(args["seed"]) + warnings.append(f"seed=-1 expanded to {args['seed']}") + elif randomize_seed_if_unset and "seed" not in args: + args = dict(args) + args["seed"] = coerce_seed(None) + warnings.append(f"seed auto-randomized to {args['seed']}") + + for name, value in args.items(): + if name not in params: + warnings.append(f"unknown parameter '{name}' (not in schema), skipping") + continue + m = params[name] + nid, field = m["node_id"], m["field"] + node = wf.get(nid) + if not isinstance(node, dict) or "inputs" not in node: + warnings.append(f"node '{nid}' for parameter '{name}' missing in workflow") + continue + # Refuse to overwrite a link with a literal — would silently break wiring + cur = node["inputs"].get(field) + if is_link(cur): + warnings.append( + f"parameter '{name}' targets {nid}.{field} which is currently a link; " + f"refusing to overwrite (set the schema to point at the source node instead)" + ) + continue + node["inputs"][field] = value + + return wf, warnings + + +# ============================================================================= +# Output download helper +# ============================================================================= + +def download_outputs( + runner: ComfyRunner, outputs: dict, output_dir: Path, + *, preserve_subfolder: bool = True, overwrite: bool = False, +) -> list[dict]: + """Walk the outputs dict and download every file. Cloud uses `video` (singular); + local uses `videos` (plural). We accept both.""" + output_dir.mkdir(parents=True, exist_ok=True) + downloaded: list[dict] = [] + + OUTPUT_KEYS = ("images", "gifs", "videos", "video", "audio", "files", "models", "3d") + + for node_id, node_output in (outputs or {}).items(): + if not isinstance(node_output, dict): + continue + for key in OUTPUT_KEYS: + entries = node_output.get(key) + if not entries: + continue + if not isinstance(entries, list): + entries = [entries] + for fi in entries: + if not isinstance(fi, dict): + continue + filename = fi.get("filename") or "" + if not filename: + continue + subfolder = fi.get("subfolder") or "" + file_type = fi.get("type") or "output" + try: + out_path = runner.download_output( + filename=filename, subfolder=subfolder, file_type=file_type, + output_dir=output_dir, preserve_subfolder=preserve_subfolder, + overwrite=overwrite, + ) + downloaded.append({ + "file": str(out_path), + "node_id": node_id, + "type": media_type_from_filename(filename), + "filename": filename, + "subfolder": subfolder, + "source_type": file_type, + }) + except Exception as e: + log(f"WARN: failed to download {filename}: {e}") + return downloaded + + +# ============================================================================= +# CLI +# ============================================================================= + +def parse_input_image_arg(spec: str) -> tuple[str, Path]: + """Parse `name=path` (or `path` alone, defaulting to name='image').""" + if "=" in spec: + name, path = spec.split("=", 1) + return name.strip(), Path(path).expanduser() + return "image", Path(spec).expanduser() + + +def main(argv: list[str] | None = None) -> int: + p = argparse.ArgumentParser( + description="Run a ComfyUI workflow with parameter injection.", + formatter_class=argparse.RawDescriptionHelpFormatter, + ) + p.add_argument("--workflow", required=True, help="Path to workflow API JSON file") + p.add_argument("--args", default="{}", + help="JSON parameters to inject (or `@/path/to/args.json`)") + p.add_argument("--schema", help="Path to schema JSON (auto-generated if omitted)") + p.add_argument("--host", default=DEFAULT_LOCAL_HOST, help="ComfyUI server URL") + p.add_argument("--api-key", + help=f"API key for cloud (or set ${ENV_API_KEY} env var)") + p.add_argument("--partner-key", + help="Partner-node API key (extra_data.api_key_comfy_org). " + "Required for Flux Pro / Ideogram / etc. Defaults to --api-key if not set.") + p.add_argument("--output-dir", default="./outputs", help="Directory to save outputs") + p.add_argument("--timeout", type=int, default=0, + help="Max seconds to wait (0=auto: 300 / 900 for video workflows)") + p.add_argument("--input-image", action="append", default=[], + help="Upload local image before running. Format: `name=path` or `path`. " + "The `name` becomes the value injected into the matching schema parameter.") + p.add_argument("--randomize-seed", action="store_true", + help="If schema has a 'seed' parameter and --args didn't set one, randomize it") + p.add_argument("--ws", action="store_true", + help="Use WebSocket for real-time progress (requires `websocket-client`)") + p.add_argument("--no-download", action="store_true", help="Skip downloading outputs") + p.add_argument("--flat-output", action="store_true", + help="Don't preserve server-side subfolder structure when saving outputs") + p.add_argument("--overwrite", action="store_true", + help="Overwrite existing files instead of appending _1, _2, ...") + p.add_argument("--submit-only", action="store_true", + help="Submit and return prompt_id without waiting") + p.add_argument("--client-id", help="Override generated client_id (UUID)") + p.add_argument("--use-partner-key-as-auth", action="store_true", + help="(Compat) Use --partner-key value as cloud X-API-Key. Don't use unless you know why.") + + args = p.parse_args(argv) + + # ---- Load workflow ---- + wf_path = Path(args.workflow).expanduser() + if not wf_path.exists(): + emit_json({"error": f"Workflow file not found: {args.workflow}"}) + return 1 + try: + with wf_path.open() as f: + workflow_raw = json.load(f) + workflow = unwrap_workflow(workflow_raw) + except ValueError as e: + emit_json({"error": str(e)}) + return 1 + except json.JSONDecodeError as e: + emit_json({"error": f"Invalid JSON in workflow file: {e}"}) + return 1 + + # ---- Parse user args ---- + args_str = args.args + if args_str.startswith("@"): + try: + args_str = Path(args_str[1:]).read_text() + except OSError as e: + emit_json({"error": f"Cannot read args file: {e}"}) + return 1 + try: + user_args = json.loads(args_str) if args_str.strip() else {} + except json.JSONDecodeError as e: + emit_json({"error": f"Invalid --args JSON: {e}"}) + return 1 + if not isinstance(user_args, dict): + emit_json({"error": "--args must be a JSON object"}) + return 1 + + # ---- Resolve API key ---- + api_key = resolve_api_key(args.api_key) + partner_key = args.partner_key or None + if args.use_partner_key_as_auth and not api_key and partner_key: + api_key = partner_key + + # ---- Connect ---- + runner = ComfyRunner( + host=args.host, api_key=api_key, partner_key=partner_key, + client_id=args.client_id, + ) + + # Server reachability + ok, info = runner.check_server() + if not ok: + emit_json({ + "error": f"Cannot reach server at {args.host}", + "details": info, + "hint": ( + "Check `comfy launch --background` is running for local, " + f"or set ${ENV_API_KEY} for cloud." + ), + }) + return 1 + + # ---- Upload input images ---- + upload_warnings: list[str] = [] + for spec in args.input_image: + try: + param_name, path = parse_input_image_arg(spec) + except Exception as e: + emit_json({"error": f"Bad --input-image spec '{spec}': {e}"}) + return 1 + try: + ref = runner.upload_image(path) + except Exception as e: + emit_json({"error": f"Upload failed for {path}: {e}"}) + return 1 + # Register as a user arg so inject_params consumes it through the schema + uploaded_name = ref.get("name") or path.name + if param_name not in user_args: + user_args[param_name] = uploaded_name + + # ---- Inject params ---- + schema = load_schema(args.schema, workflow) + workflow, inj_warnings = inject_params( + workflow, schema, user_args, randomize_seed_if_unset=args.randomize_seed, + ) + warnings = upload_warnings + inj_warnings + for w in warnings: + log(f"WARN: {w}") + + # ---- Submit ---- + submit_resp = runner.submit(workflow) + if "_http_error" in submit_resp: + emit_json({ + "error": "Submission HTTP error", + "http_status": submit_resp["_http_error"], + "body": submit_resp.get("body"), + }) + return 1 + + if isinstance(submit_resp.get("error"), dict): + emit_json({ + "error": "Workflow validation failed", + "details": submit_resp["error"], + "node_errors": submit_resp.get("node_errors"), + }) + return 1 + + prompt_id = submit_resp.get("prompt_id") + if not prompt_id: + emit_json({"error": "No prompt_id in submit response", "response": submit_resp}) + return 1 + + node_errors = submit_resp.get("node_errors") or {} + if node_errors: + emit_json({"error": "Workflow validation failed", "node_errors": node_errors}) + return 1 + + if args.submit_only: + emit_json({"status": "submitted", "prompt_id": prompt_id, "warnings": warnings}) + return 0 + + # ---- Wait ---- + timeout = args.timeout + if timeout <= 0: + timeout = 900 if looks_like_video_workflow(workflow) else 300 + + log(f"Submitted: prompt_id={prompt_id}, waiting (timeout={timeout}s)…") + + def _on_progress(evt: dict) -> None: + t = evt.get("type") + if t == "progress": + log(f" step {evt.get('value')}/{evt.get('max')} on node {evt.get('node')}") + elif t == "executing": + node = evt.get("node") + if node: + log(f" executing node {node}") + + try: + if args.ws: + wait_result = runner.monitor_ws(prompt_id, timeout=timeout, on_progress=_on_progress) + else: + wait_result = runner.poll_status(prompt_id, timeout=timeout) + except KeyboardInterrupt: + log(f"Interrupted — cancelling job {prompt_id} on server…") + try: + runner.cancel(prompt_id) + except Exception as e: + log(f" (cancel request failed: {e})") + emit_json({ + "status": "interrupted", + "prompt_id": prompt_id, + "note": "Ctrl+C received; sent cancellation to server.", + }) + return 130 + + if wait_result["status"] == "timeout": + emit_json({ + "status": "timeout", + "prompt_id": prompt_id, + "elapsed": wait_result.get("elapsed"), + "hint": "Re-run with larger --timeout, or use --submit-only and check later.", + }) + return 1 + if wait_result["status"] == "error": + emit_json({"status": "error", "prompt_id": prompt_id, "details": wait_result.get("data")}) + return 1 + if wait_result["status"] == "cancelled": + emit_json({"status": "cancelled", "prompt_id": prompt_id}) + return 1 + + # ---- Outputs ---- + outputs = wait_result.get("outputs") + if not outputs: + outputs = runner.get_outputs(prompt_id) + + if args.no_download: + emit_json({ + "status": "success", "prompt_id": prompt_id, + "outputs": outputs, "warnings": warnings, + }) + return 0 + + downloaded = download_outputs( + runner, outputs, Path(args.output_dir).expanduser(), + preserve_subfolder=not args.flat_output, overwrite=args.overwrite, + ) + + emit_json({ + "status": "success", + "prompt_id": prompt_id, + "outputs": downloaded, + "warnings": warnings, + }) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/image-creation/comfyui/scripts/ws_monitor.py b/image-creation/comfyui/scripts/ws_monitor.py new file mode 100644 index 0000000..e2b6689 --- /dev/null +++ b/image-creation/comfyui/scripts/ws_monitor.py @@ -0,0 +1,267 @@ +#!/usr/bin/env python3 +""" +ws_monitor.py — Real-time ComfyUI WebSocket monitor. + +Connects to /ws and pretty-prints execution events: node start/finish, sampling +progress, cached nodes, errors. Optionally writes preview frames to disk. + +Useful for: + - Watching a long-running job in real time without parsing JSON yourself + - Saving in-progress preview frames for video / animation workflows + - Debugging "why is this hanging?" — see exactly which node is stuck + +Usage: + # Local — watch all jobs from this client_id + python3 ws_monitor.py + + # Cloud — watch a specific prompt_id + python3 ws_monitor.py --host https://cloud.comfy.org \ + --prompt-id abc-123-def + + # Save preview frames to ./previews/ + python3 ws_monitor.py --previews ./previews + +Requires: websocket-client (`pip install websocket-client`). +Falls back to a clear error message when not installed. +""" + +from __future__ import annotations + +import argparse +import json +import struct +import sys +from pathlib import Path +from urllib.parse import urlparse + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from _common import ( # noqa: E402 + DEFAULT_LOCAL_HOST, ENV_API_KEY, log, new_client_id, resolve_api_key, is_cloud_host, +) + + +# Binary frame types from ComfyUI WebSocket protocol +BINARY_PREVIEW_IMAGE = 1 +BINARY_TEXT = 3 +BINARY_PREVIEW_IMAGE_WITH_METADATA = 4 + +# Image type codes inside PREVIEW_IMAGE +IMAGE_TYPE_JPEG = 1 +IMAGE_TYPE_PNG = 2 + +# ANSI escape codes (works on most modern terminals) +RESET = "\033[0m" +DIM = "\033[2m" +BOLD = "\033[1m" +GREEN = "\033[32m" +YELLOW = "\033[33m" +RED = "\033[31m" +CYAN = "\033[36m" + + +def fmt_color(s: str, color: str, *, color_on: bool = True) -> str: + return f"{color}{s}{RESET}" if color_on else s + + +def parse_binary_frame(data: bytes) -> dict | None: + if len(data) < 8: + return None + type_code = struct.unpack(">I", data[0:4])[0] + if type_code == BINARY_PREVIEW_IMAGE: + image_type = struct.unpack(">I", data[4:8])[0] + ext = "jpg" if image_type == IMAGE_TYPE_JPEG else "png" if image_type == IMAGE_TYPE_PNG else "bin" + return { + "kind": "preview", + "image_type": image_type, + "ext": ext, + "image_bytes": data[8:], + } + if type_code == BINARY_PREVIEW_IMAGE_WITH_METADATA: + if len(data) < 12: + return None + meta_len = struct.unpack(">I", data[4:8])[0] + meta_end = 8 + meta_len + if len(data) < meta_end: + return None + try: + meta = json.loads(data[8:meta_end].decode("utf-8")) + except Exception: + meta = {"raw": data[8:meta_end][:200].decode("utf-8", "replace")} + return { + "kind": "preview_with_metadata", + "metadata": meta, + "image_bytes": data[meta_end:], + "ext": "png", + } + if type_code == BINARY_TEXT: + if len(data) < 8: + return None + nid_len = struct.unpack(">I", data[4:8])[0] + nid_end = 8 + nid_len + if len(data) < nid_end: + return None + return { + "kind": "text", + "node_id": data[8:nid_end].decode("utf-8", "replace"), + "text": data[nid_end:].decode("utf-8", "replace"), + } + return {"kind": "unknown", "type_code": type_code, "size": len(data)} + + +def main(argv: list[str] | None = None) -> int: + p = argparse.ArgumentParser(description="Real-time ComfyUI WebSocket monitor") + p.add_argument("--host", default=DEFAULT_LOCAL_HOST, help="ComfyUI server URL") + p.add_argument("--api-key", help=f"API key for cloud (or set ${ENV_API_KEY} env var)") + p.add_argument("--client-id", default=None, help="Client ID (default: random UUID)") + p.add_argument("--prompt-id", default=None, + help="Filter to a specific prompt_id (default: all jobs)") + p.add_argument("--previews", default=None, + help="Directory to save in-progress preview frames") + p.add_argument("--no-color", action="store_true", help="Disable ANSI colour") + p.add_argument("--timeout", type=float, default=600.0, + help="Hard cap on monitor duration (default 600s)") + args = p.parse_args(argv) + + try: + import websocket # type: ignore[import-not-found] + except ImportError: + print(json.dumps({ + "error": "websocket-client not installed", + "install": "pip install websocket-client", + })) + return 1 + + api_key = resolve_api_key(args.api_key) + cloud = is_cloud_host(args.host) + client_id = args.client_id or new_client_id() + + # Build WS URL preserving any base-path component (e.g. behind reverse proxy). + parsed = urlparse(args.host if "://" in args.host else f"http://{args.host}") + scheme = "wss" if parsed.scheme == "https" else "ws" + netloc = parsed.netloc + base_path = parsed.path.rstrip("/") + ws_url = f"{scheme}://{netloc}{base_path}/ws?clientId={client_id}" + if cloud and api_key: + ws_url += f"&token={api_key}" + + color_on = not args.no_color and sys.stdout.isatty() + + preview_dir = Path(args.previews).expanduser() if args.previews else None + if preview_dir: + preview_dir.mkdir(parents=True, exist_ok=True) + log(f"Saving previews to {preview_dir}") + + log(f"Connecting to {ws_url} (client_id={client_id})") + if args.prompt_id: + log(f"Filtering messages to prompt_id={args.prompt_id}") + + ws = websocket.create_connection(ws_url, timeout=args.timeout) + ws.settimeout(args.timeout) + + preview_counter = 0 + try: + while True: + try: + msg = ws.recv() + except websocket.WebSocketTimeoutException: + log(f"Idle for {args.timeout}s — exiting") + return 0 + if isinstance(msg, bytes): + parsed = parse_binary_frame(msg) + if parsed is None: + continue + if parsed["kind"] in {"preview", "preview_with_metadata"} and preview_dir: + img_bytes = parsed.get("image_bytes", b"") + if img_bytes: + ext = parsed.get("ext", "png") + out = preview_dir / f"preview_{preview_counter:05d}.{ext}" + out.write_bytes(img_bytes) + preview_counter += 1 + log(f" [preview] saved {out.name} ({len(img_bytes)} bytes)") + continue + + try: + payload = json.loads(msg) + except Exception: + continue + mtype = payload.get("type", "") + mdata = payload.get("data", {}) or {} + pid = mdata.get("prompt_id") + + if args.prompt_id and pid and pid != args.prompt_id: + continue + + if mtype == "status": + qr = mdata.get("status", {}).get("exec_info", {}).get("queue_remaining", "?") + print(fmt_color(f"[status] queue_remaining={qr}", DIM, color_on=color_on)) + elif mtype == "execution_start": + print(fmt_color(f"[start] prompt_id={pid}", BOLD, color_on=color_on)) + elif mtype == "executing": + node = mdata.get("node") + if node: + print(fmt_color(f" [executing] node={node}", CYAN, color_on=color_on)) + else: + print(fmt_color(f" [executing] (workflow done) prompt_id={pid}", DIM, color_on=color_on)) + elif mtype == "progress": + v, m = mdata.get("value", 0), mdata.get("max", 0) + pct = (v / m * 100) if m else 0 + print(f" [progress] {v}/{m} ({pct:5.1f}%) node={mdata.get('node')}") + elif mtype == "progress_state": + # Newer extended progress message + nodes = mdata.get("nodes") or {} + running = [k for k, v in nodes.items() if v.get("running")] + if running: + print(fmt_color(f" [progress_state] running={running}", DIM, color_on=color_on)) + elif mtype == "executed": + node = mdata.get("node") + out = mdata.get("output") or {} + summary_parts = [] + for key in ("images", "video", "videos", "gifs", "audio", "files"): + if out.get(key): + summary_parts.append(f"{key}={len(out[key])}") + summary = ", ".join(summary_parts) if summary_parts else "(no files)" + print(fmt_color(f" [executed] node={node} {summary}", GREEN, color_on=color_on)) + elif mtype == "execution_cached": + cached = mdata.get("nodes") or [] + if cached: + print(fmt_color(f" [cached] {len(cached)} nodes skipped", DIM, color_on=color_on)) + elif mtype == "execution_success": + print(fmt_color(f"[success] prompt_id={pid}", GREEN + BOLD, color_on=color_on)) + if args.prompt_id: + return 0 + elif mtype == "execution_error": + exc_type = mdata.get("exception_type", "?") + exc_msg = mdata.get("exception_message", "?") + print(fmt_color(f"[error] {exc_type}: {exc_msg}", RED + BOLD, color_on=color_on)) + tb = mdata.get("traceback") + if tb: + if isinstance(tb, list): + for line in tb: + print(fmt_color(f" {line}", RED, color_on=color_on)) + else: + print(fmt_color(f" {tb}", RED, color_on=color_on)) + if args.prompt_id: + return 1 + elif mtype == "execution_interrupted": + print(fmt_color(f"[interrupted] prompt_id={pid}", YELLOW, color_on=color_on)) + if args.prompt_id: + return 1 + elif mtype == "notification": + v = mdata.get("value", "") + print(fmt_color(f"[notification] {v}", DIM, color_on=color_on)) + else: + # Unknown / lightly-used types: print compactly + print(fmt_color(f"[{mtype}] {json.dumps(mdata, default=str)[:200]}", DIM, color_on=color_on)) + + except KeyboardInterrupt: + log("Interrupted") + return 130 + finally: + try: + ws.close() + except Exception: + pass + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/image-creation/comfyui/tests/README.md b/image-creation/comfyui/tests/README.md new file mode 100644 index 0000000..833632a --- /dev/null +++ b/image-creation/comfyui/tests/README.md @@ -0,0 +1,50 @@ +# ComfyUI Skill Tests + +Pytest suite covering the skill's scripts. Pure-stdlib unit tests run +without any setup; cloud integration tests need a Comfy Cloud API key. + +## Running + +```bash +# Unit tests only (no network required) — runs in <1s +python3 -m pytest tests/ -c tests/pytest.ini -o addopts="-p no:xdist" + +# Including cloud integration tests +COMFY_CLOUD_API_KEY="comfyui-..." python3 -m pytest tests/ \ + -c tests/pytest.ini -o addopts="-p no:xdist" + +# Just cloud tests +COMFY_CLOUD_API_KEY="comfyui-..." python3 -m pytest tests/test_cloud_integration.py \ + -c tests/pytest.ini -o addopts="-p no:xdist" -v +``` + +The `-c` and `-o` overrides isolate this suite from any parent +`pyproject.toml` pytest config (e.g. the `-n auto` from a parent repo). + +## Test files + +| File | Coverage | +|------|----------| +| `test_common.py` | Cloud detection, URL routing, format validation, embeddings, paths, seeds, model-list parsing, folder aliases | +| `test_extract_schema.py` | Connection tracing, positive/negative prompt detection, dedup logic, embedding deps | +| `test_run_workflow.py` | Param injection (incl. -1 seed, link refusal), output download walk, runner construction | +| `test_check_deps.py` | Model-name fuzzy matching, install command suggestions | +| `test_cloud_integration.py` | Live cloud API contract tests (auto-skipped without API key) | + +## Adding tests + +When you change a script: + +1. Add a unit test if the change is pure logic (cloud detection, parsing, etc.) +2. Add a cloud integration test if the change depends on cloud API behavior + (use `pytestmark = pytest.mark.cloud` so it auto-skips without a key) +3. Workflow fixtures live in `conftest.py` (`sd15_workflow`, `flux_workflow`, + `video_workflow`) + +## Why the explicit `-c` / `-o`? + +The parent hermes-agent repo's `pyproject.toml` enables `pytest-xdist` by +default (`-n auto`). This suite is small enough that parallelism isn't +worth the complexity, and pytest-xdist isn't always installed in the user's +environment. The `-c tests/pytest.ini -o addopts="-p no:xdist"` flags make +the suite run identically regardless of the parent project's config. diff --git a/image-creation/comfyui/tests/conftest.py b/image-creation/comfyui/tests/conftest.py new file mode 100644 index 0000000..a800fa7 --- /dev/null +++ b/image-creation/comfyui/tests/conftest.py @@ -0,0 +1,64 @@ +"""Pytest configuration for the comfyui skill test suite. + +Adds `scripts/` to sys.path so tests can `from _common import ...`, and +provides a few common fixtures. +""" + +from __future__ import annotations + +import json +import os +import sys +from pathlib import Path + +import pytest + +ROOT = Path(__file__).resolve().parent.parent +SCRIPTS = ROOT / "scripts" +WORKFLOWS = ROOT / "workflows" + +sys.path.insert(0, str(SCRIPTS)) + + +@pytest.fixture +def sd15_workflow() -> dict: + return json.loads((WORKFLOWS / "sd15_txt2img.json").read_text()) + + +@pytest.fixture +def flux_workflow() -> dict: + return json.loads((WORKFLOWS / "flux_dev_txt2img.json").read_text()) + + +@pytest.fixture +def video_workflow() -> dict: + return json.loads((WORKFLOWS / "wan_video_t2v.json").read_text()) + + +@pytest.fixture +def workflows_dir() -> Path: + return WORKFLOWS + + +@pytest.fixture +def scripts_dir() -> Path: + return SCRIPTS + + +@pytest.fixture +def cloud_key() -> str | None: + """Cloud API key if set, otherwise None. + + Tests that need cloud connectivity should skip when this is None. + """ + return os.environ.get("COMFY_CLOUD_API_KEY") + + +def pytest_collection_modifyitems(config, items): + """Auto-skip cloud tests when no API key is set.""" + if os.environ.get("COMFY_CLOUD_API_KEY"): + return + skip_cloud = pytest.mark.skip(reason="Set COMFY_CLOUD_API_KEY to run cloud tests") + for item in items: + if "cloud" in item.keywords: + item.add_marker(skip_cloud) diff --git a/image-creation/comfyui/tests/pytest.ini b/image-creation/comfyui/tests/pytest.ini new file mode 100644 index 0000000..2111fe2 --- /dev/null +++ b/image-creation/comfyui/tests/pytest.ini @@ -0,0 +1,5 @@ +[pytest] +markers = + cloud: tests that hit live Comfy Cloud API (require COMFY_CLOUD_API_KEY) +testpaths = . +addopts = -p no:xdist diff --git a/image-creation/comfyui/tests/test_check_deps.py b/image-creation/comfyui/tests/test_check_deps.py new file mode 100644 index 0000000..30116a7 --- /dev/null +++ b/image-creation/comfyui/tests/test_check_deps.py @@ -0,0 +1,68 @@ +"""Tests for check_deps.py — focuses on parsing logic that doesn't need a server.""" + +from __future__ import annotations + +from check_deps import ( + NODE_TO_PACKAGE, + model_present, + normalize_for_match, + suggest_install_command, +) + + +class TestNormalizeForMatch: + def test_basic(self): + s = normalize_for_match("model.safetensors") + assert "model.safetensors" in s + assert "model" in s + + def test_subfolder(self): + s = normalize_for_match("subdir/model.pt") + assert "subdir/model.pt" in s + assert "model.pt" in s + assert "model" in s + + +class TestModelPresent: + def test_exact_match(self): + assert model_present("a.safetensors", {"a.safetensors", "b.safetensors"}) is True + + def test_extension_difference(self): + # User said "model" but installed is "model.safetensors" + assert model_present("model", {"model.safetensors"}) is True + # Reverse direction — also matches + assert model_present("model.safetensors", {"model"}) is True + + def test_subfolder_match(self): + # Installed list has "subdir/model.safetensors", workflow asks "model.safetensors" + assert model_present("model.safetensors", {"subdir/model.safetensors"}) is True + + def test_missing(self): + assert model_present("missing.safetensors", {"a.safetensors", "b.safetensors"}) is False + + def test_empty_installed(self): + assert model_present("anything.safetensors", set()) is False + + +class TestSuggestInstallCommand: + def test_known_node(self): + cmd = suggest_install_command("VHS_VideoCombine") + assert cmd == "comfy node install comfyui-videohelpersuite" + + def test_unknown_node(self): + assert suggest_install_command("SomeRandomNodeName123") is None + + +class TestNodePackageMap: + def test_no_duplicates(self): + # Each node should map to exactly one package + keys = list(NODE_TO_PACKAGE.keys()) + assert len(keys) == len(set(keys)) + + def test_packages_are_safe_for_shell(self): + # Registry slugs must be alphanumerics + hyphens/underscores only + # (passed straight to `comfy node install `). + import re + safe = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._\-]*$") + for pkg in NODE_TO_PACKAGE.values(): + assert safe.match(pkg), f"Unsafe package slug: {pkg!r}" diff --git a/image-creation/comfyui/tests/test_cloud_integration.py b/image-creation/comfyui/tests/test_cloud_integration.py new file mode 100644 index 0000000..0ce88ef --- /dev/null +++ b/image-creation/comfyui/tests/test_cloud_integration.py @@ -0,0 +1,95 @@ +"""Integration tests against the live Comfy Cloud API. + +These tests are auto-skipped when COMFY_CLOUD_API_KEY is not set. +They never SUBMIT workflows (would need a paid subscription) — they only +verify the read-only endpoints we rely on. +""" + +from __future__ import annotations + +import pytest + +from _common import http_get, parse_model_list, resolve_url + + +pytestmark = pytest.mark.cloud + + +class TestCloudEndpointsLive: + def test_system_stats_reachable(self, cloud_key): + url = resolve_url("https://cloud.comfy.org", "/system_stats") + r = http_get(url, headers={"X-API-Key": cloud_key}) + assert r.status == 200 + data = r.json() + assert "system" in data + + def test_models_endpoint_routed_to_experiment(self, cloud_key): + # We expect the skill to route /models/checkpoints → /api/experiment/models/checkpoints + url = resolve_url("https://cloud.comfy.org", "/models/checkpoints") + assert "/api/experiment/models/checkpoints" in url + r = http_get(url, headers={"X-API-Key": cloud_key}) + assert r.status == 200 + + def test_models_endpoint_returns_dicts(self, cloud_key): + url = resolve_url("https://cloud.comfy.org", "/models/checkpoints") + r = http_get(url, headers={"X-API-Key": cloud_key}) + data = r.json() + assert isinstance(data, list) + if data: + # Cloud format: list of dicts with `name` + assert isinstance(data[0], dict) + assert "name" in data[0] + # Our parser normalizes both + normalized = parse_model_list(data) + assert len(normalized) == len(data) + + def test_history_renamed_to_v2(self, cloud_key): + # /history → /api/history_v2 on cloud + url = resolve_url("https://cloud.comfy.org", "/history/some-fake-id") + assert "/api/history_v2/some-fake-id" in url + + def test_object_info_paid_tier(self, cloud_key): + # On free tier, /object_info returns 403 with a recognizable message + url = resolve_url("https://cloud.comfy.org", "/object_info") + r = http_get(url, headers={"X-API-Key": cloud_key}) + # Should be either 200 (paid) or 403 (free) — not 404 / 500 + assert r.status in {200, 403} + if r.status == 403: + # Body should mention the limitation + assert "free tier" in r.text().lower() or "subscription" in r.text().lower() + + +class TestCloudCheckDepsLive: + def test_check_deps_against_cloud(self, cloud_key, sd15_workflow): + from check_deps import check_deps + report = check_deps(sd15_workflow, host="https://cloud.comfy.org", api_key=cloud_key) + # Either node check passed OR was skipped (free tier) + assert "missing_models" in report + assert "is_cloud" in report and report["is_cloud"] is True + + def test_flux_workflow_models_resolved_via_aliases(self, cloud_key, flux_workflow): + """Flux uses unet/clip folders; cloud has them in diffusion_models/text_encoders. + With folder aliasing, the check should still find them.""" + from check_deps import check_deps + report = check_deps(flux_workflow, host="https://cloud.comfy.org", api_key=cloud_key) + # The exact required Flux files (flux1-dev.safetensors, t5xxl_fp16, clip_l, ae) + # are present on cloud; with folder aliasing, none should be missing. + # If this fails, either the cloud removed the model or the aliasing logic broke. + missing_filenames = {m["value"] for m in report["missing_models"]} + assert "ae.safetensors" not in missing_filenames, \ + "ae.safetensors should be on cloud's vae folder" + # t5xxl_fp16 / clip_l should be reachable via the clip → text_encoders alias + # flux1-dev.safetensors likewise via unet → diffusion_models + + +class TestHealthCheckLive: + def test_health_check_passes(self, cloud_key, capsys): + from health_check import main as health_main + rc = health_main(["--host", "https://cloud.comfy.org", "--api-key", cloud_key]) + captured = capsys.readouterr() + # Should produce JSON + import json + report = json.loads(captured.out) + assert report["server"]["reachable"] is True + assert report["checkpoints"]["queryable"] is True + assert report["checkpoints"]["count"] > 0 diff --git a/image-creation/comfyui/tests/test_common.py b/image-creation/comfyui/tests/test_common.py new file mode 100644 index 0000000..a5ce6a3 --- /dev/null +++ b/image-creation/comfyui/tests/test_common.py @@ -0,0 +1,443 @@ +"""Unit tests for _common.py — pure logic only, no network.""" + +from __future__ import annotations + + +import pytest + +from _common import ( + EMBEDDING_REGEX, + cloud_endpoint, + coerce_seed, + folder_aliases_for, + is_api_format, + is_cloud_host, + is_link, + iter_embedding_refs, + iter_model_deps, + iter_nodes, + looks_like_video_workflow, + media_type_from_filename, + parse_model_list, + resolve_url, + safe_path_join, + unwrap_workflow, +) + + +# ============================================================================= +# Cloud detection / URL routing +# ============================================================================= + +class TestCloudDetection: + def test_cloud_host_exact(self): + assert is_cloud_host("https://cloud.comfy.org") is True + assert is_cloud_host("https://cloud.comfy.org/foo/bar") is True + + def test_cloud_host_subdomain(self): + assert is_cloud_host("https://staging.cloud.comfy.org") is True + assert is_cloud_host("https://api.cloud.comfy.org") is True + + def test_local_not_cloud(self): + assert is_cloud_host("http://127.0.0.1:8188") is False + assert is_cloud_host("http://localhost:8188") is False + assert is_cloud_host("http://my-server.local:8188") is False + + def test_no_scheme(self): + # Defaults to http:// + assert is_cloud_host("cloud.comfy.org") is True + assert is_cloud_host("127.0.0.1:8188") is False + + +class TestCloudEndpointRename: + def test_history_renamed(self): + assert cloud_endpoint("/history") == "/history_v2" + assert cloud_endpoint("/history/abc-123") == "/history_v2/abc-123" + + def test_history_v2_preserved(self): + assert cloud_endpoint("/history_v2") == "/history_v2" + + def test_models_renamed(self): + assert cloud_endpoint("/models") == "/experiment/models" + assert cloud_endpoint("/models/checkpoints") == "/experiment/models/checkpoints" + assert cloud_endpoint("/models/loras") == "/experiment/models/loras" + + def test_other_paths_unchanged(self): + assert cloud_endpoint("/prompt") == "/prompt" + assert cloud_endpoint("/queue") == "/queue" + + +class TestResolveURL: + def test_local_no_prefix(self): + assert resolve_url("http://127.0.0.1:8188", "/prompt") == "http://127.0.0.1:8188/prompt" + + def test_cloud_adds_api_prefix(self): + assert resolve_url("https://cloud.comfy.org", "/prompt") == "https://cloud.comfy.org/api/prompt" + + def test_cloud_history_renamed(self): + assert resolve_url("https://cloud.comfy.org", "/history/abc") == "https://cloud.comfy.org/api/history_v2/abc" + + def test_cloud_models_renamed(self): + assert resolve_url("https://cloud.comfy.org", "/models/loras") == "https://cloud.comfy.org/api/experiment/models/loras" + + def test_cloud_already_has_api(self): + # Don't double-prefix + assert resolve_url("https://cloud.comfy.org", "/api/prompt") == "https://cloud.comfy.org/api/prompt" + + def test_trailing_slash_stripped(self): + assert resolve_url("http://127.0.0.1:8188/", "/prompt") == "http://127.0.0.1:8188/prompt" + + +# ============================================================================= +# Workflow validation +# ============================================================================= + +class TestAPIFormatDetection: + def test_valid_api(self, sd15_workflow): + assert is_api_format(sd15_workflow) is True + + def test_editor_format_rejected(self): + editor = {"nodes": [], "links": [], "version": 0.4} + assert is_api_format(editor) is False + + def test_empty_dict(self): + assert is_api_format({}) is False + + def test_non_dict(self): + assert is_api_format([]) is False + assert is_api_format(None) is False + assert is_api_format("string") is False + + def test_node_with_class_type(self): + wf = {"3": {"class_type": "KSampler", "inputs": {}}} + assert is_api_format(wf) is True + + +class TestUnwrapWorkflow: + def test_passthrough_api_format(self, sd15_workflow): + result = unwrap_workflow(sd15_workflow) + assert result is sd15_workflow + + def test_unwrap_prompt_key(self, sd15_workflow): + wrapped = {"prompt": sd15_workflow, "client_id": "abc"} + result = unwrap_workflow(wrapped) + assert result is sd15_workflow + + def test_editor_format_raises(self): + with pytest.raises(ValueError, match="editor format"): + unwrap_workflow({"nodes": [], "links": []}) + + def test_garbage_raises(self): + with pytest.raises(ValueError): + unwrap_workflow({"foo": "bar"}) + + +class TestIsLink: + def test_valid_link(self): + assert is_link(["3", 0]) is True + assert is_link(["10", 1]) is True + + def test_non_link(self): + assert is_link("string") is False + assert is_link(42) is False + assert is_link([]) is False + assert is_link(["3"]) is False # missing slot + assert is_link(["3", "0"]) is False # slot must be int + assert is_link([3, 0]) is False # node_id must be string + + +# ============================================================================= +# Workflow iterators +# ============================================================================= + +class TestIterators: + def test_iter_nodes(self, sd15_workflow): + nodes = dict(iter_nodes(sd15_workflow)) + assert "3" in nodes + assert nodes["3"]["class_type"] == "KSampler" + + def test_iter_nodes_skips_comments(self, sd15_workflow): + # _comment is not a node + nodes = dict(iter_nodes(sd15_workflow)) + assert "_comment" not in nodes + + def test_iter_model_deps(self, sd15_workflow): + deps = list(iter_model_deps(sd15_workflow)) + names = [d["value"] for d in deps] + assert "v1-5-pruned-emaonly.safetensors" in names + + def test_iter_model_deps_flux(self, flux_workflow): + deps = list(iter_model_deps(flux_workflow)) + names = {d["value"]: d["folder"] for d in deps} + assert names["flux1-dev.safetensors"] == "unet" + assert names["t5xxl_fp16.safetensors"] == "clip" + assert names["clip_l.safetensors"] == "clip" + assert names["ae.safetensors"] == "vae" + + +# ============================================================================= +# Embedding extraction +# ============================================================================= + +class TestEmbeddingRegex: + def test_basic_embedding(self): + m = EMBEDDING_REGEX.search("a cat, embedding:goodvibes, more text") + assert m is not None + assert m.group(1) == "goodvibes" + + def test_embedding_with_strength(self): + m = EMBEDDING_REGEX.search("embedding:bad-hands-5:1.2") + assert m is not None + assert m.group(1) == "bad-hands-5" + + def test_embedding_with_extension(self): + # Strips .pt / .safetensors / .bin + m = EMBEDDING_REGEX.search("embedding:my-emb.pt") + assert m is not None + assert m.group(1) == "my-emb" + + def test_embedding_in_parens(self): + m = EMBEDDING_REGEX.search("(embedding:foo:0.8)") + assert m is not None + assert m.group(1) == "foo" + + def test_multiple_in_one_string(self): + text = "a cat, embedding:foo:1.2, and embedding:bar" + matches = [m.group(1) for m in EMBEDDING_REGEX.finditer(text)] + assert matches == ["foo", "bar"] + + def test_no_false_positive_on_word_embedding(self): + # "embedding " (with space, no colon) should not match + m = EMBEDDING_REGEX.search("the embedding is great") + assert m is None + + +class TestIterEmbeddingRefs: + def test_finds_in_clip_text_encode(self): + wf = { + "1": {"class_type": "CLIPTextEncode", + "inputs": {"text": "embedding:foo, embedding:bar:0.5", "clip": ["2", 0]}}, + "2": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "x"}}, + } + refs = list(iter_embedding_refs(wf)) + names = [name for _, name in refs] + assert names == ["foo", "bar"] + + def test_ignores_non_prompt_fields(self): + wf = { + "1": {"class_type": "CheckpointLoaderSimple", + "inputs": {"ckpt_name": "embedding:foo.safetensors"}}, + } + refs = list(iter_embedding_refs(wf)) + # ckpt_name is not a prompt field — ignored + assert refs == [] + + +# ============================================================================= +# Path safety +# ============================================================================= + +class TestSafePathJoin: + def test_normal_join(self, tmp_path): + p = safe_path_join(tmp_path, "subdir", "file.png") + assert p.is_relative_to(tmp_path) + + def test_blocks_traversal(self, tmp_path): + with pytest.raises(ValueError, match="path traversal"): + safe_path_join(tmp_path, "..", "..", "etc", "passwd") + + def test_blocks_absolute(self, tmp_path): + with pytest.raises(ValueError): + safe_path_join(tmp_path, "/etc/passwd") + + def test_subfolder_with_filename(self, tmp_path): + p = safe_path_join(tmp_path, "outputs", "img.png") + assert p.name == "img.png" + assert p.parent.name == "outputs" + + +# ============================================================================= +# Seed coercion +# ============================================================================= + +class TestCoerceSeed: + def test_explicit_int(self): + assert coerce_seed(42) == 42 + assert coerce_seed(0) == 0 + + def test_minus_one_randomizes(self): + s = coerce_seed(-1) + assert isinstance(s, int) + assert 0 <= s < 2**63 + + def test_none_randomizes(self): + s = coerce_seed(None) + assert isinstance(s, int) + + def test_string_int(self): + # str() that converts cleanly is allowed (relaxed) + assert coerce_seed("12345") == 12345 + + def test_string_minus_one_randomizes(self): + # CLI / JSON sometimes carries seed as a string. + s = coerce_seed("-1") + assert isinstance(s, int) + assert 0 <= s < 2**63 + # And whitespace tolerated + s2 = coerce_seed(" -1 ") + assert isinstance(s2, int) + assert 0 <= s2 < 2**63 + + +# ============================================================================= +# Model list normalization (cloud format) +# ============================================================================= + +class TestParseModelList: + def test_local_format_strings(self): + result = parse_model_list(["a.safetensors", "b.safetensors"]) + assert result == {"a.safetensors", "b.safetensors"} + + def test_cloud_format_dicts(self): + result = parse_model_list([ + {"name": "a.safetensors", "pathIndex": 0}, + {"name": "b.safetensors", "pathIndex": 1}, + ]) + assert result == {"a.safetensors", "b.safetensors"} + + def test_empty(self): + assert parse_model_list([]) == set() + + def test_garbage(self): + assert parse_model_list("not a list") == set() + assert parse_model_list(None) == set() + + def test_mixed_format(self): + result = parse_model_list([ + "string-form.safetensors", + {"name": "dict-form.safetensors"}, + ]) + assert result == {"string-form.safetensors", "dict-form.safetensors"} + + +# ============================================================================= +# Folder aliases +# ============================================================================= + +class TestFolderAliases: + def test_unet_aliases_diffusion_models(self): + aliases = folder_aliases_for("unet") + assert "unet" in aliases + assert "diffusion_models" in aliases + + def test_clip_aliases_text_encoders(self): + aliases = folder_aliases_for("clip") + assert "clip" in aliases + assert "text_encoders" in aliases + + def test_unknown_folder_returns_self(self): + assert folder_aliases_for("checkpoints") == ["checkpoints"] + + def test_primary_first(self): + # Order matters: primary should be first for human-friendly fix hints + assert folder_aliases_for("unet")[0] == "unet" + assert folder_aliases_for("diffusion_models")[0] == "diffusion_models" + + +# ============================================================================= +# Media-type detection +# ============================================================================= + +class TestMediaType: + def test_video_extensions(self): + assert media_type_from_filename("vid.mp4") == "video" + assert media_type_from_filename("foo.webm") == "video" + assert media_type_from_filename("bar.gif") == "video" + + def test_audio_extensions(self): + assert media_type_from_filename("song.wav") == "audio" + assert media_type_from_filename("music.mp3") == "audio" + + def test_image_default(self): + assert media_type_from_filename("pic.png") == "image" + assert media_type_from_filename("image.jpg") == "image" + assert media_type_from_filename("unknown.xyz") == "image" + + def test_3d(self): + assert media_type_from_filename("model.glb") == "3d" + assert media_type_from_filename("scene.gltf") == "3d" + + +# ============================================================================= +# Cross-host header stripping (security) +# ============================================================================= + +class TestRedirectHeaderStripping: + """Verify X-API-Key is dropped when redirect crosses to a different host + (e.g. cloud /api/view → S3 signed URL). Critical to prevent leaking auth + tokens to the storage backend. + """ + + def _build_session(self): + from _common import _StripSensitiveOnRedirectSession, HAS_REQUESTS + if not HAS_REQUESTS: + import pytest + pytest.skip("requests not installed") + return _StripSensitiveOnRedirectSession() + + def test_strips_x_api_key_cross_host(self): + import requests + s = self._build_session() + prep = requests.PreparedRequest() + prep.prepare(method="GET", url="https://other.example.com/file", + headers={"X-API-Key": "leak", "Authorization": "Bearer x"}) + resp = requests.Response() + orig = requests.PreparedRequest() + orig.prepare(method="GET", url="https://cloud.comfy.org/api/view", headers={}) + resp.request = orig + s.rebuild_auth(prep, resp) + assert "X-API-Key" not in prep.headers + assert "Authorization" not in prep.headers + + def test_preserves_x_api_key_same_host(self): + import requests + s = self._build_session() + prep = requests.PreparedRequest() + prep.prepare(method="GET", url="https://cloud.comfy.org/foo", + headers={"X-API-Key": "keep"}) + resp = requests.Response() + orig = requests.PreparedRequest() + orig.prepare(method="GET", url="https://cloud.comfy.org/bar", headers={}) + resp.request = orig + s.rebuild_auth(prep, resp) + assert prep.headers.get("X-API-Key") == "keep" + + def test_strips_cookie_cross_host(self): + import requests + s = self._build_session() + prep = requests.PreparedRequest() + prep.prepare(method="GET", url="https://other.example.com/x", + headers={"Cookie": "session=secret"}) + resp = requests.Response() + orig = requests.PreparedRequest() + orig.prepare(method="GET", url="https://cloud.comfy.org/foo", headers={}) + resp.request = orig + s.rebuild_auth(prep, resp) + assert "Cookie" not in prep.headers + + +# ============================================================================= +# Video workflow detection +# ============================================================================= + +class TestVideoWorkflow: + def test_image_workflow(self, sd15_workflow): + assert looks_like_video_workflow(sd15_workflow) is False + + def test_animatediff_workflow(self, workflows_dir): + import json + wf = json.loads((workflows_dir / "animatediff_video.json").read_text()) + assert looks_like_video_workflow(wf) is True + + def test_wan_workflow(self, video_workflow): + assert looks_like_video_workflow(video_workflow) is True diff --git a/image-creation/comfyui/tests/test_extract_schema.py b/image-creation/comfyui/tests/test_extract_schema.py new file mode 100644 index 0000000..c4557ba --- /dev/null +++ b/image-creation/comfyui/tests/test_extract_schema.py @@ -0,0 +1,184 @@ +"""Tests for extract_schema.py.""" + +from __future__ import annotations + + +from extract_schema import ( + extract_schema, + find_negative_prompt_node, + find_positive_prompt_node, + trace_to_node, +) + + +# ============================================================================= +# Connection tracing +# ============================================================================= + +class TestConnectionTracing: + def test_direct_link(self): + wf = { + "1": {"class_type": "CLIPTextEncode", "inputs": {"text": "x"}}, + "2": {"class_type": "KSampler", + "inputs": {"positive": ["1", 0], "negative": ["1", 0]}}, + } + assert trace_to_node(wf, ["1", 0]) == "1" + + def test_through_reroute(self): + wf = { + "1": {"class_type": "CLIPTextEncode", "inputs": {"text": "x"}}, + "2": {"class_type": "Reroute", "inputs": {"input": ["1", 0]}}, + "3": {"class_type": "Reroute", "inputs": {"input": ["2", 0]}}, + } + assert trace_to_node(wf, ["3", 0]) == "1" + + def test_circular_safe(self): + wf = { + "1": {"class_type": "Reroute", "inputs": {"input": ["2", 0]}}, + "2": {"class_type": "Reroute", "inputs": {"input": ["1", 0]}}, + } + # Should hit max_hops without infinite loop + result = trace_to_node(wf, ["1", 0], max_hops=5) + assert result in {"1", "2"} # any node, just don't hang + + +class TestPositiveNegativeDetection: + def test_basic(self, sd15_workflow): + # In sd15_workflow.json node 6 is positive, node 7 is negative + assert find_positive_prompt_node(sd15_workflow) == "6" + assert find_negative_prompt_node(sd15_workflow) == "7" + + def test_swapped_order(self): + wf = { + "3": {"class_type": "KSampler", + "inputs": { + "positive": ["7", 0], "negative": ["6", 0], + "model": ["4", 0], "latent_image": ["5", 0], + "seed": 1, "steps": 20, "cfg": 7.5, + "sampler_name": "euler", "scheduler": "normal", "denoise": 1.0, + }}, + "4": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "x"}}, + "5": {"class_type": "EmptyLatentImage", "inputs": {"width": 512, "height": 512, "batch_size": 1}}, + "6": {"class_type": "CLIPTextEncode", "inputs": {"text": "ugly", "clip": ["4", 1]}}, + "7": {"class_type": "CLIPTextEncode", "inputs": {"text": "beautiful", "clip": ["4", 1]}}, + } + # Now 7 is the positive (despite higher node ID) + assert find_positive_prompt_node(wf) == "7" + assert find_negative_prompt_node(wf) == "6" + + +# ============================================================================= +# Schema extraction +# ============================================================================= + +class TestExtractSchema: + def test_basic_sd15(self, sd15_workflow): + schema = extract_schema(sd15_workflow) + params = schema["parameters"] + assert "prompt" in params + assert "negative_prompt" in params + assert "seed" in params + assert "steps" in params + assert "cfg" in params + assert "width" in params + assert "height" in params + + def test_prompt_value_correct(self, sd15_workflow): + schema = extract_schema(sd15_workflow) + # The positive prompt in the example is the landscape one + assert "landscape" in schema["parameters"]["prompt"]["value"] + assert "ugly" in schema["parameters"]["negative_prompt"]["value"] + + def test_model_dependencies(self, sd15_workflow): + schema = extract_schema(sd15_workflow) + deps = schema["model_dependencies"] + ckpts = [d["value"] for d in deps if d["folder"] == "checkpoints"] + assert "v1-5-pruned-emaonly.safetensors" in ckpts + + def test_output_nodes(self, sd15_workflow): + schema = extract_schema(sd15_workflow) + assert "9" in schema["output_nodes"] + + def test_summary(self, sd15_workflow): + schema = extract_schema(sd15_workflow) + s = schema["summary"] + assert s["has_negative_prompt"] is True + assert s["has_seed"] is True + assert s["is_video_workflow"] is False + assert s["parameter_count"] > 5 + + def test_flux_workflow(self, flux_workflow): + schema = extract_schema(flux_workflow) + # Flux uses RandomNoise for seed + assert schema["summary"]["has_seed"] is True + # Flux has only positive prompt (no negative encoder) + assert schema["summary"]["has_negative_prompt"] is False + + def test_video_detected(self, video_workflow): + schema = extract_schema(video_workflow) + assert schema["summary"]["is_video_workflow"] is True + + +class TestEmbeddingDeps: + def test_extract_from_prompt(self): + wf = { + "1": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "x"}}, + "5": {"class_type": "EmptyLatentImage", + "inputs": {"width": 512, "height": 512, "batch_size": 1}}, + "6": {"class_type": "CLIPTextEncode", + "inputs": { + "text": "a cat, embedding:goodvibes, embedding:art:1.2", + "clip": ["1", 1] + }}, + "7": {"class_type": "CLIPTextEncode", + "inputs": { + "text": "ugly, embedding:badhands", + "clip": ["1", 1] + }}, + "3": {"class_type": "KSampler", + "inputs": { + "positive": ["6", 0], "negative": ["7", 0], + "model": ["1", 0], "latent_image": ["5", 0], + "seed": 1, "steps": 20, "cfg": 7.5, + "sampler_name": "euler", "scheduler": "normal", "denoise": 1.0, + }}, + "9": {"class_type": "SaveImage", "inputs": {"filename_prefix": "x", "images": ["3", 0]}}, + } + schema = extract_schema(wf) + names = [d["embedding_name"] for d in schema["embedding_dependencies"]] + assert sorted(names) == ["art", "badhands", "goodvibes"] + + +class TestDuplicateDeduplication: + def test_two_ksamplers_get_unique_names(self): + wf = { + "1": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "x"}}, + "5": {"class_type": "EmptyLatentImage", + "inputs": {"width": 512, "height": 512, "batch_size": 1}}, + "6": {"class_type": "CLIPTextEncode", "inputs": {"text": "a", "clip": ["1", 1]}}, + "7": {"class_type": "CLIPTextEncode", "inputs": {"text": "b", "clip": ["1", 1]}}, + "3": {"class_type": "KSampler", + "inputs": { + "positive": ["6", 0], "negative": ["7", 0], + "model": ["1", 0], "latent_image": ["5", 0], + "seed": 42, "steps": 20, "cfg": 7.5, + "sampler_name": "euler", "scheduler": "normal", "denoise": 1.0, + }}, + "4": {"class_type": "KSampler", + "inputs": { + "positive": ["6", 0], "negative": ["7", 0], + "model": ["1", 0], "latent_image": ["5", 0], + "seed": 99, "steps": 30, "cfg": 8.0, + "sampler_name": "euler", "scheduler": "normal", "denoise": 0.6, + }}, + "9": {"class_type": "SaveImage", "inputs": {"filename_prefix": "x", "images": ["3", 0]}}, + } + schema = extract_schema(wf) + params = schema["parameters"] + # Both seeds present with disambiguated names + seed_keys = [k for k in params if "seed" in k] + # Symmetric: both renamed (no bare "seed") + assert "seed" not in params + assert "seed_3" in params and "seed_4" in params + assert params["seed_3"]["value"] == 42 + assert params["seed_4"]["value"] == 99 diff --git a/image-creation/comfyui/tests/test_run_workflow.py b/image-creation/comfyui/tests/test_run_workflow.py new file mode 100644 index 0000000..69957dd --- /dev/null +++ b/image-creation/comfyui/tests/test_run_workflow.py @@ -0,0 +1,210 @@ +"""Tests for run_workflow.py — focuses on logic that doesn't require a server.""" + +from __future__ import annotations + + + +from extract_schema import extract_schema +from run_workflow import ( + ComfyRunner, + download_outputs, + inject_params, + parse_input_image_arg, +) + + +class TestParseInputImageArg: + def test_with_name(self, tmp_path): + f = tmp_path / "x.png" + f.write_text("x") + n, p = parse_input_image_arg(f"image={f}") + assert n == "image" + assert p == f + + def test_without_name_defaults(self, tmp_path): + f = tmp_path / "x.png" + f.write_text("x") + n, p = parse_input_image_arg(str(f)) + assert n == "image" + + def test_custom_name(self, tmp_path): + f = tmp_path / "x.png" + f.write_text("x") + n, p = parse_input_image_arg(f"mask_image={f}") + assert n == "mask_image" + + +class TestInjectParams: + def test_basic_injection(self, sd15_workflow): + schema = extract_schema(sd15_workflow) + wf, warnings = inject_params(sd15_workflow, schema, { + "prompt": "new prompt", + "seed": 999, + "steps": 25, + }) + assert wf["6"]["inputs"]["text"] == "new prompt" + assert wf["3"]["inputs"]["seed"] == 999 + assert wf["3"]["inputs"]["steps"] == 25 + assert warnings == [] + + def test_unknown_param_warns(self, sd15_workflow): + schema = extract_schema(sd15_workflow) + _, warnings = inject_params(sd15_workflow, schema, {"foobar": "x"}) + assert any("foobar" in w for w in warnings) + + def test_seed_minus_one_randomizes(self, sd15_workflow): + schema = extract_schema(sd15_workflow) + wf, warnings = inject_params(sd15_workflow, schema, {"seed": -1}) + assert wf["3"]["inputs"]["seed"] != -1 + assert isinstance(wf["3"]["inputs"]["seed"], int) + assert any("expanded" in w.lower() for w in warnings) + + def test_randomize_seed_when_unset(self, sd15_workflow): + schema = extract_schema(sd15_workflow) + original = sd15_workflow["3"]["inputs"]["seed"] + wf, warnings = inject_params(sd15_workflow, schema, {}, randomize_seed_if_unset=True) + assert wf["3"]["inputs"]["seed"] != original + assert isinstance(wf["3"]["inputs"]["seed"], int) + + def test_does_not_mutate_original(self, sd15_workflow): + schema = extract_schema(sd15_workflow) + original_text = sd15_workflow["6"]["inputs"]["text"] + inject_params(sd15_workflow, schema, {"prompt": "MUTATED"}) + assert sd15_workflow["6"]["inputs"]["text"] == original_text + + def test_refuses_to_overwrite_link(self): + wf = { + "1": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "x"}}, + "5": {"class_type": "EmptyLatentImage", + "inputs": {"width": 512, "height": 512, "batch_size": 1}}, + "6": {"class_type": "CLIPTextEncode", + "inputs": {"text": ["3", 0], "clip": ["1", 1]}}, # text is a link! + "3": {"class_type": "KSampler", + "inputs": {"seed": 1, "steps": 20, "cfg": 7.5, + "sampler_name": "euler", "scheduler": "normal", "denoise": 1.0, + "model": ["1", 0], "positive": ["6", 0], "negative": ["6", 0], + "latent_image": ["5", 0]}}, + "9": {"class_type": "SaveImage", "inputs": {"filename_prefix": "x", "images": ["3", 0]}}, + } + # Manually create a schema that has prompt pointing at 6.text + schema = { + "parameters": { + "prompt": {"node_id": "6", "field": "text", "type": "string", "value": ""}, + } + } + wf2, warnings = inject_params(wf, schema, {"prompt": "literal value"}) + # The link should NOT have been overwritten + assert wf2["6"]["inputs"]["text"] == ["3", 0] + assert any("link" in w.lower() for w in warnings) + + +# ============================================================================= +# Output download walk +# ============================================================================= + +class TestDownloadOutputsWalk: + """Test that download_outputs walks the structure correctly.""" + + def test_handles_videos_plural(self, tmp_path, monkeypatch): + """Local ComfyUI uses 'videos'/'gifs' (plural) keys.""" + downloads = [] + + class FakeRunner: + def download_output(self, *, filename, subfolder, file_type, output_dir, preserve_subfolder, overwrite): + downloads.append((filename, subfolder, file_type)) + p = output_dir / filename + p.parent.mkdir(parents=True, exist_ok=True) + p.write_bytes(b"x") + return p + + outputs = { + "9": {"images": [{"filename": "img1.png", "subfolder": "", "type": "output"}]}, + "10": {"videos": [{"filename": "vid1.mp4", "subfolder": "", "type": "output"}]}, + "11": {"gifs": [{"filename": "anim1.gif", "subfolder": "", "type": "output"}]}, + } + + result = download_outputs(FakeRunner(), outputs, tmp_path) + files = sorted(d["filename"] for d in result) + assert files == ["anim1.gif", "img1.png", "vid1.mp4"] + + def test_handles_video_singular_cloud(self, tmp_path): + """Cloud uses 'video' (singular).""" + class FakeRunner: + def download_output(self, *, filename, subfolder, file_type, output_dir, preserve_subfolder, overwrite): + p = output_dir / filename + p.parent.mkdir(parents=True, exist_ok=True) + p.write_bytes(b"x") + return p + + outputs = { + "10": {"video": [{"filename": "cloud.mp4", "subfolder": "", "type": "output"}]}, + } + result = download_outputs(FakeRunner(), outputs, tmp_path) + assert len(result) == 1 + assert result[0]["filename"] == "cloud.mp4" + + def test_preserves_subfolder(self, tmp_path): + """When preserve_subfolder=True, server subfolder becomes local subdir.""" + class FakeRunner: + def download_output(self, *, filename, subfolder, file_type, output_dir, preserve_subfolder, overwrite): + if preserve_subfolder and subfolder: + p = output_dir / subfolder / filename + else: + p = output_dir / filename + p.parent.mkdir(parents=True, exist_ok=True) + p.write_bytes(b"x") + return p + + outputs = { + "9": {"images": [ + {"filename": "img.png", "subfolder": "myrun", "type": "output"}, + {"filename": "img.png", "subfolder": "otherrun", "type": "output"}, + ]}, + } + result = download_outputs(FakeRunner(), outputs, tmp_path, preserve_subfolder=True) + files = [d["file"] for d in result] + assert any("myrun" in f for f in files) + assert any("otherrun" in f for f in files) + # Both must exist (no collision) + assert len({str(f) for f in files}) == 2 + + +# ============================================================================= +# ComfyRunner construction +# ============================================================================= + +class TestRunnerConstruction: + def test_local_default(self): + r = ComfyRunner() + assert r.is_cloud is False + assert r.host == "http://127.0.0.1:8188" + + def test_cloud_detection(self): + r = ComfyRunner(host="https://cloud.comfy.org", api_key="abc") + assert r.is_cloud is True + assert "X-API-Key" in r.headers + + def test_cloud_subdomain_detected(self): + r = ComfyRunner(host="https://staging.cloud.comfy.org", api_key="abc") + assert r.is_cloud is True + + def test_partner_key_does_not_pollute_extra_data(self): + r = ComfyRunner(host="https://cloud.comfy.org", api_key="auth-key") + # No partner-key set → no extra_data should appear in submitted prompt + # (This is a static check; runtime check happens in submit()) + assert r.partner_key is None + + def test_url_routing_local(self): + r = ComfyRunner() + url = r._url("/prompt") + assert url == "http://127.0.0.1:8188/prompt" + + def test_url_routing_cloud(self): + r = ComfyRunner(host="https://cloud.comfy.org", api_key="x") + url = r._url("/prompt") + assert url == "https://cloud.comfy.org/api/prompt" + + def test_url_routing_cloud_history_renamed(self): + r = ComfyRunner(host="https://cloud.comfy.org", api_key="x") + url = r._url("/history/abc-123") + assert url == "https://cloud.comfy.org/api/history_v2/abc-123" diff --git a/image-creation/comfyui/workflows/README.md b/image-creation/comfyui/workflows/README.md new file mode 100644 index 0000000..f3f40c2 --- /dev/null +++ b/image-creation/comfyui/workflows/README.md @@ -0,0 +1,86 @@ +# Example Workflows + +These are starter API-format workflows for the most common tasks. They're +ready to run with `scripts/run_workflow.py` once you've installed (or have +cloud access to) the listed models. + +| File | Purpose | Required models | Min VRAM | +|------|---------|-----------------|----------| +| `sd15_txt2img.json` | SD 1.5 text-to-image (512×512) | SD1.5 checkpoint, e.g. `v1-5-pruned-emaonly.safetensors` | 4 GB | +| `sdxl_txt2img.json` | SDXL text-to-image (1024×1024) | `sd_xl_base_1.0.safetensors` | 8 GB | +| `flux_dev_txt2img.json` | Flux Dev text-to-image (1024×1024) | `flux1-dev.safetensors`, `t5xxl_fp16.safetensors`, `clip_l.safetensors`, `ae.safetensors` | 24 GB (or use `flux1-dev-fp8`) | +| `sdxl_img2img.json` | SDXL image-to-image | SDXL checkpoint | 8 GB | +| `sdxl_inpaint.json` | SDXL inpainting (image + mask) | SDXL checkpoint | 8 GB | +| `upscale_4x.json` | Standalone 4× ESRGAN upscale | `4x-UltraSharp.pth` (or any upscaler) | 4 GB | +| `animatediff_video.json` | AnimateDiff text-to-video (16 frames) | SD1.5 checkpoint, `mm_sd_v15_v2.ckpt` motion module | 8 GB | +| `wan_video_t2v.json` | Wan 2.x text-to-video (~33 frames) | `wan2.2_t2v_1.3B_fp16.safetensors`, `umt5_xxl_fp16.safetensors`, `wan_2.1_vae.safetensors` | 24 GB | + +## Quick start + +```bash +# Run a workflow with prompt injection +python3 ../scripts/run_workflow.py \ + --workflow sdxl_txt2img.json \ + --args '{"prompt": "majestic eagle in flight", "seed": 12345, "steps": 35}' \ + --output-dir ./out + +# Img2img: upload an input image first via the script's helper +python3 ../scripts/run_workflow.py \ + --workflow sdxl_img2img.json \ + --input-image image=./photo.png \ + --args '{"prompt": "make it watercolor", "denoise": 0.6}' \ + --output-dir ./out + +# Cloud (set API key once) +export COMFY_CLOUD_API_KEY="comfyui-..." +python3 ../scripts/run_workflow.py \ + --workflow flux_dev_txt2img.json \ + --args '{"prompt": "a fox in a misty forest"}' \ + --host https://cloud.comfy.org \ + --output-dir ./out + +# What can I tweak in this workflow? +python3 ../scripts/extract_schema.py sdxl_txt2img.json --summary-only + +# Are all required models / nodes installed? +python3 ../scripts/check_deps.py wan_video_t2v.json +``` + +## Notes + +- **Inpaint masks**: white pixels = "regenerate this region", black = preserve. + ComfyUI's `LoadImageMask` reads the **red channel** by default; export your + mask as a single-channel image or as a normal RGB where red==intensity. + +- **Denoise strength** in img2img: `0.0` = output identical to input, + `1.0` = ignore input entirely. Sweet spot is usually 0.4–0.7. + +- **Flux Dev** needs ~24 GB VRAM in its base form. The `flux1-dev-fp8.safetensors` + variant (already on Comfy Cloud) cuts that roughly in half. + +- **Video workflows** can take many minutes. The skill auto-detects video + output nodes and bumps the default timeout to 900s. Override with `--timeout 1800`. + +- These JSON files are deliberately **API format** (top-level keys are node IDs + with `class_type`), not editor format. To open them in ComfyUI's web UI for + visual editing, use `Workflow → Load (API Format)` or `Workflow → Open` and + follow the prompt. + +## Cloud vs local model names + +Comfy Cloud's preinstalled checkpoints sometimes have a `-fp16` suffix +(`v1-5-pruned-emaonly-fp16.safetensors`) while the canonical local download +keeps the original name (`v1-5-pruned-emaonly.safetensors`). The example +workflows use the local-canonical names. When running on cloud, override with: + +```bash +python3 ../scripts/run_workflow.py \ + --workflow sd15_txt2img.json \ + --args '{"ckpt_name": "v1-5-pruned-emaonly-fp16.safetensors", "prompt": "..."}' \ + --host https://cloud.comfy.org +``` + +The `ckpt_name`, `vae_name`, `lora_name`, `unet_name`, etc. are all exposed +as controllable parameters by `extract_schema.py` — discover what's installed +with `comfy model list` (local) or `curl /api/experiment/models/checkpoints` +(cloud). diff --git a/image-creation/comfyui/workflows/animatediff_video.json b/image-creation/comfyui/workflows/animatediff_video.json new file mode 100644 index 0000000..cc2b296 --- /dev/null +++ b/image-creation/comfyui/workflows/animatediff_video.json @@ -0,0 +1,64 @@ +{ + "_comment": "AnimateDiff text-to-video at 16 frames. Required: comfyui-animatediff-evolved + comfyui-videohelpersuite custom nodes; SD1.5 checkpoint; AnimateDiff motion module (e.g. mm_sd_v15_v2.ckpt in models/animatediff_models/). Outputs a webp animation.", + "3": { + "class_type": "KSampler", + "_meta": {"title": "KSampler"}, + "inputs": { + "seed": 42, "steps": 25, "cfg": 7.5, + "sampler_name": "dpmpp_sde", "scheduler": "karras", "denoise": 1.0, + "model": ["10", 0], + "positive": ["6", 0], + "negative": ["7", 0], + "latent_image": ["5", 0] + } + }, + "4": { + "class_type": "CheckpointLoaderSimple", + "_meta": {"title": "Checkpoint"}, + "inputs": {"ckpt_name": "v1-5-pruned-emaonly.safetensors"} + }, + "5": { + "class_type": "EmptyLatentImage", + "_meta": {"title": "Latent (16 frames)"}, + "inputs": {"width": 512, "height": 512, "batch_size": 16} + }, + "6": { + "class_type": "CLIPTextEncode", + "_meta": {"title": "Positive Prompt"}, + "inputs": {"text": "a hot air balloon drifting over a mountain valley, sunset, cinematic", "clip": ["4", 1]} + }, + "7": { + "class_type": "CLIPTextEncode", + "_meta": {"title": "Negative Prompt"}, + "inputs": {"text": "low quality, blurry, deformed, watermark", "clip": ["4", 1]} + }, + "8": { + "class_type": "VAEDecode", + "_meta": {"title": "VAE Decode"}, + "inputs": {"samples": ["3", 0], "vae": ["4", 2]} + }, + "9": { + "class_type": "VHS_VideoCombine", + "_meta": {"title": "Video Combine"}, + "inputs": { + "frame_rate": 8.0, + "loop_count": 0, + "filename_prefix": "animatediff", + "format": "video/h264-mp4", + "pingpong": false, + "save_output": true, + "images": ["8", 0] + } + }, + "10": { + "class_type": "ADE_AnimateDiffLoaderWithContext", + "_meta": {"title": "AnimateDiff Loader"}, + "inputs": { + "model": ["4", 0], + "model_name": "mm_sd_v15_v2.ckpt", + "beta_schedule": "sqrt_linear (AnimateDiff)", + "motion_scale": 1.0, + "apply_v2_models_properly": true + } + } +} diff --git a/image-creation/comfyui/workflows/flux_dev_txt2img.json b/image-creation/comfyui/workflows/flux_dev_txt2img.json new file mode 100644 index 0000000..1791280 --- /dev/null +++ b/image-creation/comfyui/workflows/flux_dev_txt2img.json @@ -0,0 +1,78 @@ +{ + "_comment": "Flux Dev text-to-image using the modern sampler chain (BasicScheduler/Guider/SamplerCustomAdvanced). Required: flux1-dev.safetensors (UNET), t5xxl_fp16.safetensors + clip_l.safetensors (CLIP), ae.safetensors (VAE).", + "6": { + "class_type": "CLIPTextEncode", + "_meta": {"title": "Prompt"}, + "inputs": {"text": "a serene mountain landscape at golden hour, photorealistic", "clip": ["11", 0]} + }, + "8": { + "class_type": "VAEDecode", + "_meta": {"title": "VAE Decode"}, + "inputs": {"samples": ["13", 0], "vae": ["10", 0]} + }, + "9": { + "class_type": "SaveImage", + "_meta": {"title": "Save Image"}, + "inputs": {"filename_prefix": "flux_dev", "images": ["8", 0]} + }, + "10": { + "class_type": "VAELoader", + "_meta": {"title": "VAE"}, + "inputs": {"vae_name": "ae.safetensors"} + }, + "11": { + "class_type": "DualCLIPLoader", + "_meta": {"title": "DualCLIPLoader"}, + "inputs": { + "clip_name1": "t5xxl_fp16.safetensors", + "clip_name2": "clip_l.safetensors", + "type": "flux" + } + }, + "12": { + "class_type": "UNETLoader", + "_meta": {"title": "UNET Loader"}, + "inputs": {"unet_name": "flux1-dev.safetensors", "weight_dtype": "default"} + }, + "13": { + "class_type": "SamplerCustomAdvanced", + "_meta": {"title": "Sampler Custom"}, + "inputs": { + "noise": ["25", 0], + "guider": ["22", 0], + "sampler": ["16", 0], + "sigmas": ["17", 0], + "latent_image": ["27", 0] + } + }, + "16": { + "class_type": "KSamplerSelect", + "_meta": {"title": "Sampler Select"}, + "inputs": {"sampler_name": "euler"} + }, + "17": { + "class_type": "BasicScheduler", + "_meta": {"title": "Scheduler"}, + "inputs": { + "scheduler": "simple", + "steps": 20, + "denoise": 1.0, + "model": ["12", 0] + } + }, + "22": { + "class_type": "BasicGuider", + "_meta": {"title": "Guider"}, + "inputs": {"model": ["12", 0], "conditioning": ["6", 0]} + }, + "25": { + "class_type": "RandomNoise", + "_meta": {"title": "Noise"}, + "inputs": {"noise_seed": 42} + }, + "27": { + "class_type": "EmptySD3LatentImage", + "_meta": {"title": "Latent"}, + "inputs": {"width": 1024, "height": 1024, "batch_size": 1} + } +} diff --git a/image-creation/comfyui/workflows/sd15_txt2img.json b/image-creation/comfyui/workflows/sd15_txt2img.json new file mode 100644 index 0000000..f67eb79 --- /dev/null +++ b/image-creation/comfyui/workflows/sd15_txt2img.json @@ -0,0 +1,49 @@ +{ + "_comment": "SD 1.5 text-to-image. Smallest model, fastest. Required model: v1-5-pruned-emaonly.safetensors (or any SD1.5 checkpoint)", + "3": { + "class_type": "KSampler", + "_meta": {"title": "KSampler"}, + "inputs": { + "seed": 156680208700286, + "steps": 20, + "cfg": 8.0, + "sampler_name": "euler", + "scheduler": "normal", + "denoise": 1.0, + "model": ["4", 0], + "positive": ["6", 0], + "negative": ["7", 0], + "latent_image": ["5", 0] + } + }, + "4": { + "class_type": "CheckpointLoaderSimple", + "_meta": {"title": "Load Checkpoint"}, + "inputs": {"ckpt_name": "v1-5-pruned-emaonly.safetensors"} + }, + "5": { + "class_type": "EmptyLatentImage", + "_meta": {"title": "Empty Latent"}, + "inputs": {"width": 512, "height": 512, "batch_size": 1} + }, + "6": { + "class_type": "CLIPTextEncode", + "_meta": {"title": "Positive Prompt"}, + "inputs": {"text": "a beautiful landscape painting, masterpiece, highly detailed", "clip": ["4", 1]} + }, + "7": { + "class_type": "CLIPTextEncode", + "_meta": {"title": "Negative Prompt"}, + "inputs": {"text": "ugly, blurry, low quality, deformed", "clip": ["4", 1]} + }, + "8": { + "class_type": "VAEDecode", + "_meta": {"title": "VAE Decode"}, + "inputs": {"samples": ["3", 0], "vae": ["4", 2]} + }, + "9": { + "class_type": "SaveImage", + "_meta": {"title": "Save Image"}, + "inputs": {"filename_prefix": "sd15", "images": ["8", 0]} + } +} diff --git a/image-creation/comfyui/workflows/sdxl_img2img.json b/image-creation/comfyui/workflows/sdxl_img2img.json new file mode 100644 index 0000000..a835567 --- /dev/null +++ b/image-creation/comfyui/workflows/sdxl_img2img.json @@ -0,0 +1,54 @@ +{ + "_comment": "SDXL img2img: load an input image, encode to latent, denoise partially. Use --input-image image=./photo.png with run_workflow.py. Lower 'denoise' value preserves more of the source image.", + "1": { + "class_type": "LoadImage", + "_meta": {"title": "Load Source Image"}, + "inputs": {"image": "REPLACE_WITH_UPLOADED_FILENAME.png"} + }, + "3": { + "class_type": "KSampler", + "_meta": {"title": "KSampler"}, + "inputs": { + "seed": 42, + "steps": 30, + "cfg": 7.5, + "sampler_name": "dpmpp_2m", + "scheduler": "karras", + "denoise": 0.65, + "model": ["4", 0], + "positive": ["6", 0], + "negative": ["7", 0], + "latent_image": ["12", 0] + } + }, + "4": { + "class_type": "CheckpointLoaderSimple", + "_meta": {"title": "Load SDXL Base"}, + "inputs": {"ckpt_name": "sd_xl_base_1.0.safetensors"} + }, + "6": { + "class_type": "CLIPTextEncode", + "_meta": {"title": "Positive Prompt"}, + "inputs": {"text": "make it cyberpunk, neon lights, futuristic", "clip": ["4", 1]} + }, + "7": { + "class_type": "CLIPTextEncode", + "_meta": {"title": "Negative Prompt"}, + "inputs": {"text": "ugly, blurry, low quality, deformed", "clip": ["4", 1]} + }, + "8": { + "class_type": "VAEDecode", + "_meta": {"title": "VAE Decode"}, + "inputs": {"samples": ["3", 0], "vae": ["4", 2]} + }, + "9": { + "class_type": "SaveImage", + "_meta": {"title": "Save Image"}, + "inputs": {"filename_prefix": "sdxl_img2img", "images": ["8", 0]} + }, + "12": { + "class_type": "VAEEncode", + "_meta": {"title": "VAE Encode"}, + "inputs": {"pixels": ["1", 0], "vae": ["4", 2]} + } +} diff --git a/image-creation/comfyui/workflows/sdxl_inpaint.json b/image-creation/comfyui/workflows/sdxl_inpaint.json new file mode 100644 index 0000000..20e50cc --- /dev/null +++ b/image-creation/comfyui/workflows/sdxl_inpaint.json @@ -0,0 +1,59 @@ +{ + "_comment": "SDXL inpainting: given an image + mask, regenerate the masked region. Upload both: --input-image image=./photo.png --input-image mask_image=./mask.png. White pixels in mask = regenerate; black = preserve.", + "1": { + "class_type": "LoadImage", + "_meta": {"title": "Load Source"}, + "inputs": {"image": "REPLACE_WITH_UPLOADED_FILENAME.png"} + }, + "2": { + "class_type": "LoadImageMask", + "_meta": {"title": "Load Mask"}, + "inputs": {"image": "REPLACE_WITH_UPLOADED_MASK.png", "channel": "red"} + }, + "3": { + "class_type": "KSampler", + "_meta": {"title": "KSampler"}, + "inputs": { + "seed": 42, + "steps": 30, + "cfg": 7.5, + "sampler_name": "dpmpp_2m", + "scheduler": "karras", + "denoise": 1.0, + "model": ["4", 0], + "positive": ["6", 0], + "negative": ["7", 0], + "latent_image": ["12", 0] + } + }, + "4": { + "class_type": "CheckpointLoaderSimple", + "_meta": {"title": "Checkpoint"}, + "inputs": {"ckpt_name": "sd_xl_base_1.0.safetensors"} + }, + "6": { + "class_type": "CLIPTextEncode", + "_meta": {"title": "Positive Prompt"}, + "inputs": {"text": "fill with blooming flowers, photorealistic", "clip": ["4", 1]} + }, + "7": { + "class_type": "CLIPTextEncode", + "_meta": {"title": "Negative Prompt"}, + "inputs": {"text": "ugly, blurry, deformed, bad anatomy", "clip": ["4", 1]} + }, + "8": { + "class_type": "VAEDecode", + "_meta": {"title": "VAE Decode"}, + "inputs": {"samples": ["3", 0], "vae": ["4", 2]} + }, + "9": { + "class_type": "SaveImage", + "_meta": {"title": "Save"}, + "inputs": {"filename_prefix": "sdxl_inpaint", "images": ["8", 0]} + }, + "12": { + "class_type": "VAEEncodeForInpaint", + "_meta": {"title": "VAE Encode for Inpaint"}, + "inputs": {"pixels": ["1", 0], "mask": ["2", 0], "vae": ["4", 2], "grow_mask_by": 6} + } +} diff --git a/image-creation/comfyui/workflows/sdxl_txt2img.json b/image-creation/comfyui/workflows/sdxl_txt2img.json new file mode 100644 index 0000000..cb590b4 --- /dev/null +++ b/image-creation/comfyui/workflows/sdxl_txt2img.json @@ -0,0 +1,49 @@ +{ + "_comment": "SDXL text-to-image at 1024x1024. Required model: sd_xl_base_1.0.safetensors (or any SDXL checkpoint).", + "3": { + "class_type": "KSampler", + "_meta": {"title": "KSampler"}, + "inputs": { + "seed": 42, + "steps": 30, + "cfg": 7.5, + "sampler_name": "dpmpp_2m", + "scheduler": "karras", + "denoise": 1.0, + "model": ["4", 0], + "positive": ["6", 0], + "negative": ["7", 0], + "latent_image": ["5", 0] + } + }, + "4": { + "class_type": "CheckpointLoaderSimple", + "_meta": {"title": "Load SDXL Base"}, + "inputs": {"ckpt_name": "sd_xl_base_1.0.safetensors"} + }, + "5": { + "class_type": "EmptyLatentImage", + "_meta": {"title": "Empty Latent"}, + "inputs": {"width": 1024, "height": 1024, "batch_size": 1} + }, + "6": { + "class_type": "CLIPTextEncode", + "_meta": {"title": "Positive Prompt"}, + "inputs": {"text": "cinematic photograph, dramatic lighting, intricate detail", "clip": ["4", 1]} + }, + "7": { + "class_type": "CLIPTextEncode", + "_meta": {"title": "Negative Prompt"}, + "inputs": {"text": "ugly, blurry, low quality, deformed, watermark", "clip": ["4", 1]} + }, + "8": { + "class_type": "VAEDecode", + "_meta": {"title": "VAE Decode"}, + "inputs": {"samples": ["3", 0], "vae": ["4", 2]} + }, + "9": { + "class_type": "SaveImage", + "_meta": {"title": "Save Image"}, + "inputs": {"filename_prefix": "sdxl", "images": ["8", 0]} + } +} diff --git a/image-creation/comfyui/workflows/upscale_4x.json b/image-creation/comfyui/workflows/upscale_4x.json new file mode 100644 index 0000000..91ad7eb --- /dev/null +++ b/image-creation/comfyui/workflows/upscale_4x.json @@ -0,0 +1,27 @@ +{ + "_comment": "Standalone 4x upscale of an input image using ESRGAN. Required model: 4x-UltraSharp.pth (or any upscaler in models/upscale_models/). Upload with --input-image image=./photo.png.", + "1": { + "class_type": "LoadImage", + "_meta": {"title": "Load Image"}, + "inputs": {"image": "REPLACE_WITH_UPLOADED_FILENAME.png"} + }, + "2": { + "class_type": "UpscaleModelLoader", + "_meta": {"title": "Load Upscale Model"}, + "inputs": {"model_name": "4x-UltraSharp.pth"} + }, + "3": { + "class_type": "ImageUpscaleWithModel", + "_meta": {"title": "Upscale Image (with Model)"}, + "inputs": { + "upscale_method": "lanczos", + "upscale_model": ["2", 0], + "image": ["1", 0] + } + }, + "4": { + "class_type": "SaveImage", + "_meta": {"title": "Save"}, + "inputs": {"filename_prefix": "upscaled_4x", "images": ["3", 0]} + } +} diff --git a/image-creation/comfyui/workflows/wan_video_t2v.json b/image-creation/comfyui/workflows/wan_video_t2v.json new file mode 100644 index 0000000..7514e3a --- /dev/null +++ b/image-creation/comfyui/workflows/wan_video_t2v.json @@ -0,0 +1,69 @@ +{ + "_comment": "Wan 2.1 text-to-video. Cloud: confirmed available. Local: download wan2.1_t2v_1.3B_fp16.safetensors → models/diffusion_models/ (or models/unet/), umt5_xxl_fp16.safetensors → models/text_encoders/ (or models/clip/), wan_2.1_vae.safetensors → models/vae/. Output: MP4. Large model — only on cloud or 24 GB+ local GPU.", + "6": { + "class_type": "CLIPTextEncode", + "_meta": {"title": "Prompt"}, + "inputs": { + "text": "a graceful crane taking flight from a misty lake at dawn, slow motion, 4k", + "clip": ["38", 0] + } + }, + "7": { + "class_type": "CLIPTextEncode", + "_meta": {"title": "Negative Prompt"}, + "inputs": { + "text": "static, blurry, watermark, low quality", + "clip": ["38", 0] + } + }, + "8": { + "class_type": "VAEDecode", + "_meta": {"title": "VAE Decode"}, + "inputs": {"samples": ["3", 0], "vae": ["39", 0]} + }, + "37": { + "class_type": "UNETLoader", + "_meta": {"title": "Wan UNET"}, + "inputs": {"unet_name": "wan2.1_t2v_1.3B_fp16.safetensors", "weight_dtype": "default"} + }, + "38": { + "class_type": "CLIPLoader", + "_meta": {"title": "Wan CLIP"}, + "inputs": {"clip_name": "umt5_xxl_fp16.safetensors", "type": "wan"} + }, + "39": { + "class_type": "VAELoader", + "_meta": {"title": "Wan VAE"}, + "inputs": {"vae_name": "wan_2.1_vae.safetensors"} + }, + "3": { + "class_type": "KSampler", + "_meta": {"title": "KSampler"}, + "inputs": { + "seed": 42, "steps": 30, "cfg": 6.0, + "sampler_name": "uni_pc", "scheduler": "simple", "denoise": 1.0, + "model": ["37", 0], + "positive": ["6", 0], + "negative": ["7", 0], + "latent_image": ["40", 0] + } + }, + "40": { + "class_type": "EmptyHunyuanLatentVideo", + "_meta": {"title": "Latent Video (33 frames)"}, + "inputs": {"width": 832, "height": 480, "length": 33, "batch_size": 1} + }, + "9": { + "class_type": "VHS_VideoCombine", + "_meta": {"title": "Video Combine"}, + "inputs": { + "frame_rate": 16.0, + "loop_count": 0, + "filename_prefix": "wan_t2v", + "format": "video/h264-mp4", + "pingpong": false, + "save_output": true, + "images": ["8", 0] + } + } +}