Flatten skill category directory structure
This commit is contained in:
@@ -0,0 +1,312 @@
|
||||
---
|
||||
name: beat-sync-editor
|
||||
display-name-zh: 音乐卡点
|
||||
description: |
|
||||
音乐卡点剪辑 Skill。输入音乐(URL / 本地文件 / AI 生成)+ 视频或图片素材,
|
||||
自动完成情绪张力分析 → 智能裁剪 → 节拍分析 → 卡点时间轴生成 → 素材匹配 → ffmpeg 拼接合成,
|
||||
输出与音乐完美对齐的卡点视频成片。
|
||||
支持图片幻灯片、视频片段、混合素材三种输入,支持每 N 拍换素材、
|
||||
前奏/副歌自动分段、用户手动标注关键点等多种卡点模式。
|
||||
触发词包括:音乐卡点、卡点剪辑、beat sync、beat-sync video、
|
||||
节拍剪辑、卡点视频、音乐节奏剪辑、节奏卡点、beat detection、
|
||||
按节拍剪辑、自动卡点、music sync edit。
|
||||
summary-cn: 输入音乐和素材,自动按节拍卡点剪片
|
||||
summary-en: Auto beat-sync edit by emotion + cuts
|
||||
version: 0.2.3
|
||||
tags: [Video, Audio, Editing, Music]
|
||||
tags-cn: [视频, 音频, 剪辑, 音乐]
|
||||
trigger-words:
|
||||
- 音乐卡点
|
||||
- 卡点剪辑
|
||||
- 卡点视频
|
||||
- 节拍剪辑
|
||||
- 自动卡点
|
||||
- beat sync
|
||||
- beat-sync video
|
||||
- beat detection
|
||||
- music sync edit
|
||||
- 按节拍剪辑
|
||||
exported-by: MiniMax-hub
|
||||
---
|
||||
|
||||
# Beat-Sync Editor — 音乐卡点剪辑
|
||||
|
||||
当用户想要「把一批图片/视频素材配上音乐,按节拍自动切换」时,使用此 Skill。
|
||||
核心流程:**音乐来源确认 → 情绪张力分析 → 目标时长裁剪 → 节拍分析 → 时间轴规划 → 素材分配 → ffmpeg 拼接 → 成片**。
|
||||
|
||||
---
|
||||
|
||||
## STEP 0: 收集输入
|
||||
|
||||
确认以下三项输入,缺哪项就问用户:
|
||||
|
||||
### 0a. 音乐来源(三选一)
|
||||
|
||||
| 类型 | 操作 |
|
||||
|------|------|
|
||||
| HTTP/HTTPS URL | 用 `curl -L -o music.ext <url>` 下载到会话目录,记录本地路径 |
|
||||
| 本地文件路径 | 直接使用,调用 `save_file_to_session` 注册到会话 |
|
||||
| AI 生成音乐 | 调用 **audio agent**,告知风格/BPM/时长需求,获取生成后的本地路径 |
|
||||
|
||||
> **注意**:下载后用 `ffprobe` 验证文件时长,记录 `total_duration`(秒)。
|
||||
|
||||
### 0b. 素材列表
|
||||
|
||||
支持三种类型(可混合):
|
||||
|
||||
- **图片**(jpg/png/webp):每张图将被处理成一段静态视频片段
|
||||
- **视频片段**(mp4/mov):直接按时间轴截取或整段使用
|
||||
- **AI 生图补充**:若用户素材数量少于节拍数,告知用户并询问是否调用 image agent 自动补充生图
|
||||
|
||||
记录所有素材的本地路径列表。
|
||||
|
||||
### 0c. 卡点参数
|
||||
|
||||
询问用户(若未指定):
|
||||
|
||||
| 参数 | 默认值 | 说明 |
|
||||
|------|--------|------|
|
||||
| 检测模式 | `beat` | `beat`=标准节拍 / `onset`=音头(更敏感) / `segment`=自动分段 |
|
||||
| 每 N 拍换一次 | `1` | `2` 表示每 2 拍换一次素材,节奏感更舒缓 |
|
||||
| 手动关键点 | 无 | 用户可提供额外时间点(秒),强制在此处切换素材 |
|
||||
| 输出比例 | `9:16` | 竖屏短视频 / `16:9` 横屏 / `1:1` 方形 |
|
||||
|
||||
---
|
||||
|
||||
## STEP 0.5: 情绪张力分析 + 目标时长裁剪
|
||||
|
||||
> **触发条件**:音频时长 > 用户预期的成片时长,或用户未指定时长但音频超过 60s。
|
||||
> 若音频已经很短(≤ 30s)或用户明确说"用整首歌",跳过此步骤直接进入 STEP 1。
|
||||
|
||||
### 0.5a. 运行情绪张力分析脚本
|
||||
|
||||
使用内置脚本 `scripts/energy_analyze.py`,分析音频的能量分布,找出情绪张力最强的区段:
|
||||
|
||||
```bash
|
||||
python scripts/energy_analyze.py <音乐本地路径> \
|
||||
--targets "15,30,60" \
|
||||
--out json
|
||||
```
|
||||
|
||||
脚本输出:
|
||||
- `energy_peak`:全曲能量峰值时间点(秒)
|
||||
- `sections`:各段落的能量得分排序(高→低)
|
||||
- `trim_options`:针对每个目标时长(15s / 30s / 60s)的**最佳裁剪窗口**(start, end, energy_score, description)
|
||||
|
||||
> **脚本路径**:`scripts/energy_analyze.py`(与 SKILL.md 同目录)
|
||||
|
||||
### 0.5b. 向用户展示裁剪方案
|
||||
|
||||
将分析结果整理成选项,**用 question 组件**呈现给用户选择:
|
||||
|
||||
展示格式示例:
|
||||
```
|
||||
🎵 音频总时长:202s | BPM:143 | 情绪峰值点:87.3s
|
||||
|
||||
请选择成片时长:
|
||||
|
||||
① 15秒 ▎起 80.1s → 95.1s ▏张力 ★★★★★ 覆盖最强爆发点,情绪张力最高
|
||||
② 30秒 ▎起 72.0s → 102.0s ▏张力 ★★★★☆ 副歌完整段落
|
||||
③ 60秒 ▎起 62.0s → 122.0s ▏张力 ★★★☆☆ 含前奏引入 + 完整副歌
|
||||
④ 使用完整音频(202秒)
|
||||
⑤ 自定义时间段(手动输入 start/end)
|
||||
```
|
||||
|
||||
**能量星级换算**:energy_score ≥ 0.85 = ★★★★★,≥ 0.70 = ★★★★☆,≥ 0.55 = ★★★☆☆,其余 = ★★☆☆☆
|
||||
|
||||
**默认推荐**:高亮显示 energy_score 最高的选项(通常是时长最短的那个,因为窗口越小越容易聚焦高能区)。
|
||||
|
||||
### 0.5c. 按用户选择裁剪音频
|
||||
|
||||
用户确认后,用 ffmpeg 裁剪音频为目标片段,后续所有步骤(节拍分析、素材分配、合成)均基于裁剪后的音频:
|
||||
|
||||
```bash
|
||||
ffmpeg -i <原始音频> -ss <start> -to <end> -c copy <audio_trimmed.mp3>
|
||||
```
|
||||
|
||||
> **为什么裁剪后再做节拍分析**:节拍分析的计算量与音频时长成正比;
|
||||
> 同时裁剪后的 beat_times 直接对应最终视频的时间轴,无需再做偏移换算。
|
||||
|
||||
---
|
||||
|
||||
## STEP 1: 节拍分析
|
||||
|
||||
使用内置脚本 `scripts/beat_detect.py` 进行节拍检测:
|
||||
|
||||
```bash
|
||||
# 安装依赖(首次运行)
|
||||
pip install librosa soundfile
|
||||
|
||||
# 运行检测
|
||||
python scripts/beat_detect.py <音乐本地路径> \
|
||||
--mode <beat|onset|segment> \
|
||||
--every-n <N> \
|
||||
--manual "<t1,t2,...>" \
|
||||
--min-gap 0.25 \
|
||||
--out json > beat_result.json
|
||||
```
|
||||
|
||||
脚本输出 JSON,包含:
|
||||
- `bpm`:检测到的 BPM
|
||||
- `total_duration`:音乐总时长(秒)
|
||||
- `beat_count`:有效节拍数
|
||||
- `beat_times`:节拍时间点列表(秒)
|
||||
- `sections`(仅 segment 模式):段落分析结果
|
||||
|
||||
> **脚本路径**:`scripts/beat_detect.py`(与 SKILL.md 同目录)
|
||||
|
||||
向用户展示检测摘要:BPM、节拍数、时长,确认是否合理后继续。
|
||||
|
||||
---
|
||||
|
||||
## STEP 2: 时间轴规划
|
||||
|
||||
根据节拍分析结果,**在 orchestrator 自行计算**每个素材片段的时间区间:
|
||||
|
||||
```
|
||||
segments = []
|
||||
for i in range(len(beat_times)):
|
||||
start = beat_times[i]
|
||||
end = beat_times[i+1] if i+1 < len(beat_times) else total_duration
|
||||
duration = end - start
|
||||
segments.append({ "index": i, "start": start, "end": end, "duration": duration })
|
||||
```
|
||||
|
||||
**素材分配规则**:
|
||||
- 素材数量 ≥ 节拍数:按顺序取前 N 个,多余的丢弃
|
||||
- 素材数量 < 节拍数:**循环复用**(`material_index = segment_index % len(materials)`),并告知用户有复用
|
||||
- 若用户不接受复用:提示调用 image agent 按缺口数量补充生图
|
||||
|
||||
**segment 模式额外逻辑**:
|
||||
- 前奏/尾奏(`intro`/`outro`)节拍间隔较宽,建议每拍分配一张素材(慢切)
|
||||
- 副歌/高潮(含"chorus"的 label)节拍间隔较窄,建议保持原始节拍密度(快切)
|
||||
- 若用户提供了手动关键点,这些位置强制切换,不受 every-N 参数影响
|
||||
|
||||
向用户展示规划摘要表(前 5 行 + 总计),确认继续。
|
||||
|
||||
---
|
||||
|
||||
## STEP 3: 素材预处理
|
||||
|
||||
调用 **editing agent** 批量处理所有素材,统一规格:
|
||||
|
||||
告知 editing agent:
|
||||
|
||||
> 「请帮我处理以下 N 个素材文件,将每个文件转换为以下规格:
|
||||
> - 分辨率:[目标宽x高,根据输出比例决定,如 1080x1920]
|
||||
> - 帧率:30fps
|
||||
> - 编码:h264,像素格式 yuv420p
|
||||
> - 对于**图片**:转为 [duration]s 的静态视频(duration 来自时间轴规划)
|
||||
> - 对于**视频**:截取前 [duration]s,若片段不足则 loop 补齐或 pad 黑帧
|
||||
> - 对于**尺寸不匹配**:使用 scale+pad(保持原始比例,黑边补齐)
|
||||
> 输出文件命名:`clip_001.mp4`, `clip_002.mp4` ...」
|
||||
|
||||
**为什么预处理**:ffmpeg concat 要求所有片段分辨率、帧率、编码完全一致,否则合成会报错或产生画面撕裂。
|
||||
|
||||
---
|
||||
|
||||
## STEP 4: 确认预览(可选)
|
||||
|
||||
若用户明确要求预览,或素材数量 > 20,**在合成前**展示:
|
||||
- 时间轴规划表(序号 / 素材文件名 / 开始时间 / 时长)
|
||||
- 总成片时长预估
|
||||
|
||||
询问用户是否有调整:
|
||||
- 更换某个位置的素材
|
||||
- 调整 every-N 参数重新规划
|
||||
- 手动添加/删除关键卡点
|
||||
|
||||
如有调整,回到 STEP 2 重新规划,无需重新做节拍分析。
|
||||
|
||||
---
|
||||
|
||||
## STEP 5: ffmpeg 拼接合成
|
||||
|
||||
调用 **editing agent**,提供以下信息:
|
||||
|
||||
> 「请将以下片段按顺序拼接,并混入音轨:
|
||||
>
|
||||
> 片段列表:`clip_001.mp4`, `clip_002.mp4`, ... `clip_N.mp4`
|
||||
> (按 STEP 3 生成的文件顺序)
|
||||
>
|
||||
> 音轨:`<音乐本地路径>`
|
||||
>
|
||||
> 要求:
|
||||
> - 使用 ffmpeg concat(不转码,直接流合并)
|
||||
> - 音频使用原始音乐,视频合成后去掉所有素材的原始音轨(-an),避免叠音
|
||||
> - 音乐若长于视频则截断(`-shortest`),若短于视频则循环(`-stream_loop -1`)
|
||||
> - 输出文件:`beat_sync_output.mp4`
|
||||
> - 编码:h264 + aac,质量 CRF 18」
|
||||
|
||||
**为什么去掉素材原始音轨**:合成步骤会将音乐轨叠加进来,若保留素材原音会产生双重音频。
|
||||
|
||||
---
|
||||
|
||||
## STEP 6: 展示成片
|
||||
|
||||
向用户展示 `beat_sync_output.mp4`,并附上摘要:
|
||||
|
||||
```
|
||||
✅ 卡点视频生成完成
|
||||
|
||||
🎵 音乐:<文件名> | BPM: <bpm>
|
||||
🎬 素材:<N> 个(图片 X 张 + 视频 Y 段)
|
||||
⏱ 成片时长:<duration>s
|
||||
📐 输出比例:<ratio>
|
||||
🔄 素材是否有复用:<是/否>
|
||||
```
|
||||
|
||||
若用户对成片有以下调整需求:
|
||||
- **换素材**:更换指定位置的素材 → 回 STEP 3 只重处理该素材,再回 STEP 5 合成
|
||||
- **调节奏**(改 every-N)→ 回 STEP 2 重新规划
|
||||
- **换音乐**:全流程重来(STEP 1 起)
|
||||
- **加转场效果**(淡入淡出等):告知 editing agent 在合成时对每个片段加 fade filter
|
||||
|
||||
---
|
||||
|
||||
## 技术说明
|
||||
|
||||
### 依赖安装
|
||||
```bash
|
||||
pip install librosa soundfile # 两个脚本共用
|
||||
```
|
||||
|
||||
### energy_analyze.py 脚本参数速查
|
||||
|
||||
| 参数 | 说明 | 示例 |
|
||||
|------|------|------|
|
||||
| `--targets "15,30,60"` | 要计算的目标时长列表(秒) | 根据场景调整,如 `"30,45,90"` |
|
||||
| `--out json` | 输出格式(json / text) | text 适合快速调试 |
|
||||
|
||||
**能量分析维度**(加权合成):
|
||||
- 响度(RMS)× 0.5 — 最直接反映音量强弱
|
||||
- 节拍冲击强度(Onset Strength)× 0.3 — 反映鼓点/节拍爆发感
|
||||
- 音色明亮度(Spectral Centroid)× 0.2 — 高能段通常高频更丰富
|
||||
|
||||
### beat_detect.py 脚本参数速查
|
||||
|
||||
| 参数 | 说明 | 示例 |
|
||||
|------|------|------|
|
||||
| `--mode beat` | 标准节拍跟踪(默认) | 流行/电子音乐 |
|
||||
| `--mode onset` | 音头检测,对打击乐更敏感 | 说唱/鼓点密集曲目 |
|
||||
| `--mode segment` | 分段分析,自动识别前奏/副歌 | 有明显段落的流行曲 |
|
||||
| `--every-n 2` | 每 2 拍输出一个切点 | 素材少、节奏舒缓时 |
|
||||
| `--manual "8.5,32.0"` | 强制在 8.5s 和 32.0s 处加切点 | 用户有特定关键帧需求 |
|
||||
| `--min-gap 0.5` | 相邻切点最少间隔 0.5s | 防止切换过快 |
|
||||
|
||||
### 常见问题
|
||||
|
||||
**Q: 情绪分析找到的"高能段"感觉不准?**
|
||||
→ 可能是弦乐/人声主导的歌曲,能量特征不如打击乐明显。让用户选"自定义时间段"手动输入。
|
||||
|
||||
**Q: 检测到的节拍太多,画面切换太快?**
|
||||
→ 增大 `--every-n`,或增大 `--min-gap`,或换 `segment` 模式
|
||||
|
||||
**Q: 某段音乐节拍检测不准(如弦乐、人声段落)?**
|
||||
→ 改用 `--mode onset`,或让用户手动提供关键时间点 `--manual`
|
||||
|
||||
**Q: 素材数量远少于节拍数?**
|
||||
→ 默认循环复用;或询问用户是否调用 image agent 按差额数量 AI 生图
|
||||
|
||||
**Q: ffmpeg 合成报错"Stream codec parameters not set"?**
|
||||
→ STEP 3 预处理中可能有文件转换失败,让 editing agent 检查并重新处理失败的素材
|
||||
@@ -0,0 +1,8 @@
|
||||
音乐卡点剪辑 Skill。输入音乐(URL / 本地文件 / AI 生成)+ 视频或图片素材,
|
||||
自动完成情绪张力分析 → 智能裁剪 → 节拍分析 → 卡点时间轴生成 → 素材匹配 → ffmpeg 拼接合成,
|
||||
输出与音乐完美对齐的卡点视频成片。
|
||||
支持图片幻灯片、视频片段、混合素材三种输入,支持每 N 拍换素材、
|
||||
前奏/副歌自动分段、用户手动标注关键点等多种卡点模式。
|
||||
触发词包括:音乐卡点、卡点剪辑、beat sync、beat-sync video、
|
||||
节拍剪辑、卡点视频、音乐节奏剪辑、节奏卡点、beat detection、
|
||||
按节拍剪辑、自动卡点、music sync edit。
|
||||
@@ -0,0 +1,147 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
beat_detect.py — librosa-based beat detection for beat-sync-editor skill
|
||||
|
||||
Usage:
|
||||
python beat_detect.py <audio_file> [options]
|
||||
|
||||
Options:
|
||||
--mode Detection mode: "beat" (default) | "onset" | "segment"
|
||||
--every-n Only emit every N-th beat (default: 1)
|
||||
--manual Comma-separated additional timestamps in seconds to merge in
|
||||
--min-gap Minimum gap between beats in seconds (default: 0.25)
|
||||
--out Output format: "json" (default) | "text"
|
||||
|
||||
Output (JSON):
|
||||
{
|
||||
"bpm": 128.0,
|
||||
"total_duration": 210.5,
|
||||
"beat_count": 512,
|
||||
"beat_times": [0.46, 0.93, 1.40, ...], # seconds
|
||||
"sections": [ # only in "segment" mode
|
||||
{"label": "intro", "start": 0.0, "end": 32.0, "beats": [...]},
|
||||
{"label": "chorus", "start": 32.0, "end": 96.0, "beats": [...]}
|
||||
]
|
||||
}
|
||||
"""
|
||||
|
||||
import sys
|
||||
import json
|
||||
import argparse
|
||||
|
||||
|
||||
def detect_beats(audio_path, mode="beat", every_n=1,
|
||||
manual_times=None, min_gap=0.25):
|
||||
try:
|
||||
import librosa
|
||||
import numpy as np
|
||||
except ImportError:
|
||||
print("ERROR: librosa not installed. Run: pip install librosa", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
y, sr = librosa.load(audio_path, mono=True)
|
||||
total_duration = librosa.get_duration(y=y, sr=sr)
|
||||
|
||||
if mode == "onset":
|
||||
# Onset detection — more sensitive to transients, great for percussion
|
||||
onset_frames = librosa.onset.onset_detect(y=y, sr=sr, units="frames")
|
||||
beat_times = librosa.frames_to_time(onset_frames, sr=sr).tolist()
|
||||
tempo = float(librosa.beat.tempo(y=y, sr=sr)[0])
|
||||
elif mode == "segment":
|
||||
# Structural segmentation via novelty curve + beat tracking
|
||||
tempo, beat_frames = librosa.beat.beat_track(y=y, sr=sr)
|
||||
beat_times = librosa.frames_to_time(beat_frames, sr=sr).tolist()
|
||||
tempo = float(tempo)
|
||||
# Segment using spectral flux novelty
|
||||
try:
|
||||
chroma = librosa.feature.chroma_cqt(y=y, sr=sr)
|
||||
bounds = librosa.segment.agglomerative(chroma, k=8)
|
||||
bound_times = librosa.frames_to_time(bounds, sr=sr).tolist()
|
||||
except Exception:
|
||||
bound_times = [total_duration * 0.15, total_duration * 0.5]
|
||||
else:
|
||||
# Default: standard beat tracking
|
||||
tempo, beat_frames = librosa.beat.beat_track(y=y, sr=sr)
|
||||
beat_times = librosa.frames_to_time(beat_frames, sr=sr).tolist()
|
||||
tempo = float(np.asarray(tempo).flat[0])
|
||||
|
||||
# Apply every-N filter
|
||||
if every_n > 1:
|
||||
beat_times = beat_times[::every_n]
|
||||
|
||||
# Merge manual timestamps
|
||||
if manual_times:
|
||||
beat_times = sorted(set(beat_times) | set(manual_times))
|
||||
|
||||
# Enforce minimum gap between beats
|
||||
if min_gap > 0:
|
||||
filtered = []
|
||||
last = -999.0
|
||||
for t in sorted(beat_times):
|
||||
if t - last >= min_gap:
|
||||
filtered.append(round(t, 4))
|
||||
last = t
|
||||
beat_times = filtered
|
||||
|
||||
result = {
|
||||
"bpm": round(tempo, 2),
|
||||
"total_duration": round(total_duration, 4),
|
||||
"beat_count": len(beat_times),
|
||||
"beat_times": [round(t, 4) for t in beat_times],
|
||||
}
|
||||
|
||||
if mode == "segment":
|
||||
sections = []
|
||||
boundaries = [0.0] + bound_times + [total_duration]
|
||||
labels = ["intro"] + [f"section_{i}" for i in range(1, len(boundaries) - 2)] + ["outro"]
|
||||
for i in range(len(boundaries) - 1):
|
||||
s, e = boundaries[i], boundaries[i + 1]
|
||||
section_beats = [t for t in beat_times if s <= t < e]
|
||||
sections.append({
|
||||
"label": labels[i] if i < len(labels) else f"section_{i}",
|
||||
"start": round(s, 4),
|
||||
"end": round(e, 4),
|
||||
"beats": section_beats
|
||||
})
|
||||
result["sections"] = sections
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Beat detection for beat-sync-editor")
|
||||
parser.add_argument("audio_file", help="Path to audio file")
|
||||
parser.add_argument("--mode", choices=["beat", "onset", "segment"], default="beat")
|
||||
parser.add_argument("--every-n", type=int, default=1, dest="every_n")
|
||||
parser.add_argument("--manual", type=str, default="", help="Comma-separated extra timestamps")
|
||||
parser.add_argument("--min-gap", type=float, default=0.25, dest="min_gap")
|
||||
parser.add_argument("--out", choices=["json", "text"], default="json")
|
||||
args = parser.parse_args()
|
||||
|
||||
manual_times = []
|
||||
if args.manual:
|
||||
try:
|
||||
manual_times = [float(x.strip()) for x in args.manual.split(",") if x.strip()]
|
||||
except ValueError:
|
||||
print("ERROR: --manual must be comma-separated numbers (e.g. '8.5,32.0')", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
result = detect_beats(
|
||||
audio_path=args.audio_file,
|
||||
mode=args.mode,
|
||||
every_n=args.every_n,
|
||||
manual_times=manual_times,
|
||||
min_gap=args.min_gap,
|
||||
)
|
||||
|
||||
if args.out == "text":
|
||||
print(f"BPM: {result['bpm']}")
|
||||
print(f"Duration: {result['total_duration']}s")
|
||||
print(f"Beat count: {result['beat_count']}")
|
||||
print("Beat times (s):", " ".join(str(t) for t in result["beat_times"]))
|
||||
else:
|
||||
print(json.dumps(result, ensure_ascii=False, indent=2))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,251 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
energy_analyze.py — 音频情绪张力分析,找出能量最高的片段
|
||||
|
||||
用于 beat-sync-editor skill 的音频预处理步骤:
|
||||
识别音频中情绪张力最强的区域,并计算若干目标时长下的最佳裁剪窗口。
|
||||
|
||||
Usage:
|
||||
python energy_analyze.py <audio_file> [options]
|
||||
|
||||
Options:
|
||||
--targets 目标时长列表(秒),逗号分隔,默认 "15,30,60"
|
||||
--top-n 返回前 N 个候选区间(默认 1)
|
||||
--out 输出格式:json(默认)| text
|
||||
|
||||
Output (JSON):
|
||||
{
|
||||
"total_duration": 202.08,
|
||||
"bpm": 143.55,
|
||||
"energy_peak": 87.3, # 能量最高点时间(秒)
|
||||
"sections": [ # 能量分段(高→低排序)
|
||||
{"start": 72.0, "end": 136.0, "energy_score": 0.94, "label": "高能段"},
|
||||
...
|
||||
],
|
||||
"trim_options": [ # 各目标时长的最佳裁剪方案
|
||||
{
|
||||
"target_duration": 15,
|
||||
"start": 80.1,
|
||||
"end": 95.1,
|
||||
"energy_score": 0.97,
|
||||
"description": "最强节拍爆发区(副歌核心)"
|
||||
},
|
||||
{
|
||||
"target_duration": 30,
|
||||
"start": 72.0,
|
||||
"end": 102.0,
|
||||
"energy_score": 0.95,
|
||||
"description": "副歌完整段落"
|
||||
},
|
||||
...
|
||||
]
|
||||
}
|
||||
"""
|
||||
|
||||
import sys
|
||||
import json
|
||||
import argparse
|
||||
|
||||
|
||||
def analyze_energy(audio_path, target_durations=None, top_n=1):
|
||||
try:
|
||||
import librosa
|
||||
import numpy as np
|
||||
except ImportError:
|
||||
print("ERROR: librosa not installed. Run: pip install librosa", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
if target_durations is None:
|
||||
target_durations = [15, 30, 60]
|
||||
|
||||
# 加载音频
|
||||
y, sr = librosa.load(audio_path, mono=True)
|
||||
total_duration = librosa.get_duration(y=y, sr=sr)
|
||||
|
||||
# ── 1. 多维度能量特征 ──────────────────────────────────────────────
|
||||
hop_length = 512
|
||||
|
||||
# RMS 能量(响度)
|
||||
rms = librosa.feature.rms(y=y, hop_length=hop_length)[0]
|
||||
|
||||
# 频谱质心(音色明亮度)—— 高能量段通常质心更高
|
||||
centroid = librosa.feature.spectral_centroid(y=y, sr=sr, hop_length=hop_length)[0]
|
||||
|
||||
# Onset 强度(节拍冲击感)
|
||||
onset_env = librosa.onset.onset_strength(y=y, sr=sr, hop_length=hop_length)
|
||||
|
||||
# 归一化各特征到 [0, 1]
|
||||
def norm(x):
|
||||
mn, mx = x.min(), x.max()
|
||||
return (x - mn) / (mx - mn + 1e-8)
|
||||
|
||||
rms_n = norm(rms)
|
||||
cent_n = norm(centroid)
|
||||
onset_n = norm(onset_env)
|
||||
|
||||
# 综合情绪张力分数(加权求和)
|
||||
# 响度权重最高,节拍冲击其次,音色明亮度辅助
|
||||
energy_score = 0.5 * rms_n + 0.3 * onset_n + 0.2 * cent_n
|
||||
|
||||
# 帧时间轴
|
||||
frame_times = librosa.frames_to_time(
|
||||
np.arange(len(energy_score)), sr=sr, hop_length=hop_length
|
||||
)
|
||||
|
||||
# ── 2. 找能量峰值点 ───────────────────────────────────────────────
|
||||
peak_frame = int(np.argmax(energy_score))
|
||||
energy_peak = float(frame_times[peak_frame])
|
||||
|
||||
# ── 3. 结构分段(用频谱聚类识别段落边界)─────────────────────────
|
||||
try:
|
||||
# 使用 MFCC 做结构分析
|
||||
mfcc = librosa.feature.mfcc(y=y, sr=sr, n_mfcc=12, hop_length=hop_length)
|
||||
# 检测边界
|
||||
bounds = librosa.segment.agglomerative(mfcc, k=min(8, int(total_duration / 20)))
|
||||
bound_times = librosa.frames_to_time(bounds, sr=sr, hop_length=hop_length).tolist()
|
||||
# 加首尾
|
||||
segment_boundaries = sorted(set([0.0] + bound_times + [total_duration]))
|
||||
except Exception:
|
||||
# 降级:按 20s 均匀分段
|
||||
n_segs = max(3, int(total_duration / 20))
|
||||
segment_boundaries = [i * total_duration / n_segs for i in range(n_segs + 1)]
|
||||
|
||||
# 计算每个段落的平均能量分数
|
||||
sections_raw = []
|
||||
for i in range(len(segment_boundaries) - 1):
|
||||
seg_start = segment_boundaries[i]
|
||||
seg_end = segment_boundaries[i + 1]
|
||||
if seg_end - seg_start < 2.0:
|
||||
continue
|
||||
# 找该时间范围内的帧
|
||||
mask = (frame_times >= seg_start) & (frame_times < seg_end)
|
||||
if mask.sum() == 0:
|
||||
continue
|
||||
seg_score = float(energy_score[mask].mean())
|
||||
sections_raw.append({
|
||||
"start": round(seg_start, 2),
|
||||
"end": round(seg_end, 2),
|
||||
"energy_score": round(seg_score, 4),
|
||||
})
|
||||
|
||||
# 按能量排序,加标签
|
||||
sections_raw.sort(key=lambda x: x["energy_score"], reverse=True)
|
||||
labels = ["高能段(副歌/高潮)", "次高能段", "中能段", "低能段(前奏/间奏)"]
|
||||
sections = []
|
||||
for idx, s in enumerate(sections_raw):
|
||||
s["label"] = labels[min(idx, len(labels) - 1)]
|
||||
sections.append(s)
|
||||
|
||||
# ── 4. 为每个目标时长计算最佳裁剪窗口 ───────────────────────────
|
||||
trim_options = []
|
||||
frame_count = len(energy_score)
|
||||
|
||||
for target_dur in target_durations:
|
||||
if target_dur >= total_duration:
|
||||
# 目标时长超过音频总长,直接用全曲
|
||||
trim_options.append({
|
||||
"target_duration": target_dur,
|
||||
"start": 0.0,
|
||||
"end": round(total_duration, 2),
|
||||
"actual_duration": round(total_duration, 2),
|
||||
"energy_score": round(float(energy_score.mean()), 4),
|
||||
"description": "使用完整音频(目标时长超过总时长)"
|
||||
})
|
||||
continue
|
||||
|
||||
# 滑动窗口:找窗口内平均能量最高的起始点
|
||||
window_frames = int(target_dur * sr / hop_length)
|
||||
best_score = -1.0
|
||||
best_start_frame = 0
|
||||
|
||||
# 步长取 0.5s 对应的帧数,平衡精度与速度
|
||||
step = max(1, int(0.5 * sr / hop_length))
|
||||
|
||||
for start_f in range(0, frame_count - window_frames, step):
|
||||
end_f = start_f + window_frames
|
||||
window_score = float(energy_score[start_f:end_f].mean())
|
||||
if window_score > best_score:
|
||||
best_score = window_score
|
||||
best_start_frame = start_f
|
||||
|
||||
best_start_time = float(frame_times[best_start_frame])
|
||||
best_end_time = min(best_start_time + target_dur, total_duration)
|
||||
|
||||
# 生成描述
|
||||
# 判断该窗口是否覆盖了能量峰值
|
||||
covers_peak = best_start_time <= energy_peak <= best_end_time
|
||||
if covers_peak:
|
||||
desc = f"覆盖最强爆发点({energy_peak:.1f}s),情绪张力最高"
|
||||
elif best_score >= 0.7:
|
||||
desc = "高能副歌核心区域"
|
||||
elif best_score >= 0.5:
|
||||
desc = "中高能量段落"
|
||||
else:
|
||||
desc = "最优可用段落"
|
||||
|
||||
trim_options.append({
|
||||
"target_duration": target_dur,
|
||||
"start": round(best_start_time, 2),
|
||||
"end": round(best_end_time, 2),
|
||||
"actual_duration": round(best_end_time - best_start_time, 2),
|
||||
"energy_score": round(best_score, 4),
|
||||
"description": desc
|
||||
})
|
||||
|
||||
# 按目标时长排序
|
||||
trim_options.sort(key=lambda x: x["target_duration"])
|
||||
|
||||
# ── 5. BPM(快速估算)────────────────────────────────────────────
|
||||
try:
|
||||
import numpy as np
|
||||
tempo_val = librosa.beat.tempo(y=y, sr=sr)
|
||||
bpm = round(float(np.asarray(tempo_val).flat[0]), 2)
|
||||
except Exception:
|
||||
bpm = None
|
||||
|
||||
return {
|
||||
"total_duration": round(total_duration, 2),
|
||||
"bpm": bpm,
|
||||
"energy_peak": round(energy_peak, 2),
|
||||
"sections": sections,
|
||||
"trim_options": trim_options,
|
||||
}
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Audio energy analysis for beat-sync-editor")
|
||||
parser.add_argument("audio_file", help="Path to audio file")
|
||||
parser.add_argument(
|
||||
"--targets",
|
||||
type=str,
|
||||
default="15,30,60",
|
||||
help="Target durations in seconds, comma-separated (default: 15,30,60)"
|
||||
)
|
||||
parser.add_argument("--top-n", type=int, default=1, dest="top_n")
|
||||
parser.add_argument("--out", choices=["json", "text"], default="json")
|
||||
args = parser.parse_args()
|
||||
|
||||
try:
|
||||
target_durations = [int(x.strip()) for x in args.targets.split(",") if x.strip()]
|
||||
except ValueError:
|
||||
print("ERROR: --targets must be comma-separated integers (e.g. '15,30,60')", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
result = analyze_energy(
|
||||
audio_path=args.audio_file,
|
||||
target_durations=target_durations,
|
||||
top_n=args.top_n,
|
||||
)
|
||||
|
||||
if args.out == "text":
|
||||
print(f"总时长: {result['total_duration']}s | BPM: {result['bpm']}")
|
||||
print(f"能量峰值点: {result['energy_peak']}s")
|
||||
print("\n=== 裁剪方案 ===")
|
||||
for opt in result["trim_options"]:
|
||||
print(f" [{opt['target_duration']}s] {opt['start']}s → {opt['end']}s | 张力:{opt['energy_score']:.2f} | {opt['description']}")
|
||||
else:
|
||||
print(json.dumps(result, ensure_ascii=False, indent=2))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Binary file not shown.
Reference in New Issue
Block a user