VoxCPM2 on AX8860 (llama.cpp + ggml-axera, 2026-11-15)
在爱芯 AX8860 的 NPU(4 核)上用 llama.cpp 运行 VoxCPM2 语音合成:零样本音色克隆、流式输出,OpenAI 兼容的 /v1/audio/speech。
VoxCPM2 text-to-speech on the 4-core NPU of the Axera AX8860, with llama.cpp: zero-shot voice cloning, streamed output and an OpenAI-compatible /v1/audio/speech.
性能 / Performance
AX8860,NPU 4 核,服务模式端到端;数字全部来自 ./bench.sh / 4 cores, server, end to end; all numbers come from ./bench.sh:
| 请求 / Request | 语音 / Audio | 耗时 / Time | RTF |
|---|---|---|---|
| 英文短句 / English, short | 3.36 s | 0.93 s | 0.276 |
| 中文短句 / Chinese, short | 3.20 s | 0.92 s | 0.286 |
| 英文中句 / English, medium | 8.16 s | 2.21 s | 0.270 |
| 中文中句 / Chinese, medium | 8.48 s | 2.34 s | 0.276 |
| 英文长句 / English, long | 22.24 s | 6.00 s | 0.270 |
| 中文长句 / Chinese, long | 32.00 s | 8.58 s | 0.268 |
| 项目 / Item | 结果 / Result |
|---|---|
| 流式首包 / streamed first audio | 英文 / English 0.18 s,中文 / Chinese 0.20 s |
| 服务就绪 / server ready | 62.5 s |
| NPU 内存 / NPU memory (CMM) | 约 / about 3.0 GB |
| 主机内存 / host RSS | 约 / about 6.3 GB |
连续 60 个混合请求(一半为音色克隆)内存不增长 / memory stays flat over 60 mixed requests, half of them voice clones.
文件 / Files
| 文件 / File | 说明 / Description |
|---|---|
gguf/VoxCPM2-BaseLM-F16.gguf |
语言模型权重(GGUF F16,3.2 GB)/ language model weights |
gguf/VoxCPM2-Acoustic-F16.gguf |
声学部分权重(GGUF F16,1.8 GB)/ acoustic weights |
bin/ |
llama.cpp 板端程序 llama-server、llama-tts(静态链接)/ the board programs (static) |
kernels/ |
NPU kernel 8 个 + manifest.json / eight kernels + manifest.json |
run.sh |
运行脚本 / run script |
bench.sh |
性能脚本,上表数字由它生成 / benchmark script that produces the tables above |
examples/reference.wav |
音色克隆用的参考录音 / reference recording for voice cloning |
SHA256SUMS |
全包校验和 / checksums |
8 个 kernel:语言模型 prefill(256 与 16 token 两种页长)、语言模型 decode、残差语言模型(含后处理与停止头)、局部编码器、扩散循环(9 步 Euler 一次调用完成)、声码器解码与编码。 Eight kernels: language-model prefill (256- and 16-token pages), language-model decode, the residual language model (with post-processing and the stop head), the local encoder, the diffusion loop (all nine Euler steps in one call), and the audio VAE decoder and encoder.
运行 / Run
环境 / Requirements:AX8860 与 SDK 运行库(/opt/lib),板上需有 curl / an AX8860 with the SDK runtime in /opt/lib, and curl on the board.
./run.sh "今天天气很好,我们一起去公园散步吧。" out.wav # 默认音色 / default voice
./run.sh "这是零样本克隆。" clone.wav examples/reference.wav # 音色克隆 / voice cloning
./run.sh "这是零样本克隆。" clone.wav examples/reference.wav "参考录音的文字" # 加上参考文字 / with the reference transcript
服务 / Server:
LD_LIBRARY_PATH=/opt/lib GGML_AXERA_KERNEL_DIR=$PWD/kernels \
MALLOC_ARENA_MAX=2 MALLOC_MMAP_THRESHOLD_=131072 MALLOC_TRIM_THRESHOLD_=131072 \
./bin/llama-server -m gguf/VoxCPM2-BaseLM-F16.gguf \
--voxcpm2-acoustic gguf/VoxCPM2-Acoustic-F16.gguf \
-ngl 99 -c 8192 -b 1024 -np 1 -t 8 --threads-http 1 --host 0.0.0.0 --port 8080
# 整段 WAV / complete WAV
curl -s -o out.wav http://127.0.0.1:8080/v1/audio/speech -H 'Content-Type: application/json' \
-d '{"model":"voxcpm2","input":"今天天气很好。","response_format":"wav"}'
# 流式 PCM / streamed raw PCM
curl -sN -o out.pcm http://127.0.0.1:8080/v1/audio/speech -H 'Content-Type: application/json' \
-d '{"model":"voxcpm2","input":"今天天气很好。","response_format":"pcm","stream":true}'
音色克隆在请求里加 ref_audio(base64 WAV),可选 ref_text;其他可用字段:seed、max_steps、inference_timesteps、cfg_value、temperature。
Voice cloning adds ref_audio (base64 WAV) and optionally ref_text; other fields: seed, max_steps, inference_timesteps, cfg_value, temperature.
正确性 / Correctness
- 11 段中英文文本各 5 个随机种子,共 55 段合成语音用 Qwen3-ASR 转写:没有丢句或重复句,与官方 CPU 实现的结果一致 / 55 synthesized clips (11 Chinese/English passages, five seeds each) transcribed with Qwen3-ASR: no missing or repeated clause, matching the official CPU implementation.
- 14–24 s 的中英文长句转写与原文一致 / 14–24 s Chinese and English passages transcribe back to the input text.
已知限制 / Known limits
- 运行时独占 NPU 4 核 / takes all 4 NPU cores; do not run other NPU programs alongside.
- 请求串行处理,服务端
-np 1/ requests are served one at a time; the server runs-np 1.
源码 / Sources
- llama.cpp:ZHEQIUSHUI/llama.cpp @
14fcd6e3b
更新 / Changelog
- 2026-11-15:首次发布 / first release.
- Downloads last month
- 769
16-bit
Model tree for AXERA-TECH/VoxCPM2-AX8860-llama.cpp
Base model
openbmb/VoxCPM2