VoxCPM2 on AX8860 (llama.cpp + ggml-axera, 2026-11-15)

在爱芯 AX8860 的 NPU(4 核)上用 llama.cpp 运行 VoxCPM2 语音合成:零样本音色克隆、流式输出,OpenAI 兼容的 /v1/audio/speech。 VoxCPM2 text-to-speech on the 4-core NPU of the Axera AX8860, with llama.cpp: zero-shot voice cloning, streamed output and an OpenAI-compatible /v1/audio/speech.

性能 / Performance

AX8860,NPU 4 核,服务模式端到端;数字全部来自 ./bench.sh / 4 cores, server, end to end; all numbers come from ./bench.sh:

请求 / Request 语音 / Audio 耗时 / Time RTF
英文短句 / English, short 3.36 s 0.93 s 0.276
中文短句 / Chinese, short 3.20 s 0.92 s 0.286
英文中句 / English, medium 8.16 s 2.21 s 0.270
中文中句 / Chinese, medium 8.48 s 2.34 s 0.276
英文长句 / English, long 22.24 s 6.00 s 0.270
中文长句 / Chinese, long 32.00 s 8.58 s 0.268
项目 / Item 结果 / Result
流式首包 / streamed first audio 英文 / English 0.18 s,中文 / Chinese 0.20 s
服务就绪 / server ready 62.5 s
NPU 内存 / NPU memory (CMM) 约 / about 3.0 GB
主机内存 / host RSS 约 / about 6.3 GB

连续 60 个混合请求(一半为音色克隆)内存不增长 / memory stays flat over 60 mixed requests, half of them voice clones.

文件 / Files

文件 / File 说明 / Description
gguf/VoxCPM2-BaseLM-F16.gguf 语言模型权重(GGUF F16,3.2 GB)/ language model weights
gguf/VoxCPM2-Acoustic-F16.gguf 声学部分权重(GGUF F16,1.8 GB)/ acoustic weights
bin/ llama.cpp 板端程序 llama-server、llama-tts(静态链接)/ the board programs (static)
kernels/ NPU kernel 8 个 + manifest.json / eight kernels + manifest.json
run.sh 运行脚本 / run script
bench.sh 性能脚本,上表数字由它生成 / benchmark script that produces the tables above
examples/reference.wav 音色克隆用的参考录音 / reference recording for voice cloning
SHA256SUMS 全包校验和 / checksums

8 个 kernel:语言模型 prefill(256 与 16 token 两种页长)、语言模型 decode、残差语言模型(含后处理与停止头)、局部编码器、扩散循环(9 步 Euler 一次调用完成)、声码器解码与编码。 Eight kernels: language-model prefill (256- and 16-token pages), language-model decode, the residual language model (with post-processing and the stop head), the local encoder, the diffusion loop (all nine Euler steps in one call), and the audio VAE decoder and encoder.

运行 / Run

环境 / Requirements:AX8860 与 SDK 运行库(/opt/lib),板上需有 curl / an AX8860 with the SDK runtime in /opt/lib, and curl on the board.

./run.sh "今天天气很好,我们一起去公园散步吧。" out.wav                 # 默认音色 / default voice
./run.sh "这是零样本克隆。" clone.wav examples/reference.wav             # 音色克隆 / voice cloning
./run.sh "这是零样本克隆。" clone.wav examples/reference.wav "参考录音的文字"  # 加上参考文字 / with the reference transcript

服务 / Server:

LD_LIBRARY_PATH=/opt/lib GGML_AXERA_KERNEL_DIR=$PWD/kernels \
  MALLOC_ARENA_MAX=2 MALLOC_MMAP_THRESHOLD_=131072 MALLOC_TRIM_THRESHOLD_=131072 \
  ./bin/llama-server -m gguf/VoxCPM2-BaseLM-F16.gguf \
    --voxcpm2-acoustic gguf/VoxCPM2-Acoustic-F16.gguf \
    -ngl 99 -c 8192 -b 1024 -np 1 -t 8 --threads-http 1 --host 0.0.0.0 --port 8080

# 整段 WAV / complete WAV
curl -s -o out.wav http://127.0.0.1:8080/v1/audio/speech -H 'Content-Type: application/json' \
  -d '{"model":"voxcpm2","input":"今天天气很好。","response_format":"wav"}'

# 流式 PCM / streamed raw PCM
curl -sN -o out.pcm http://127.0.0.1:8080/v1/audio/speech -H 'Content-Type: application/json' \
  -d '{"model":"voxcpm2","input":"今天天气很好。","response_format":"pcm","stream":true}'

音色克隆在请求里加 ref_audio(base64 WAV),可选 ref_text;其他可用字段:seed、max_steps、inference_timesteps、cfg_value、temperature。 Voice cloning adds ref_audio (base64 WAV) and optionally ref_text; other fields: seed, max_steps, inference_timesteps, cfg_value, temperature.

正确性 / Correctness

  • 11 段中英文文本各 5 个随机种子,共 55 段合成语音用 Qwen3-ASR 转写:没有丢句或重复句,与官方 CPU 实现的结果一致 / 55 synthesized clips (11 Chinese/English passages, five seeds each) transcribed with Qwen3-ASR: no missing or repeated clause, matching the official CPU implementation.
  • 14–24 s 的中英文长句转写与原文一致 / 14–24 s Chinese and English passages transcribe back to the input text.

已知限制 / Known limits

  • 运行时独占 NPU 4 核 / takes all 4 NPU cores; do not run other NPU programs alongside.
  • 请求串行处理,服务端 -np 1 / requests are served one at a time; the server runs -np 1.

源码 / Sources

更新 / Changelog

  • 2026-11-15:首次发布 / first release.
Downloads last month
769
GGUF
Model size
0.9B params
Architecture
voxcpm-acoustic
Hardware compatibility
Log In to add your hardware

16-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for AXERA-TECH/VoxCPM2-AX8860-llama.cpp

Base model

openbmb/VoxCPM2
Quantized
(26)
this model

Collection including AXERA-TECH/VoxCPM2-AX8860-llama.cpp