Deployment contract
llama.cpp / GGUF standalone
Use the ten v0.2.6 release packages or source builds to run FunASR GGUF models on CPU, Vulkan, CUDA, Blackwell, and edge devices.
Workload boundary
Decide whether it fits your production constraints
Good fit
- No Python ML environment
- Desktop applications and offline edge deployment
- CPU, Vulkan, CUDA, or Blackwell packages for Linux, macOS, and Windows
Not a fit
- vLLM-style large GPU batch scheduling
- Blackwell production deployment without target-hardware validation
- Workflows that require the full Python model ecosystem
| Models | SenseVoiceSmall-GGUF / Paraformer-GGUF / Fun-ASR-Nano-GGUF / FSMN-VAD-GGUF |
|---|---|
| Hardware | cpu / desktop-edge-gpu |
| Operating systems | Linux / macOS / Windows |
| Interfaces | CLI / local HTTP server |
Release assets
Download a verified package for your operating system and backend
SHA-256 values come from the public GitHub Release. Verify the digest, then run a smoke test on the target hardware.
| OS | Architecture | Backend | Download | SHA-256 |
|---|---|---|---|---|
| Linux | arm64 | CPU | funasr-llamacpp-linux-arm64.tar.gz | 7bca29cfa3c9a08e235a62212ca9e00f6656e59a8f07078966a2bfda1e5aa1f9 |
| Linux | x64 | CPU | funasr-llamacpp-linux-x64.tar.gz | 779967de1c528c2be966bcc47f246e7d3e6fcdb748d9491263062f4120f35e52 |
| Linux | x64 AVX2 | CPU | funasr-llamacpp-linux-x64-avx2.tar.gz | aaebc5470f846ce915200b35d6e9f9bd0a0d3ed399d39e49bdeb7a1f1782bc70 |
| Linux | x64 | Vulkan | funasr-llamacpp-linux-x64-vulkan.tar.gz | f02d41e98e9d4041f0896661007193810f025484d2175958f7c1313d5c90ec46 |
| macOS | arm64 | CPU | funasr-llamacpp-macos-arm64.tar.gz | bda59474202b887190f59d25b7b42c714469efae71276072c12fa0a38de68792 |
| Windows | x64 | CPU | funasr-llamacpp-windows-x64.zip | f6a73a548413ba9fbaf2145263ea66ec53cbdad1fb11790dbeeee493e339492e |
| Windows | x64 AVX2 | CPU | funasr-llamacpp-windows-x64-avx2.zip | 062cda8fefadd31c3e811227116daccf448a8520f4b0bb168d225c896e65ebbd |
| Windows | x64 | Vulkan | funasr-llamacpp-windows-x64-vulkan.zip | debf8007e55011cad06081e7b8a78972f1b8fe672bc324d41e650d68821f6a6a |
| Windows | x64 | CUDA | funasr-llamacpp-windows-x64-cuda.zip | 148657911fb666b7af6ec43af2e23a0984e3259012b4c39f95631b717feb6840 |
| Windows | x64 Blackwell (sm_120) | CUDA | funasr-llamacpp-windows-x64-cuda-blackwell.zip | e32961a753f40888182f352fa551159c5165a6a77718ae4ade316aedfea4b1c2 |
Run path
From installation to known-audio verification
These commands come from the current verification registry. Pin dependencies, model, and hardware before rollout.
Install
curl -fLO https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.6/funasr-llamacpp-linux-x64.tar.gzecho "779967de1c528c2be966bcc47f246e7d3e6fcdb748d9491263062f4120f35e52 funasr-llamacpp-linux-x64.tar.gz" | sha256sum -c -mkdir funasr-llamacpp && tar -xzf funasr-llamacpp-linux-x64.tar.gz -C funasr-llamacpp && cd funasr-llamacpp && bash download-funasr-model.sh sensevoice ./funasr-gguf f16Launch
cd funasr-llamacpp && ./llama-funasr-sensevoice -m funasr-gguf/sensevoice-small-f16.gguf --vad funasr-gguf/fsmn-vad.gguf -a sample.wavHealth check
cd funasr-llamacpp && ./llama-funasr-sensevoice --helpSmoke test
Minimum verification before promotion
cd funasr-llamacpp && ./llama-funasr-sensevoice -m funasr-gguf/sensevoice-small-f16.gguf --vad funasr-gguf/fsmn-vad.gguf -a sample.wav | tee transcript.txt && test -s transcript.txtOperations and capacity
Move from runnable to operable
Operational checks
- Verify all ten release assets against the listed SHA-256 values
- Keep models and binaries on the same release manifest
- Retain the previous binary and model directory for rollback
Capacity variables
- Audio duration, language, channels, and VAD segment distribution
- Concurrency, queue time, warmup, and model-cache state
- Exact hardware, driver, runtime, and thread configuration
Troubleshooting
- Validate the q8 CPU path first
- Match CUDA or Vulkan to the local driver
- Capture the ordered stderr boundaries: initializing, resolving buffer type, backend ready, model ready, graph allocated, and compute starting
- For Windows AMD Vulkan failures, include the exact archive, GPU, driver, command, exit code, and last diagnostic boundary reached
Security boundary
Treat production ingress as untrusted
- Read local audio and models by default
- Bind the HTTP server to a private address and limit uploads
- Do not load GGUF files from untrusted sources
Known limitations
v0.2.6 provides a dedicated sm_120 Blackwell package with official cuBLAS and a static MSVC runtime; ten-platform build, archive, and dependency audits do not prove operation on physical Blackwell hardware. A positive RX 9070 XT v0.2.5 retest also does not resolve every AMD Windows 0xC0000005 report; Android/Mali remains outside the prebuilt and validated targets.
Public benchmarks are reproduction starting points, not substitutes for target-workload testing.
Evidence and feedback
