FunASR v1.3.26:OpenAI-Compatible API, vLLM, and llama.cpp

July 23, 2026 · FunASR release guide

FunASR v1.3.26 organizes the same open-source speech stack into three clear entry points:Python and an OpenAI-compatible API, vLLM for high-throughput GPU inference, and Python-free llama.cpp / GGUF runtimes.

This is more than a wheel-only release. The Python package, serving examples, GPU inference guide, and eight platform runtime assets target different deployment needs while remaining part of one FunASR ecosystem.

Choose the path that matches your workload

1. Start the OpenAI-compatible transcription service

This path uses the same server implementation as the repository smoke tests. Pin the current PyPI release, then start the service:

git clone https://github.com/modelscope/FunASR.git
cd FunASR/examples/openai_api
python -m pip install "funasr==1.3.26" fastapi uvicorn python-multipart
python server.py --model sensevoice --device cuda --port 8000

In another terminal, verify health and transcription with the public sample:

curl -L https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/BAC009S0764W0121.wav -o sample.wav
curl http://localhost:8000/health
curl http://localhost:8000/v1/audio/transcriptions \
  -F file=@sample.wav \
  -F model=sensevoice \
  -F response_format=verbose_json

If you already use the OpenAI Python client, point base_url to http://localhost:8000/v1. The examples also include model listing, Swagger, Postman, OpenAPI, Gradio, and Kubernetes paths.

2. Scale Fun-ASR-Nano throughput with vLLM

Fun-ASR-Nano runs its audio encoder and adaptor in PyTorch, then sends continuous audio embeddings into the Qwen3 decoder managed by vLLM. The repository benchmark records RTFx 340 for vLLM batch inference while keeping CER close to the PyTorch baseline.

python -m pip install "vllm==0.19.1"
python -m pip install "funasr>=1.3.26" safetensors tiktoken websockets
# Follow docs/vllm_guide.md for offline, streaming, or server mode

3. Download the self-contained llama.cpp / GGUF runtime

runtime-llamacpp-v0.1.9 ships Linux x64/ARM64, macOS ARM64, and Windows x64 packages. Linux includes AVX2 and Vulkan options; Windows includes AVX2, Vulkan, and CUDA, including funasr-llamacpp-windows-x64-vulkan.zip. Each archive provides a self-contained llama-funasr-* binary entry point.

• Choose the standard CPU build when you do not want to maintain Python.
• Choose Vulkan or CUDA for desktop GPU acceleration.
• Choose ARM64 for edge devices and Apple Silicon.

See the llama.cpp / GGUF page for models, commands, and hardware selection. Runtime assets and model versions are linked together so users do not have to hunt across releases.

What each repository owns