Deployment contract

Native FunASR on vLLM

Serve the pinned official FunAudioLLM/Fun-ASR-Nano-2512-vllm snapshot on vLLM 0.27.1; Chinese, English, Japanese and hotwords passed a separate single-H100 functional smoke.

Start deployment Review benchmark method

Maturity: Community verified
FunASR: Official model revision a4362c943d48951f98ca2a62181cc028970270c5; not a FunASR package version
Runtime: vLLM 0.27.1+cu129 / Torch 2.13.0+cu129 / Transformers 5.15.0 / CUDA 12.9 / Python 3.12.3
Verified: 2026-09-07

Workload boundary

Decide whether it fits your production constraints

Good fit

A standard /v1/audio/transcriptions endpoint
Concurrent or batch Fun-ASR-Nano transcription
An NVIDIA GPU Linux serving environment

Not a fit

Replacing the pinned checkpoint below without revalidation
CPU-only or edge devices
Internet-facing service before memory, concurrency, and workload tests

Models	FunAudioLLM/Fun-ASR-Nano-2512-vllm (official checkpoint)
Hardware	nvidia-gpu / kubernetes
Operating systems	Linux
Interfaces	OpenAI-compatible HTTP

Run path

From installation to known-audio verification

These commands come from the current verification registry. Pin dependencies, model, and hardware before rollout.

Install

export VLLM_PYTHON="$PWD/.venv/bin/python"
export VALIDATION_DIR="$PWD/.official-native-validation"
export MODEL_DIR="$VALIDATION_DIR/official-model/a4362c943d48951f98ca2a62181cc028970270c5"

"$VLLM_PYTHON" - <<'PY'
import importlib.metadata as metadata
import torch
import vllm.model_executor.models.funasr

for name, expected in {"vllm": "0.27.1+cu129", "torch": "2.13.0+cu129",
                       "transformers": "5.15.0"}.items():
    assert metadata.version(name) == expected, (name, metadata.version(name))
assert torch.version.cuda == "12.9" and torch.cuda.is_available()
PY

HF_HUB_DISABLE_TELEMETRY=1 HF_XET_CACHE="$VALIDATION_DIR/isolated-xet-cache" "$VLLM_PYTHON" - <<'PY'
import hashlib
import json
import os
from pathlib import Path
from huggingface_hub import HfApi, snapshot_download

model_id = "FunAudioLLM/Fun-ASR-Nano-2512-vllm"
revision = "a4362c943d48951f98ca2a62181cc028970270c5"
root = Path(os.environ["MODEL_DIR"])
info = HfApi().model_info(model_id, revision=revision, files_metadata=True)
assert info.sha == revision and len(info.siblings) == 23
snapshot_download(model_id, revision=revision, local_dir=str(root),
                  cache_dir=str(Path(os.environ["VALIDATION_DIR"]) / "isolated-hf-cache"),
                  max_workers=4)
files = []
for item in info.siblings:
    content = (root / item.rfilename).read_bytes()
    digest = hashlib.sha256(content).hexdigest()
    assert len(content) == item.size, item.rfilename
    if item.lfs:
        assert digest == item.lfs.sha256, item.rfilename
    else:
        assert hashlib.sha1(f"blob {len(content)}\0".encode() + content).hexdigest() == item.blob_id
    files.append({"path": item.rfilename, "size": len(content), "sha256": digest})
assert json.loads((root / "config.json").read_text())["architectures"] == ["FunASRForConditionalGeneration"]
(root.parent / "download-manifest.json").write_text(json.dumps(files, indent=2) + "\n")
PY

Launch

CUDA_VISIBLE_DEVICES=0 PYTHONDONTWRITEBYTECODE=1 \
HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 \
HF_HUB_DISABLE_IMPLICIT_TOKEN=1 HF_HUB_DISABLE_TELEMETRY=1 VLLM_NO_USAGE_STATS=1 \
VLLM_CACHE_ROOT="$VALIDATION_DIR/runtime-cache" TRITON_CACHE_DIR="$VALIDATION_DIR/triton-cache" \
"$VLLM_PYTHON" -B -m vllm.entrypoints.openai.api_server \
  --model "$MODEL_DIR" --served-model-name fun-asr-nano-official-a4362c94 \
  --host 127.0.0.1 --port 57185 --dtype float32 \
  --gpu-memory-utilization 0.40 --enforce-eager

Health check

curl --max-time 15 -fsS http://127.0.0.1:57185/health

curl --max-time 15 -fsS http://127.0.0.1:57185/v1/models

Smoke test

Minimum verification before promotion

curl --max-time 45 -fsS http://127.0.0.1:57185/v1/audio/transcriptions -F "file=@$MODEL_DIR/example/zh.mp3;type=audio/mpeg" -F model=fun-asr-nano-official-a4362c94 -F language=zh -F response_format=json

curl --max-time 45 -fsS http://127.0.0.1:57185/v1/audio/transcriptions -F "file=@$MODEL_DIR/example/en.mp3;type=audio/mpeg" -F model=fun-asr-nano-official-a4362c94 -F language=en -F response_format=json

curl --max-time 45 -fsS http://127.0.0.1:57185/v1/audio/transcriptions -F "file=@$MODEL_DIR/example/ja.mp3;type=audio/mpeg" -F model=fun-asr-nano-official-a4362c94 -F language=ja -F response_format=json

curl --max-time 45 -fsS http://127.0.0.1:57185/v1/audio/transcriptions -F "file=@$MODEL_DIR/example/zh.mp3;type=audio/mpeg" -F model=fun-asr-nano-official-a4362c94 -F language=zh -F 'hotwords=开放时间,开放时间,开放时间' -F response_format=json

Operations and capacity

Move from runnable to operable

Operational checks

Pin the official model revision and local file digests plus vLLM 0.27.1+cu129, Torch 2.13.0+cu129, Transformers 5.15.0 and CUDA 12.9; a model revision is not a FunASR package version
Wait for /health and warm the model before measuring capacity
Measure queue time, final-result latency, memory, and hotword hit rate

Capacity variables

Audio duration, language, channels, and VAD segment distribution
Concurrency, queue time, warmup, and model-cache state
Exact hardware, driver, runtime, and thread configuration

Troubleshooting

For audio HTTP 400 responses, confirm vllm[audio] is installed
If startup reports insufficient KV cache, raise GPU utilization or lower max-model-len only when the workload permits
Pass language explicitly for non-English audio; comma-separate hotwords and validate bias strength on production samples

Security boundary

Treat production ingress as untrusted

Put authentication, TLS, rate limits, and audio size/duration limits at the gateway
Bind the worker to 127.0.0.1; the gateway was not validated by this smoke, and the worker must not be exposed directly
Verify wheel, model revision, and sample digests; isolate model cache and temporary uploads

Known limitations

Current commands use official FunAudioLLM/Fun-ASR-Nano-2512-vllm@a4362c943d48951f98ca2a62181cc028970270c5, tested 2026-09-07 in an existing environment: vLLM 0.27.1+cu129 / Torch 2.13.0+cu129 / Transformers 5.15.0 / CUDA 12.9. This is not a clean installation validation; preparation only checks versions and downloads a local snapshot. Eight HTTP requests establish functional compatibility only; first Chinese was not warmed, and the 0.911 s two-request probe is not capacity evidence. The 2026-08-13 benchmark below remains a historical community-converted checkpoint result, neither relabeled nor compared with the new run. #54944 updates main docs/test registry only; v0.28.0 still referenced the community artifact at audit time, not a claim of released official references.

Public benchmarks are reproduction starting points, not substitutes for target-workload testing.

Evidence and feedback

Verify this contract against primary material

Report a deployment issue View related benchmark records