Deployment contract
vLLM GPU throughput
Run Fun-ASR-Nano on NVIDIA GPUs for file batches or an OpenAI-compatible transcription endpoint.
Workload boundary
Decide whether it fits your production constraints
Good fit
- Batch recording transcription
- LLM decoder throughput is the priority
- An NVIDIA GPU serving environment already exists
Not a fit
- CPU-only or edge devices
- Non-autoregressive models such as Paraformer
- Internet-facing service before memory and load tests
| Models | Fun-ASR-Nano-2512 |
|---|---|
| Hardware | nvidia-gpu / kubernetes |
| Operating systems | Linux |
| Interfaces | Python / HTTP REST / OpenAI-compatible HTTP / WebSocket |
Run path
From installation to known-audio verification
These commands come from the current verification registry. Pin dependencies, model, and hardware before rollout.
Install
pip install funasrpip install "vllm==0.19.1"Launch
CUDA_VISIBLE_DEVICES=0 python examples/industrial_data_pretraining/fun_asr_nano/serve_vllm.py --port 8899 --model FunAudioLLM/Fun-ASR-Nano-2512 --gpu-memory-utilization 0.5Health check
curl -fsS http://localhost:8899/healthSmoke test
Minimum verification before promotion
curl -X POST http://localhost:8899/v1/audio/transcriptions -F file=@audio.wav -F model=fun-asr-nano -F response_format=verbose_jsonOperations and capacity
Move from runnable to operable
Operational checks
- Pin the FunASR, vLLM, Torch, and CUDA combination
- Warm the model before capacity measurement
- Measure queue, first-result, and final-result latency
Capacity variables
- Audio duration, language, channels, and VAD segment distribution
- Concurrency, queue time, warmup, and model-cache state
- Exact hardware, driver, runtime, and thread configuration
Troubleshooting
- Match the NVIDIA driver to vLLM's CUDA build
- Capture two complete outputs and exact versions for repetition
- Check the VAD maximum segment length when long audio loses content
Security boundary
Treat production ingress as untrusted
- Put authentication, rate limits, and upload limits at an API gateway
- Do not expose the worker directly to the internet
- Isolate model cache and temporary upload storage
Known limitation
Throughput and memory depend on segmentation, batch size, concurrency, GPU, and vLLM version; retest with target traffic.
Public benchmarks are reproduction starting points, not substitutes for target-workload testing.
Evidence and feedback
