Deployment contract
Containers and Kubernetes
Validate with Docker Compose, then deploy with the Kubernetes template, persistent model cache, probes, and private ClusterIP.
Workload boundary
Decide whether it fits your production constraints
Good fit
- Reproducible environments
- An existing container platform
- Rolling releases and persistent model cache
Not a fit
- The local model and API are not yet validated
- GPU images and cluster scheduling are not ready
- Using the example Compose file as a public production architecture
| Models | sensevoice / paraformer / fun-asr-nano |
|---|---|
| Hardware | kubernetes / cpu / nvidia-gpu |
| Operating systems | Linux containers / Kubernetes |
| Interfaces | Docker Compose / Kubernetes ClusterIP / OpenAI-compatible HTTP |
Run path
From installation to known-audio verification
These commands come from the current verification registry. Pin dependencies, model, and hardware before rollout.
Install
cd examples/openai_api && cp .env.example .envLaunch
docker compose up --buildHealth check
curl -fsS http://localhost:8000/healthSmoke test
Minimum verification before promotion
python examples/openai_api/smoke_test.py --base-url http://localhost:8000Operations and capacity
Move from runnable to operable
Operational checks
- Persist the model cache
- Use `/health` for readiness
- Warm images and models before rollout
Capacity variables
- Audio duration, language, channels, and VAD segment distribution
- Concurrency, queue time, warmup, and model-cache state
- Exact hardware, driver, runtime, and thread configuration
Troubleshooting
- Pass the CPU Compose smoke test first
- Separate image pull, model download, and inference failures
- Check device plugin, driver, and runtime for GPU pods
Security boundary
Treat production ingress as untrusted
- Use a private ClusterIP by default
- Keep secrets out of images and source control
- Put TLS, authentication, and rate limits at ingress
Known limitation
The CPU example is not a CUDA production image; GPU drivers, image, scheduling, and capacity still require platform validation.
Public benchmarks are reproduction starting points, not substitutes for target-workload testing.
Evidence and feedback