Compare test conditions before results

Runtime

vLLM 0.27.1 / Torch 2.13.0+cu129

Hardware

NVIDIA H100 80GB HBM3

Workload

OpenAI-compatible diarized_json API smoke plus FunASR AutoModel structured-response mapping

Audio

15.1685 s two-speaker probe; SHA256 43dccc068506439cb633b382b6b98185baa837363d08cc5f7152ca89b0fdc3c8

Settings

Pinned model revision; vLLM 0.27.1; vllm[audio]; response_format=diarized_json; temperature=0; max_completion_tokens=768

Timing scope

Functional response validation after /health became ready; no latency or throughput claim

Verified

2026-08-30

Qualification: A structured response contract smoke, not a diarization accuracy study, overlap test, throughput benchmark, or production capacity promise.

Open primary source

Runtime

SGLang Omni merge 8458f76a

Hardware

NVIDIA H100 80GB

Workload

Seed-TTS EN full set, non-streaming, max concurrency 16, one repeat

Audio

1088 single-speaker English clips; mean duration 4.736 s

Settings

Single H100; 1088 / 1088 requests completed; same upstream benchmark harness used for model comparison

Timing scope

Upstream end-to-end benchmark report; throughput 55.487 req/s, mean latency 0.287 s, mean RTF 0.0625

Verified

2026-07-04

Qualification: Single-speaker English ASR benchmark after stripping timestamp and speaker markup; diarization/timestamp correctness is not evaluated, and the result is not a production capacity promise.

Open primary source

Runtime

vLLM 0.27.1+cu129 / Torch 2.13.0+cu129

Hardware

NVIDIA H100 80GB

Workload

OpenAI-compatible transcription; Chinese baseline and hotword probes plus two concurrent requests (English and Japanese)

Audio

Pinned model-repository examples: 6 s Chinese, 8 s English, and 8 s Japanese

Settings

FP32; eager mode; gpu-memory-utilization 0.40; max model length 40,960; explicit language; warmed server

Timing scope

Client wall time after /health became ready; local HTTP and decoding included; cold model download and 20.3 s engine warmup excluded

Verified

2026-08-13

Qualification: Single-H100 correctness and concurrency probe with a community-converted checkpoint, not an accuracy study or production capacity promise. Revalidate the checkpoint, target GPU, languages, traffic, and hotword policy before rollout.

Open primary source

Runtime

TensorRT 10.0.1 with Triton 24.05 model contract

Hardware

NVIDIA H100; device memory capacity and host CPU are not part of the recorded result

Workload

Native FP16 engine build, profile-bound execution, and PyTorch parity validation

Audio

Bundled Chinese example plus synthetic feature tensors at 30 and 64 frames

Settings

FP16; batch profile 1/8/16; post-LFR frame profile 1/512/4096; 8 GiB builder workspace

Timing scope

Engine build wall time only; inference timings are correctness probes and not a throughput benchmark

Verified

2026-08-04

Qualification: Single-H100 compatibility and parity evidence. Build on the target GPU and TensorRT version, then load-test with production audio before capacity planning.

Open primary source

Runtime

runtime-llamacpp-v0.2.4 Linux x64 AVX2 release asset

Hardware

Intel Xeon Platinum 8480+, 8 runtime threads

Workload

100 sequential cold-process SenseVoice transcriptions

Audio

One 6.00 s Mandarin PCM sample; mono, 16 kHz, 16-bit

Settings

Published AVX2 archive; 470,197,600-byte F16 GGUF; direct SenseVoice CLI without VAD

Timing scope

Functional stability validation only; every process included model loading and decoding, and timings were not retained

Verified

2026-08-29

Qualification: Single-host, single-sample regression evidence for the former intermittent empty-output defect, not an accuracy study or production capacity promise.

Open primary source

Runtime

llama.cpp/GGUF with built-in FSMN-VAD

Hardware

CPU, 8 threads; CPU model not recorded in the cited report

Workload

Offline Mandarin CPU

Audio

184 Mandarin clips, about 44-60 seconds each; total duration not recorded in the cited report

Settings

Q8 runtime; built-in FSMN-VAD; micro-CER normalize_zh; model load excluded

Timing scope

Sum of compute time divided by sum of audio duration; model load excluded

Verified

2026-07-26

Qualification: CPU model and full software stack are not recorded. Reproduce on target hardware before comparison.

Open primary source

Runtime

llama.cpp/GGUF with built-in FSMN-VAD

Hardware

CPU, 8 threads; CPU model not recorded in the cited report

Workload

Offline Mandarin CPU

Audio

184 Mandarin clips, about 44-60 seconds each; total duration not recorded in the cited report

Settings

Q8 runtime; built-in FSMN-VAD; micro-CER normalize_zh; model load excluded

Timing scope

Sum of compute time divided by sum of audio duration; model load excluded

Verified

2026-07-26

Qualification: CPU model and full software stack are not recorded. Reproduce on target hardware before comparison.

Open primary source

Runtime

audio.cpp native Fun-ASR-Nano runtime

Hardware

NVIDIA H100 CUDA; exact driver and toolchain versions are not recorded in the cited guide

Workload

Single offline transcription through the OpenAI-compatible server

Audio

One 14.07-second reference WAV

Settings

Standalone Q8_0 GGUF; CUDA backend; concurrency one; other runtime settings are not recorded

Timing scope

End-to-end server validation; the cited guide does not state warmup, file I/O, or process-start boundaries

Verified

2026-07-29

Qualification: Single-sample parity smoke, not a capacity benchmark. Reproduce on target hardware with production audio and concurrency.

Open primary source

Runtime

audio.cpp native sense_asr main@979e070f

Hardware

CPU; validation host details are not a cross-hardware benchmark contract

Workload

Offline Chinese transcription plus buffered streaming partials

Audio

Known 16 kHz reference WAV and PCM stream

Settings

Standalone schema-v1 Q8_0 GGUF; CPU backend; two-second streaming windows during validation

Timing scope

Functional parity and loader validation; no capacity claim

Verified

2026-08-13

Qualification: Merged into audio.cpp main with all six contribution CI jobs passing, but not yet included in a tagged release; pin the exact main commit.

Open primary source

Hardware

RTX 4090 24,564 MiB; Ryzen 9 7950X; 125 GiB RAM

Software

CUDA 13.0; PyTorch 2.11.0; Transformers 5.6.0; Fun-ASR 854d88f

SeedTTS EN

1,088/1,088; c=1: 20.16 samples/s, 0.050 s; c=16: 114.26 samples/s, 0.139 s; WER 0.0175

SeedTTS ZH

2,020/2,020; c=1: 14.36 samples/s, 0.071 s; c=16: 127.25 samples/s, 0.125 s; CER 0.0164

Stability

30-minute mixed c=1/4/8/16 soak; 105,067 successful requests; 0 unexpected errors

Memory

15.98 GiB after static allocation; 16.11 GiB after graph capture; GPU memory released on exit

Timing scope

Three measured repeats after warmup; complete SeedTTS sweeps; zero skipped clips

Request boundary

Maximum 30 seconds per request; only terminal transcript events were observed

Qualification: Results came from a dirty feature worktree while upstream integration remained in progress; they do not establish released support or low-latency streaming deltas.

• Full benchmark and environment
• Consumer GPU roadmap
• Draft integration PR
• Raw JSON artifacts