How Slow Whisper Is on CPU Only, Derived

Nobody publishes a table of Whisper transcription speeds for arbitrary CPUs, and any page that gives you one has made it up. What does exist is a benchmark with its hardware attached, and arithmetic that carries it a limited distance in a direction you can check.

Real-time factor, and why it travels

The useful unit is not seconds. It is the real-time factor: audio duration divided by processing time. A factor of 4x means an hour of recording takes fifteen minutes; a factor of 1x means it takes an hour; anything below 1x cannot be used live at all.

It travels better than wall-clock seconds because it divides out the length of whatever file the benchmark used, and it degrades predictably: doubling the core count of an otherwise identical CPU moves it in a way you can reason about, whereas “83 seconds” does not tell you what the file was.

The one published CPU table

The faster-whisper README publishes CPU figures alongside its GPU ones, and states the conditions: 8 threads on an Intel Core i7-12700K, transcribing 13 minutes of audio with the small model at beam size 5. SYSTRAN publishes the table in the project README. Thirteen minutes is 780 seconds, so dividing gives real-time factors:

openai/whisper       fp32          6m58s = 418s  ->  780/418 =  1.9x
whisper.cpp          fp32          2m05s = 125s  ->  780/125 =  6.2x
whisper.cpp OpenVINO fp32          1m45s = 105s  ->  780/105 =  7.4x
faster-whisper       fp32          2m37s = 157s  ->  780/157 =  5.0x
faster-whisper       int8          1m42s = 102s  ->  780/102 =  7.6x
faster-whisper       int8 batch 8    51s         ->  780/51  = 15.3x

Three conclusions survive the transfer to other hardware. The reference Python implementation is roughly three times slower than either optimised runtime on a CPU, which is a bigger gap than the same comparison shows on a GPU. int8 matters far more on a CPU than on a GPU — 7.6x against 5.0x here, against a near-flat difference in the GPU table — because CPU inference is much closer to memory-bandwidth bound, and halving the bytes per weight halves the traffic. And batching is worth another 2x on top of that, at the cost of RAM rising from 1477 MB to 3608 MB in the same table.

Scaling to other model sizes

The table covers one model size. To reach the others, the assumption is that inference cost scales roughly linearly with parameter count at fixed sequence length — reasonable for a transformer where the dominant term is dense matrix multiplication over a fixed 1500-frame encoder input, and wrong at the margins because the decoder’s cost also depends on how many tokens it emits.

Using OpenAI’s published parameter counts and taking small (244M) as the anchor at 6.2x for whisper.cpp fp32 on an i7-12700K with 8 threads:

ratio to small        estimated real-time factor
tiny    39M / 244M = 0.16   ->  6.2 / 0.16 = ~39x
base    74M / 244M = 0.30   ->  6.2 / 0.30 = ~20x
small  244M / 244M = 1.00   ->            6.2x  (measured by SYSTRAN)
medium 769M / 244M = 3.15   ->  6.2 / 3.15 = ~2.0x
large 1550M / 244M = 6.35   ->  6.2 / 6.35 = ~1.0x

Every figure in that block except the 6.2x is an estimate produced by this arithmetic and not a measurement. The assumptions, stated plainly: linear scaling in parameters, the same runtime, the same precision, the same beam size, the same 8 threads on the same CPU, and the same audio. The estimates for tiny and base are the least trustworthy, because at small model sizes the fixed costs — audio decoding, the Mel front end, tokenizer overhead — stop being negligible and the curve flattens.

The conclusion that matters is structural and robust to the estimate being off by 50%: Whisper large on a good desktop CPU runs at roughly real time. That means CPU-only large is viable for a backlog and not viable for anything live, which is the same boundary approached from the other side in the real-time latency budget.

Why more threads stop helping

Raising -t in whisper.cpp or cpu_threads in faster-whisper improves throughput up to a point and then does not, and the point arrives earlier than the core count suggests.

The encoder pass is a sequence of large matrix multiplications, which parallelise well and are arithmetic-bound as long as the operands fit in cache. The decoder pass is a sequence of matrix-vector products at batch size one, which parallelise badly: each output element needs the whole weight matrix read from memory, so the pass is bound by memory bandwidth, and bandwidth is a property of the memory controller rather than of how many cores are asking. Adding threads to a bandwidth-bound loop adds contention.

Two practical consequences. Setting thread count to the number of physical cores is usually right and setting it to the number of hardware threads is usually not, because two SMT siblings share one core’s vector units. And on a machine with efficiency cores, letting the scheduler place inference threads on them drags the whole pass down to the slowest thread, since each layer is a barrier.

There is a third lever that is not thread count and is often larger: how many files you transcribe at once. One Whisper process at eight threads spends part of every layer waiting at a synchronisation barrier; two processes at four threads each keep the cores busy through each other’s stalls. For a backlog, where total throughput matters and per-file latency does not, running several single-threaded or lightly-threaded processes in parallel usually beats one heavily-threaded process, and it needs no configuration beyond a job queue. For a live path the opposite holds, because there you are optimising the latency of one file and the barriers are the price of that.

Instruction set matters as much as core count and is easy to overlook. whisper.cpp and CTranslate2 both dispatch to the widest vector instructions the CPU advertises, so an AVX-512 machine executes twice the fp32 lanes per instruction that an AVX2 one does at the same clock — and a machine whose kernel or container has masked those flags silently runs the narrow path. If your measured figure is roughly half what a comparable machine reports, check the flags before you change anything else.

Measuring it yourself

The estimates above exist because the measurement is cheap and nobody else’s machine is yours. whisper.cpp ships a benchmark that times the encoder directly, which isolates the part that does not depend on how talkative your audio is:

./build/bin/whisper-bench -m models/ggml-small.bin -t 8

For an end-to-end figure, time a real file of known length and divide:

ffmpeg -i sample.m4a -ar 16000 -ac 1 -c:a pcm_s16le sample.wav
ffprobe -v error -show_entries format=duration \
  -of default=noprint_wrappers=1:nokey=1 sample.wav

time ./build/bin/whisper-cli -m models/ggml-small.bin -f sample.wav -t 8

Divide the duration from ffprobe by the real time from time and you have your own real-time factor, for your CPU, your runtime, your model and your audio. Run it at two model sizes and you also have your own scaling ratio, which is worth more than the linear assumption used above. Use a file of at least a few minutes: on anything shorter, model load time dominates and the factor you compute is mostly a measurement of your disk.