Transcribing Audio Locally With faster-whisper

faster-whisper is Whisper reimplemented on CTranslate2, an inference engine built for encoder-decoder transformers. It keeps the Python API people expect and replaces the execution underneath it, which is why it can quantize to int8 on a CPU without you converting anything by hand.

Install it

pip install faster-whisper

The package is pure Python plus a CTranslate2 wheel; there is no PyTorch in the dependency tree, which is most of why the install is small. For GPU execution the SYSTRAN project documents Python 3.9 or newer, CUDA 12 with cuBLAS, and cuDNN 9. Those are hard requirements of the CTranslate2 build, not soft ones: a cuDNN 8 system will import the package fine and then fail when it loads a model, which is a confusing place to discover a version mismatch.

On a machine with no usable GPU you do not need any of that. Pass device="cpu" and compute_type="int8" and it runs on the CPU backend.

The first transcript, and the lazy generator

from faster_whisper import WhisperModel

model = WhisperModel("large-v3", device="cuda", compute_type="float16")
segments, info = model.transcribe("audio.mp3", beam_size=5)

print(info.language, info.language_probability)

for segment in segments:
    print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))

The model name is resolved against the Hugging Face Hub and cached locally, so the first run downloads and every run after it does not. For a genuinely offline machine, download once on a connected machine and point WhisperModel at the local directory instead of the name.

The thing that catches everyone is the second line of the return value. segments is a generator, not a list. Nothing is transcribed at the moment transcribe() returns — it returns almost immediately, having only run language detection to populate info. The actual decoding happens as you iterate. This matters in three ways: timing the call to transcribe() measures nothing; wrapping it in a try block does not catch decoding errors, which surface during iteration; and if you want the whole transcript before you do anything else you must force it with segments = list(segments).

Unlike whisper.cpp, faster-whisper does the audio decoding for you through PyAV, so an MP3 or an M4A is fine and you do not need the ffmpeg conversion step that the whisper.cpp path requires. It still resamples to 16 kHz mono internally, because that is what the model’s front end wants.

What the published timings actually say

The faster-whisper README publishes a benchmark table, and it is one of the few numbers in this territory that comes with its hardware attached. The project states that the GPU figures were executed with CUDA 12.4 on an NVIDIA RTX 3070 Ti 8GB, the CPU figures with 8 threads on an Intel Core i7-12700K, and that the workload is 13 minutes of audio. SYSTRAN publishes the table in the project README.

Wall-clock seconds on somebody else’s 3070 Ti tell you nothing directly. Divided into the audio length they become real-time factors, which transfer much better because they are dimensionless. Thirteen minutes is 780 seconds, so for the large-v2 GPU rows:

• openai/whisper, fp16, beam 5 — 2m23s. 780 / 143 = 5.5x real time.
• faster-whisper, fp16, beam 5 — 1m03s. 780 / 63 = 12.4x real time.
• faster-whisper, int8, beam 5 — 59s. 780 / 59 = 13.2x real time, in 2926 MB rather than 4525 MB.
• faster-whisper, fp16, batch size 8 — 17s. 780 / 17 = 45.9x real time, at 6090 MB.

Two things fall out of that arithmetic. The first is that the jump from the reference implementation to faster-whisper at the same precision is a bit over 2x, while the jump from unbatched to batched is nearly another 4x — batching is the larger lever, and it is the one people skip. The second is that int8 buys memory far more than it buys speed on a GPU: 13.2x against 12.4x is marginal, but 2926 MB against 4525 MB is the difference between fitting large-v2 on an 8 GB card alongside something else and not.

Batching and VAD

The batched path is a separate class, because batching Whisper means splitting one long recording into independent windows and decoding them in parallel, which changes how context flows between segments:

from faster_whisper import WhisperModel, BatchedInferencePipeline

model = WhisperModel("large-v3", device="cuda", compute_type="float16")
pipeline = BatchedInferencePipeline(model=model)

segments, info = pipeline.transcribe("audio.mp3", batch_size=8)

Sequential decoding conditions each 30-second window on the text produced for the previous one, which helps with consistent spelling of names and with sentences that straddle a boundary. Batched decoding cannot do that, because the windows are in flight at the same time. The trade is real, and it is the reason the project reports separate sequential and chunked long-form numbers rather than one figure.

Voice-activity detection is the other lever, and on sparse audio it is bigger than anything else here:

segments, info = model.transcribe(
    "meeting.wav",
    beam_size=5,
    vad_filter=True,
    vad_parameters=dict(min_silence_duration_ms=500),
)

Whisper processes fixed 30-second windows whether or not anybody is talking, so an hour of recording with twenty minutes of speech in it costs an hour of inference. A VAD pass drops the silent stretches before they reach the model, which on that recording is a two-thirds saving that has nothing to do with precision or batch size. It also suppresses a specific failure mode: Whisper asked to transcribe near- silence has a habit of emitting whatever its training data associates with silence, which is why spurious subtitle credits show up in transcripts of quiet rooms.

The whole thing, in order

1. pip install faster-whisper. For GPU, confirm CUDA 12 and cuDNN 9 are present before you blame the model.
2. Construct WhisperModel("large-v3", ...) with compute_type="float16" on GPU or "int8" on CPU.
3. Call transcribe(), then iterate the segments — nothing runs until you do.
4. Turn vad_filter=True on for anything with silence in it, and check that it did not clip quiet speech.
5. If throughput matters more than cross-window consistency, switch to BatchedInferencePipeline and raise batch_size until VRAM stops you.