Thermal Throttling Running a Local Model on a Gaming Laptop

Generation starts at a rate you are happy with and thirty seconds later it is visibly slower, with no change to the prompt or the settings. Run nvidia-smi -q -d PERFORMANCE while it is happening and you will see something like this.

    Clocks Event Reasons
        Idle                              : Not Active
        Applications Clocks Setting       : Not Active
        SW Power Cap                      : Active
        HW Slowdown                       : Not Active
            HW Thermal Slowdown           : Not Active
            HW Power Brake Slowdown       : Not Active
        Sync Boost                        : Not Active
        SW Thermal Slowdown               : Active
        Display Clock Setting             : Not Active

The flags you are looking at

Those lines are the whole diagnosis, and the distinction between the four active-capable ones is the thing worth learning:

• SW Power Cap — the driver is holding clocks down because the board is at its power limit. Normal under sustained load; a problem only if the limit is set below the board default.
• SW Thermal Slowdown — the driver has reduced clocks because the GPU passed its maximum operating temperature. This is the ordinary thermal throttle and the one most laptops hit.
• HW Thermal Slowdown — hardware has cut clocks by a factor of two or more because temperature crossed a protective threshold. This is not a tuning issue; it is a cooling failure.
• HW Power Brake Slowdown — an external power signal is asserting. On a laptop this most often means the power adapter cannot supply what the system is asking for, which includes running on battery or on an underrated USB-C charger.

NVIDIA renamed this group from clocks_throttle_reasons to clocks_event_reasons in recent driver branches, keeping the old names as aliases. Both spellings are in circulation; check nvidia-smi --help-query-gpu for what your driver accepts.

Logging the whole generation, not one moment

A single nvidia-smi invocation catches one instant, and throttling is a curve. Log it for the whole of a long generation and the shape tells you the story:

nvidia-smi --query-gpu=timestamp,temperature.gpu,power.draw,\
clocks.sm,clocks.mem,clocks_event_reasons.active \
  --format=csv -l 1 > throttle.csv &

# Now generate something long enough to heat the card: 2000 tokens
curl -s localhost:8080/completion \
  -d '{"prompt":"Write a detailed essay about tide tables.","n_predict":2000}' \
  > /dev/null
kill %1

What you are looking for in throttle.csv is the moment clocks.sm steps down and what temperature.gpu was doing when it did. Two shapes, two diagnoses. A clock that falls smoothly as temperature climbs towards a ceiling and then holds is normal thermal equilibrium — the cooler is working at its limit. A clock that drops off a cliff, with HW Thermal Slowdown active, means the protective threshold was crossed and the card is in self-preservation.

If the clock never drops and the rate still falls, you do not have a thermal problem at all, and the answer is somewhere in why local inference feels slower — most likely context growth making each token more expensive as the KV cache fills.

Thermal cap and power cap are different faults

It is worth being precise about which of the two you have, because the fixes point in opposite directions.

If SW Power Cap is active and temperature is comfortable, the card is doing exactly what it was designed to do and there is nothing wrong. Raising the power limit, where the board allows it, buys a little clock speed and a lot of heat, and on a laptop it usually just moves you into the thermal case below.

If SW Thermal Slowdown is active, heat is the constraint, and the counter-intuitive fix is often to lower the power limit. A card oscillating between boost and throttle spends part of every cycle ramping rather than computing; capping it just below the point where it overheats lets it hold one clock indefinitely, and a steady lower clock frequently produces more tokens over a five-minute generation than a higher one that keeps collapsing.

The part that is specific to a laptop

On a desktop the CPU and GPU have separate coolers and separate power budgets. On a gaming laptop they very often share a vapour chamber and are managed under one package power budget by a dynamic-boost mechanism, which means the CPU and the GPU are competing for the same watts and the same heat capacity.

That has a direct consequence for local inference that does not exist on a desktop: llama.cpp’s --threads setting affects GPU performance. If some layers are on the CPU, or if the tokenizer and sampling threads are spun up aggressively, the CPU draws power and generates heat that the shared budget then takes away from the GPU. Dropping -t from “all cores” to physical cores, or lower, can raise sustained GPU clocks on exactly the machines where it looks like it should not matter.

The other laptop-specific fault is power delivery. Many machines silently reduce the GPU power budget on battery, and a USB-C charger rated below the laptop’s barrel supply will do the same. If HW Power Brake Slowdown is ever active, check what the machine is plugged into before you touch anything else.

Fixes, in the order worth trying

1. Confirm the machine is on its rated adapter and that the vendor’s performance mode is selected. Most gaming laptops ship in a balanced profile that caps both fan speed and power, and the fastest fix on many machines is a setting in the vendor utility.
2. Cap the power limit deliberately. Read the range first with nvidia-smi -q -d POWER, then set a limit inside it with sudo nvidia-smi -pl 100. Re-run the logging command from above and compare the sustained clocks.sm, not the peak.
3. Lock a clock instead of capping power where the card supports it: sudo nvidia-smi -lgc 1200,1500 pins the graphics clock into a range. Many mobile cards refuse this; if it errors, stay with the power limit.
4. Reduce CPU contribution. Set -t to physical core count, and make sure every layer is on the GPU with -ngl all if it fits. A model that spills to CPU generates heat in the worst possible place on a shared-cooling machine.
5. Fix the airflow physically. Elevate the rear of the chassis so the intake is not against the desk, clear the exhaust, and clean the fins. This is not a joke fix — on a two-year-old laptop it is frequently the largest single change available.
6. Re-measure with the same prompt. A repeatable tokens-per-second measurement is the only way to know whether any of the above did anything, and the measurement has to be of a long generation, because a short one finishes before the card gets hot.

If the sustained rate is still unacceptable after all of that, the honest conclusion is that the chassis cannot dissipate what the workload produces. Generation is a sustained load in a way that gaming is not — there are no loading screens and no menus — so a laptop that handles games comfortably can still be thermally undersized for an hour of continuous inference.