TinyML on Microcontrollers: What Fits in 256 KB
9 min read · updated August 4, 2026
TinyML is machine learning on parts with kilobytes of RAM and no operating system worth the name. The arithmetic is the same as on a phone but the budgets are four orders of magnitude smaller, and the single most useful thing to understand is that weights and working memory come out of two different pools with two different sizes.
Why this is a different discipline
A microcontroller has a few hundred kilobytes of SRAM, one to two megabytes of flash, a clock in the tens to hundreds of megahertz, no virtual memory, and frequently no floating-point unit worth using. It runs one program which never exits. There is no swapping, no dynamic allocation you should trust, and no second chance if the model does not fit — the link step fails, or worse, the stack quietly runs into the heap.
That means every number is decided before you flash the device. Which is uncomfortable, and also the good news: it is all calculable in advance.
Two memory budgets, not one
This is the point that most introductions blur, and getting it right changes what you think is possible.
| Pool | Description |
|---|---|
| flash | Holds the model weights and the program. Weights are read-only and are executed in place — they are not copied into RAM. Typically 512 KB to 2 MB. This is your model-size budget. |
| SRAM | Holds the tensor arena (intermediate activations), the stack, peripheral buffers and everything else the program needs at run time. Typically 64 KB to 512 KB. This is your architecture budget, and it is usually the binding one. |
So the question “what fits in 256 KB?” has two answers. If 256 KB is your flash, an int8 model of roughly 250,000 parameters fits, minus whatever the program itself needs — and the program is not small. If 256 KB is your SRAM, the parameter count is almost irrelevant and what matters is the largest pair of tensors that must be live simultaneously.
weight_flash_bytes = parameters × bytes_per_weight
250,000 params × 1 byte (int8) = 250 KB
250,000 params × 4 bytes (fp32) = 1000 KB ← does not fit a 1 MB part
alongside any programWhich is the first reason everything here is int8: not speed, but that fp32 weights are four times the flash for a model that gains almost nothing from the precision. The mechanism is the same as on larger hardware — see quantisation for edge devices — but on a microcontroller it is not an optimisation, it is a precondition.
Sizing the arena from the widest layer
The tensor arena is a single fixed block of SRAM that the inference runtime allocates all intermediate tensors from. Its required size is the peak, over the whole graph, of the total bytes live at once — which for a feed-forward network is usually the input and the output of the widest layer, plus anything skipped forward by a residual connection.
Worked, a small vision model on 96×96 greyscale int8 input: input tensor 96 × 96 × 1 = 9,216 bytes conv1 output 48 × 48 × 8 = 18,432 bytes --------------------------------------------------- live at conv1 = 27,648 bytes ≈ 27 KB conv2 in 48 × 48 × 8 = 18,432 conv2 out 24 × 24 × 16 = 9,216 --------------------------------------------------- live at conv2 = 27,648 bytes Peak arena ≈ 28 KB. Round up: runtimes add per-tensor bookkeeping, and alignment padding is real.
Notice what did not appear in that sum: the parameter count. A model with ten times the weights but the same activation shapes needs the same arena. This is why depth is cheap on a microcontroller and resolution is expensive — doubling the input edge quadruples every spatial tensor, and the arena with it.
The rest of your SRAM then has to hold everything else. For an audio application:
SRAM budget on a 256 KB part: tensor arena 28 KB audio ring buffer, 1 s @ 16 kHz int16 32 KB feature buffer (mel frames) 4 KB stack + globals + driver buffers ~40 KB ---------------------------------------------- used ~104 KB headroom ~152 KB
That headroom is not spare capacity, it is your margin against a stack overflow that will present as an impossible bug three weeks later. Microcontroller runtimes let you set the arena size explicitly and report the high-water mark actually used; set it generously, measure the high-water mark on real data, and only then tighten it.
How fast: MACs, clocks and cycles
Inference latency on a microcontroller is estimable to within a factor of two from three numbers, all of which you know. Count the multiply-accumulate operations in the model, divide by the MACs the core completes per cycle, divide by the clock.
latency_seconds ≈ MACs / (macs_per_cycle × clock_hz) Assumptions, stated: MACs 5,000,000 (from the model summary) macs_per_cycle 1 (scalar integer core, no DSP extensions) clock 100,000,000 (100 MHz) 5e6 / (1 × 1e8) = 0.050 s = 50 ms With SIMD/DSP extensions completing 4 int8 MACs per cycle: 5e6 / (4 × 1e8) = 0.0125 s = 12.5 ms
Two honest caveats. This ignores memory stalls, which on a part with slow flash can dominate — hence the factor of two. And macs_per_cycle is the assumption doing all the work: it depends on whether your kernels use the core’s DSP or vector extensions at all, which depends on whether you built with the optimised kernel library for your architecture. Building without it is the most common reason a TinyML model is ten times slower than expected, and it is a build-flag fix rather than a model fix.
Turn the latency into the number that matters to the product: an inference every 50 ms is twenty per second, which is comfortably above a keyword-spotting cadence of ten per second and far below a 30 fps video cadence. Decide the inference rate the application needs first, then let the arithmetic tell you the MAC budget you are allowed to spend on the model.
Energy, and why duty cycle dominates
A battery-powered sensor node is not limited by how much energy one inference costs. It is limited by how much time the part spends awake. Average current is the weighted mean of the active and sleep currents:
I_avg = duty × I_active + (1 − duty) × I_sleep duty = inference_time × inferences_per_second 50 ms every second → duty = 0.05 50 ms every 10 seconds → duty = 0.005 Battery life ≈ capacity_mAh / I_avg_mA
The consequence is a design rule that overrides almost every model optimisation: reduce how often you run, before you reduce what you run. Halving inference cost halves the active term. Running a tenth as often divides it by ten, and a cheap always-on trigger — a threshold on raw sensor amplitude, a hardware interrupt on motion — is what buys that. This cascade structure is the same one behind wake word detection, where a tiny stage-one model exists purely so stage two is idle almost all the time.
The three ways it fails on the board
Model conversion succeeds on your laptop and then the board does nothing. There are essentially three causes, they present differently, and telling them apart takes seconds once you know the signatures.
The arena is too small
The runtime fails during initialisation, before any inference, and reports a tensor allocation failure. It is deterministic — it fails on every boot, at the same point. The fix is to enlarge the arena, and the diagnostic worth wiring in permanently is the runtime’s used-bytes report: print the high-water mark after a successful allocation so that the margin is a number in your logs rather than a hope. Shrinking the arena to its exact measured requirement is a false economy; a different input shape can need more.
An operator is missing from the resolver
Also an initialisation failure, and also deterministic, but it names an operator rather than a size. Microcontroller runtimes require you to register the operators your model uses, because linking all of them would not fit. Registering only what the model needs is the intended behaviour and the resulting error is the intended feedback. If the named operator is one you did not expect, the converter decomposed something — go back and change the layer in the source model rather than registering an expensive operator you did not mean to ship.
The stack overflows
This is the difficult one, because it is not deterministic and it does not present as an inference error. It presents as a hard fault, a watchdog reset, or corrupted variables somewhere unrelated — often after the system has been running for a while, and often only with particular inputs. The cause is that the arena, the stack and the heap all draw on the same SRAM and the linker does not stop them meeting. Enable a stack-painting or stack-watermark check in your build, leave it enabled, and treat any reduction in free stack across a release as a regression worth investigating.
What actually ships on microcontrollers
The successful applications share a shape: a narrow classification over a low-dimensional signal, where the value is in not transmitting the raw data.
- Keyword spotting. A handful of wake words from a mel-spectrogram. Tens of kilobytes of weights, tens of milliseconds per inference.
- Anomaly detection on vibration or current. An autoencoder or a small classifier over accelerometer windows, reporting “this motor sounds wrong” instead of streaming accelerometer data over a radio it cannot afford to use.
- Person and object presence. Binary or few-class detection on very small greyscale images, typically to wake a larger system.
- Gesture and activity recognition. Inertial sensor windows into a small label set.
What does not ship: language models. A model whose weights alone exceed your flash by three orders of magnitude is not a tuning problem. If the product needs language understanding, the microcontroller’s job is to decide when to wake something bigger, and the design question is the trigger’s false-positive rate, not the model architecture.