Skip to content
Multigrid

Field notes

First-hand accounts of building and running an LLM gateway, alongside the engineering procedures that came out of them.

Everything else in this library could have been written by a competent stranger. These pages could not. They come out of building the thing this site sells — a gateway that routes inference across several providers, holds customer balances and bills to the micro-dollar — and out of the mistakes that came with it.

Eight of the pages here are first-hand: an incident on a money path, a security review including the findings deliberately left unfixed, what a thousand pages of content actually taught us, and what we already know we got wrong. Each names the change it is about, so anyone who wants to check the account against the code can.

The other twelve are not first-hand, and they say so by being written as what they are: engineering guides. A row in the plan promising “a week of production errors, categorised” became a taxonomy of production error classes, because there was no such week and inventing one would have made every other page here worth less. That trade is the editorial position of this cluster.

What Building a Thousand Pages Taught Us

A first-hand account of taking a content library from four pages to a thousand with parallel agents: what worked, what broke, and what is still unknown.

10 min read

What an Unused AI Feature Actually Costs

The three cost lines an unadopted AI feature keeps generating after launch, how to total them, and the four demand signals available before you build.

8 min read

Production LLM Error Classes, and How to Categorise Your Own

The eight classes an LLM application's production errors fall into, why HTTP status alone misclassifies most of them, and how to instrument so your own counts mean something.

9 min read

When to Remove an AI Chatbot: A Decision Procedure

The four measurements that decide whether a chatbot should be removed, the thresholds worth arguing about, and the three interfaces that usually replace it.

8 min read

Switching LLM Provider: The Incompatibility Checklist

The incompatibilities that break an LLM provider switch, ordered by when they surface, plus a differential harness that finds yours before you cut over.

10 min read

Why a Prompt That Worked Stops Working

The four mechanisms behind a prompt that quietly stops working, how to tell which one you have, and the regression net that catches each of them.

9 min read

What Our Evals Missed

Real defects that a green test suite never saw, why each was structurally invisible, and the assertions that would have caught them.

10 min read

Refactoring an Agent Loop Into a State Machine

How to convert a while-loop agent into an explicit state machine, with working code, and exactly which failure modes the conversion removes.

10 min read

One Incident, Written Up Properly

A complete blameless postmortem of a real defect on a money path: what broke, why it was invisible, how it was found, and a template you can copy.

11 min read

Telling a Retrieval Bug From a Model Bug in RAG

Three measurements that decide whether a RAG failure is in retrieval or in generation, and the seven retrieval faults that masquerade as model failures.

9 min read

What We Learned Publishing on a Schedule

Why merge-day publishing is a trap, the mechanism that separates writing from publishing, and what happened when the cadence was overridden.

9 min read

Six Levers That Cut an LLM Bill Without Changing Model

Six cost levers with the arithmetic worked out for each, so you can compute your own saving before implementing any of them.

10 min read

The Support Load an AI Feature Creates

The six categories of support ticket an AI feature generates, why most of them are interface faults rather than model faults, and how to instrument before launch.

8 min read

Running a Security Review on Our Own Stack

How we reviewed our own LLM gateway, the classes of finding it produced, and the nine we verified and deliberately did not fix.

11 min read

What a New Model Release Actually Requires From Your Code

Which parts of a model release require a code change, which require nothing, and the abstraction boundaries that decide which answer you get.

8 min read

Writing Software With Coding Agents All Day

Where coding agents saved days on this codebase, where they cost a day, and the instructions that made the difference between the two.

10 min read

Designing the Off-Switch for an AI Feature

The four layers an AI feature's off-switch needs, why each must work without a deploy, and the default that has to fail closed.

8 min read

Eight Things That Surprise Backend Engineers About AI Systems

The eight structural differences between an LLM system and ordinary backend work, each explained mechanically rather than asserted.

9 min read

Things We Believed in Month One That Were Wrong

Seven assumptions this project held early, the specific evidence that killed each one, and what holding it had already cost by then.

10 min read

What We Would Do Differently, Starting Over

The decisions worth keeping from building this system, the six we would reverse on day one, and the two we still cannot call.

9 min read

Other topics