Skip to content
Multigrid

Python for AI: hands-on recipes

Twenty short Python recipes for calling, streaming, retrying, parsing, caching, testing and profiling language models, each ending in code that runs.

Most Python-and-LLM tutorials are a screenshot of one SDK at one moment. They stop working when the SDK renames a method, and they teach nothing that survives the rename. The pages here are written the other way round: the model call is HTTP, the HTTP is boring and stable, and the interesting work is everything wrapped around it — the retry policy, the parser, the cache key, the rate limiter, the test that does not hit the network.

That is also where the bugs are. A production AI service in Python is about five per cent model call and ninety-five per cent the handling of what comes back: partial JSON, a 429 at three in the morning, a client that closed the tab mid-stream, a batch job that died at row 34,000. Each recipe below takes one of those, shows the shape of code that survives it, and names the failure it is there to prevent.

Every snippet targets Python 3.10 or newer and uses httpx for HTTP. Where a page names a library — tenacity, Pydantic, FastAPI, pandas, pytest — it says which major version the names belong to, because that is the detail that silently rots a tutorial.

Your First LLM Call in Python

From an empty virtual environment to a streamed model response in about forty lines of Python, with the API key never written into a file.

9 min read

Streaming a Response Without Blocking Your App

How to turn a streamed chat completion into a Python generator, an async iterator, and bytes that actually reach a browser unbuffered.

10 min read

Forty Requests at Once With asyncio

A working concurrency pattern for model calls: one client, a semaphore, bounded gather, and the three mistakes that quietly serialise the whole thing.

11 min read

Retrying Model Calls With tenacity

A retry policy for LLM calls using tenacity: which status codes to retry, which to never retry, how to honour Retry-After, and why jitter is not optional.

11 min read

Parsing Model Output Safely

A defensive parser for model output in Python: stripping code fences, finding the JSON object, and failing loudly instead of returning a plausible empty result.

10 min read

Pydantic Models as Your Output Contract

Using Pydantic v2 to define, request and enforce the shape of model output, including generating the JSON schema from the model and repairing on validation failure.

11 min read

A FastAPI Endpoint That Streams to the Browser

A complete FastAPI server-sent-events endpoint that proxies a streamed model response, cancels the upstream call when the client disconnects, and does not buffer.

11 min read

Background Jobs for Long AI Tasks

A job table, a worker loop and a progress endpoint for AI work too slow for a request cycle, with a dead-letter path somebody actually reads.

12 min read

Caching Model Responses in Python

A cache key that includes everything which changes the answer, a SQLite store with a TTL, and a rule for deciding how long entries should live.

10 min read

Reading PDFs, DOCX and HTML Into Clean Text

One extraction function per format in Python, each with its real failure cases named, plus the dispatcher and the normalisation step that comes after.

12 min read

Embeddings in NumPy Before You Add a Database

Brute-force cosine similarity over 100,000 embeddings in NumPy, with the memory and arithmetic derived on the page and a harness to time it on your own machine.

11 min read

Classifying 50,000 Rows Without Melting the Budget

A resumable pandas-to-LLM classification job: deduplicate first, batch second, checkpoint always, and estimate the bill before you spend it.

12 min read

Logging Every Model Call

One Python decorator that records the prompt, the token counts, the computed cost and the latency of every model call, and the queries that make the log worth having.

11 min read

Testing Code That Calls a Model

A pytest setup for LLM code: fixtures that never touch the network, recorded responses for the shapes that matter, and the two tests worth running live.

11 min read

Rate Limiting Yourself Before They Do

A working token-bucket limiter in Python that governs requests per minute and tokens per minute at the same time, for both sync and asyncio code.

11 min read

Secrets in a Python AI Project

How to hold an API key in a Python project: dotenv for development, the OS keyring for a laptop, injected environment for production, and what must never reach a notebook.

10 min read

Notebooks for LLM Work Without the Usual Mess

A Jupyter setup for LLM experiments: a spend guard in the first cell, a disk cache so re-running is free, and a path for getting the code out into a module.

10 min read

Type Hints That Make an AI Codebase Survivable

Concrete Python types for messages, tool schemas, tool results and model responses, written against the OpenAI-compatible wire format so they can be checked against a real body.

11 min read

Packaging an AI Script as a CLI

Turning an LLM script into an installable command: argparse, a documented config resolution order, and a --dry-run that prints the cost before anything is spent.

11 min read

Profiling an AI Pipeline to Find the Real Bottleneck

How to measure where an LLM pipeline's time goes: phase timers first, then cProfile and py-spy, with the columns that mislead about I/O and async explained.

11 min read

Other topics