Your First LLM Call in Python

A first model call in Python is one HTTP POST with a JSON body, an Authorization header, and a response you read one field out of. Everything else in this page exists so that the second call, and the ten-thousandth, still work.

An empty directory to a working environment

1. Make the project and a virtual environment. Python 3.10 or newer; python3 on macOS and Linux, py -3 on Windows.
2. Activate it. The prompt changes to show the environment name, which is the only confirmation you get.
3. Install one dependency. httpx speaks HTTP/1.1 and HTTP/2, has a synchronous and an asynchronous client with the same method names, and streams responses without extra machinery.
4. Freeze what you installed, so the next machine gets the same thing.

mkdir llm-first && cd llm-first

python3 -m venv .venv            # Windows: py -3 -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate

pip install httpx python-dotenv
pip freeze > requirements.txt

echo ".venv/" >> .gitignore
echo ".env"   >> .gitignore      # do this BEFORE you create .env

The last two lines are the load-bearing ones and they are the two every first tutorial leaves out. Write the ignore rules before the secret exists, because a .env that has already been committed once is in the history for ever and the key has to be rotated, not deleted.

The request, at the HTTP level

Almost every hosted model today is reachable through an OpenAI-compatible chat completions endpoint. Knowing its shape is worth more than knowing any one SDK, because the shape is what the SDKs are wrapping and it is what you will see in a proxy log when something is wrong.

POST {BASE_URL}/chat/completions
Authorization: Bearer {API_KEY}
Content-Type: application/json

{
  "model": "openai/gpt-4o-mini",
  "messages": [
    {"role": "system", "content": "You answer in one sentence."},
    {"role": "user",   "content": "Why is the sky blue?"}
  ],
  "temperature": 0.2,
  "max_tokens": 200
}

The response body you care about is three fields deep:

{
  "id": "chatcmpl-...",
  "model": "openai/gpt-4o-mini",
  "choices": [
    {
      "index": 0,
      "message": {"role": "assistant", "content": "Because ..."},
      "finish_reason": "stop"
    }
  ],
  "usage": {"prompt_tokens": 24, "completion_tokens": 31, "total_tokens": 55}
}

Where the key goes instead of the file

The key belongs in the process environment, and the environment gets populated from a file that git has never seen. That is two rules, and the second one is why python-dotenv is installed above.

# .env  — already in .gitignore, never committed
LLM_BASE_URL=https://api.multigrid.ai/v1
LLM_API_KEY=sk-...
LLM_MODEL=openai/gpt-4o-mini

Read it with os.environ["LLM_API_KEY"], not os.environ.get("LLM_API_KEY"). The subscript form raises KeyError at import time with the name of the missing variable in the message; the .get form returns None, sends Authorization: Bearer None, and gives you a 401 to debug instead of a one-line answer. Fail on the missing config, not on the consequence.

For anything beyond a scratch project, a .env file on disk is still a secret in plaintext. Secrets in a Python AI project covers the keyring and process-injection options, and API key security covers what an exposed key actually costs you.

The whole script

# main.py
import os
import sys

import httpx
from dotenv import load_dotenv

load_dotenv()  # reads .env into os.environ; does not overwrite real env vars

BASE_URL = os.environ["LLM_BASE_URL"].rstrip("/")
API_KEY = os.environ["LLM_API_KEY"]
MODEL = os.environ.get("LLM_MODEL", "openai/gpt-4o-mini")


def ask(question: str, *, max_tokens: int = 400) -> str:
    payload = {
        "model": MODEL,
        "messages": [
            {"role": "system", "content": "Answer in at most three sentences."},
            {"role": "user", "content": question},
        ],
        "temperature": 0.2,
        "max_tokens": max_tokens,
    }
    with httpx.Client(timeout=httpx.Timeout(connect=5.0, read=60.0,
                                            write=10.0, pool=5.0)) as client:
        response = client.post(
            f"{BASE_URL}/chat/completions",
            headers={"Authorization": f"Bearer {API_KEY}"},
            json=payload,
        )
        response.raise_for_status()
        body = response.json()

    choice = body["choices"][0]
    if choice.get("finish_reason") == "length":
        print("warning: answer truncated at max_tokens", file=sys.stderr)

    usage = body.get("usage", {})
    print(
        f"[in {usage.get('prompt_tokens')} / out {usage.get('completion_tokens')} tokens]",
        file=sys.stderr,
    )
    return choice["message"]["content"]


if __name__ == "__main__":
    print(ask(" ".join(sys.argv[1:]) or "Why is the sky blue?"))

$ python main.py "What does an ORM actually do?"
[in 29 / out 64 tokens]
An ORM maps rows in a relational database to objects in your program ...

Two details in that function are not decoration. The timeout is four numbers rather than one, because connecting and reading fail for unrelated reasons and want unrelated budgets — five seconds to establish a connection is generous, five seconds to read a long answer is a guaranteed failure. And the diagnostics go to stderr so that python main.py ... > out.txt still writes only the answer.

The same call, streamed

Add "stream": true and the response stops being one JSON document and becomes a sequence of server-sent events: lines beginning data: , each carrying a fragment, terminated by the literal data: [DONE].

import json

def ask_streaming(question: str) -> str:
    payload = {
        "model": MODEL,
        "messages": [{"role": "user", "content": question}],
        "stream": True,
    }
    parts: list[str] = []
    with httpx.Client(timeout=httpx.Timeout(connect=5.0, read=60.0,
                                            write=10.0, pool=5.0)) as client:
        with client.stream(
            "POST",
            f"{BASE_URL}/chat/completions",
            headers={"Authorization": f"Bearer {API_KEY}"},
            json=payload,
        ) as response:
            response.raise_for_status()
            for line in response.iter_lines():
                line = line.strip()
                if not line or not line.startswith("data:"):
                    continue                      # comments and keep-alives
                data = line[len("data:"):].strip()
                if data == "[DONE]":
                    break
                delta = json.loads(data)["choices"][0].get("delta", {})
                chunk = delta.get("content")
                if chunk:
                    print(chunk, end="", flush=True)
                    parts.append(chunk)
    print()
    return "".join(parts)

flush=True is the whole point. Python buffers stdout when it is not a terminal, so without it a piped or redirected stream arrives in one lump at the end and the streaming was for nothing. Streaming a response without blocking your app takes this further — turning it into a generator, handling the delta fields other than content, and pushing the bytes to a browser.

The four things that go wrong first

• 401 with a key you can see is correct. Nearly always a stray character: a trailing newline from a copy-paste, or quotes around the value in .env that dotenv strips but a shell export does not. Print repr(API_KEY[-4:]) — if it shows '...\n' you have found it.
• 404 on the endpoint. The base URL already ends in /v1 and the code appends /v1/chat/completions, giving /v1/v1/.... The .rstrip("/") above fixes the other half of this, the double slash.
• The answer stops mid-sentence. That is finish_reason == "length", not a model fault. Raise max_tokens — and note that it caps the output only, which is a different thing from the context window.
• A 429, immediately, on the second script you write. Rate limits are per-key, so a loop over a hundred rows hits them in seconds. That needs a retry policy — retrying model calls with tenacity — and ideally a limiter on your own side, in rate limiting yourself before they do.

At this point you have a script that calls one model. The reason to keep it at the HTTP layer is that switching models is now editing one string, and switching providers is editing one URL — the code above has no vendor in it anywhere except an environment variable.

What this script is not ready for

Being explicit about the gap is more useful than a longer script. Forty lines is the right size for a first call and the wrong size for anything that runs unattended. Five things are missing, and each has a recipe of its own in this cluster.

Two things it deliberately does not need. There is no SDK, so there is no SDK version to track and no method name that can be renamed under you. And there is no vendor-specific logic, which means the same script points at a different provider by changing one environment variable — worth keeping true as the code grows, because it is the property that makes trying a second model a five-second experiment rather than a refactor.