“Torch Not Compiled With CUDA Enabled”

Your GPU is fine, the driver is fine, and nvidia-smi is fine. The assertion is about the PyTorch build sitting in your environment, which was compiled without any CUDA support at all — and that is entirely a question of which wheel pip fetched.

The assertion

Traceback (most recent call last):
  ...
  File ".../torch/cuda/__init__.py", line 310, in _lazy_init
    raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled

It comes from PyTorch’s lazy CUDA initialisation, which runs the first time anything touches the GPU: torch.cuda.is_available() returning False is the quiet version, and this assertion is what happens when something calls .cuda(), passes device="cuda" or sets device_map anyway. The line number moves between releases; the string does not. The upstream discussion is long-running — pytorch issue 30664 is the canonical thread.

What it is and is not saying

PyTorch ships as several distinct builds from the same source. The CPU-only build genuinely does not contain the CUDA code — the symbols are not in the binary, so there is nothing to enable at runtime. The assertion is a check that the build has the capability at all, before any question of hardware arises.

It is therefore not telling you the driver is wrong, the toolkit is missing, the card is unsupported, or the CUDA version mismatches. Those are real problems with different messages — a card too new for the wheel’s compiled architectures gives a no-kernel-image error instead, and that distinction saves people hours.

One consequence worth stating because it saves an unnecessary install: you do not need the CUDA toolkit on the machine. The CUDA-enabled wheels bundle the runtime libraries they need. What you do need is an NVIDIA driver recent enough for the CUDA version the wheel was built against, and nvidia-smi reporting a driver version is enough to confirm one is present.

One command that names the cause

python -c "import torch; print(torch.__version__, torch.version.cuda, torch.cuda.is_available())"

# 2.5.1+cpu   None    False   <- CPU-only wheel. This is your cause.
# 2.5.1+cu124 12.4    True    <- CUDA wheel, working.
# 2.5.1+cu124 12.4    False   <- CUDA wheel, but the GPU is not visible.

The local version suffix after the plus sign is the whole answer. A +cpu suffix, or a torch.version.cuda of None, means you have the CPU build and the rest of this page applies. A +cu suffix with False means the build is right and something else is hiding the device — a container without --gpus all, an empty CUDA_VISIBLE_DEVICES, or a driver that failed to load — and reinstalling torch will not help.

On Apple Silicon there is no third possibility: no CUDA build of PyTorch exists for macOS, and no reinstall will produce one. The GPU backend there is Metal, reached through the mps device, and code that hard-codes cuda has to be changed rather than the environment.

Five ways the CPU wheel got there

• A plain pip install torch on a platform whose default wheel is CPU-only. Which platforms those are has changed more than once, so this is worth checking rather than remembering.
• A conda environment with the cpuonly package. It is a real package and it pins you to CPU builds; it has to be removed, not overridden.
• A requirements file that pins a bare version. torch==2.5.1 without the index URL resolves against PyPI, which is how a CI image that was working ends up on CPU after an unrelated change.
• A dependency reinstalled it. Installing another package that lists torch as a dependency can replace your CUDA wheel with the default one. If GPU support worked yesterday and does not today, look at what else you installed.
• The wrong environment. Two virtual environments, or a system Python and a venv, and the shell is in the other one. python -c "import torch; print(torch.__file__)" settles it.

The fourth of those is worth dwelling on because it is the one that produces the most confusing week. pip resolves the whole dependency graph at once, and a package that lists torch without an index is free to satisfy it from PyPI. The install log will say it uninstalled your 2.5.1+cu124 and installed 2.5.1, which looks like a no-op if you are skimming and is in fact the entire problem. Read the uninstall lines in a pip log, not just the errors.

The container case, which looks identical

Inside Docker there are two independent ways to arrive here and they need opposite fixes, so it is worth separating them before touching anything.

The first is the same wheel problem as everywhere else: a Dockerfile that runs pip install torch without the index URL bakes a CPU-only build into the image, and it will fail identically on every host regardless of how the container is run. The diagnostic is the same version string, and the fix belongs in the Dockerfile.

The second is that the container cannot see the device at all. That requires the NVIDIA container toolkit on the host and the container started with GPU access — --gpus all for Docker, a resource limit for Kubernetes. In that case the wheel is correct and torch.version.cuda reports a version, but torch.cuda.is_available() is False and torch.cuda.device_count() is zero. Run nvidia-smi inside the container: if it is not found or reports no devices, nothing about Python is your problem.

docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi

# inside your own image
python -c "import torch; print(torch.cuda.device_count(), torch.version.cuda)"

A third variant catches people on shared machines: CUDA_VISIBLE_DEVICES set to an empty string hides every device from the process without any error anywhere, and it is exactly what a scheduler sets when it has allocated you no GPU. Check the variable before you check anything else on a machine you do not administer; echo "[$CUDA_VISIBLE_DEVICES]" distinguishes unset from empty, which the bare form does not.

The reinstall, done so it sticks

1. Confirm which environment you are fixing, and activate it. Every command below is environment-scoped.
2. Remove the existing packages together, so a stale companion package does not pull the CPU build back: pip uninstall -y torch torchvision torchaudio.
3. Install from PyTorch’s own index with the CUDA build selected explicitly.
4. Verify with the diagnostic command above, in a new interpreter. A notebook holds the old module until the kernel restarts, which is a classic false negative.
5. Pin the index in your requirements file, not just in your shell history, or the next clean install repeats the problem.

pip uninstall -y torch torchvision torchaudio
pip install torch torchvision torchaudio \
  --index-url https://download.pytorch.org/whl/cu124

python -c "import torch; print(torch.__version__, torch.cuda.is_available())"