PyTorch: _pickle.UnpicklingError: invalid load key when Loading Model Checkpoints

Verified FixPython 3.10+PyTorch 2.2+Silo: pytorch

Quick Fix / Solution Rapide

This error occurs when Python tries to load a corrupted checkpoint, a Git LFS text pointer, or an HTML 404 page via torch.load(). Inspect the file with open(path, 'rb').read(100) and download complete binary weights or use safetensors.

Root Cause Analysis

This error occurs when Python executes torch.load(checkpoint_path) to load saved model weights or optimizer states, but Python's underlying pickle deserializer encounters bytes that do not begin with a valid pickle header or PyTorch ZIP magic number.

Root Cause 1: Git LFS Pointer File Instead of Binary Weights

When cloning HuggingFace or GitHub repositories without Git LFS (git-lfs) installed, model weight files (e.g. pytorch_model.bin) contain a small plaintext pointer (e.g. version https://git-lfs.github.com/spec/v1 oid sha256:...) instead of the multi-gigabyte binary weights. Passing this text file to torch.load() fails with _pickle.UnpicklingError: invalid load key, 'v'.

Root Cause 2: Interrupted Download Resulting in HTML Error Pages

When downloading weights from a remote URL via requests or wget that failed with a 404 or 403 error, the downloaded file contains HTML markup (<!DOCTYPE html>...). Passing the HTML file to torch.load() fails with _pickle.UnpicklingError: invalid load key, '<'.

Root Cause 3: Loading Safetensors Files with torch.load()

Attempting to load modern HuggingFace .safetensors files using torch.load() fails because safetensors uses a safe non-pickle JSON/binary format. Safetensors files must be loaded with safetensors.torch.load_file().

Root Cause 4: File Truncation During Process Crash

If a machine crashes or runs out of disk space midway through torch.save(), the checkpoint is truncated and cannot be unpickled.

Reproduction Code (MCVE)

Example: Bug Reproduction

import pickle

# Simulating loading a corrupt checkpoint containing HTML text (invalid load key '<')
corrupt_checkpoint_data = b"<!DOCTYPE html><html><head><title>404 Not Found</title></head></html>"

# Attempting to unpickle non-pickle binary raises _pickle.UnpicklingError
pickle.loads(corrupt_checkpoint_data)

Solution 1: Inspect File Header Bytes and Re-download with Git LFS

Inspect the first 100 bytes of the checkpoint to identify whether it is an HTML error page, Git LFS pointer, or valid PyTorch ZIP archive.

Example: Recommended Solution

# Solution 1: Header inspection and validation helper
def inspect_checkpoint_header(raw_bytes: bytes) -> str:
    if raw_bytes.startswith(b"PK\x03\x04"):
        return "Valid PyTorch Zip/Binary Format"
    elif raw_bytes.startswith(b"version https://git-lfs"):
        return "Git LFS Pointer - Run 'git lfs pull' to fetch real weights"
    elif b"<!DOCTYPE html" in raw_bytes or b"<html" in raw_bytes:
        return "HTML Error Page - File download failed with 404/403"
    else:
        return "Unknown or Corrupt Binary"

test_lfs = b"version https://git-lfs.github.com/spec/v1\noid sha256:12345"
status = inspect_checkpoint_header(test_lfs)
print("Detected file format:", status)
assert "Git LFS" in status

Solution 2: Use safetensors for Secure and Fast Weight Loading

Migrate your model storage to safetensors to avoid Python pickle vulnerabilities and gain zero-copy memory-mapped loading.

Example: Alternative Solution

# Solution 2: Safetensors loading pattern
# In production:
# from safetensors.torch import load_file
# weights = load_file("model.safetensors")

print("Safetensors eliminates pickle security vulnerabilities and invalid load key errors.")
assert True

If the checkpoint was saved with PyTorch 2.4+ with weights_only=True by default, attempting to load custom Python objects (like custom dataset classes or non-tensor configs) will raise torch.serialization.WeightsOnlyError. Pass torch.load(path, weights_only=False) if you trust the source.

Contrast _pickle.UnpicklingError with FileNotFoundError: FileNotFoundError occurs when the file path does not exist on disk; UnpicklingError occurs when the file exists but its content is corrupt or in the wrong format.

Previous Next