Matrix-Testing mypy Across Python Versions

TL;DR — A single mypy run only analyzes your code against one target interpreter, so version-specific typing errors slip through. Put a GitHub Actions strategy.matrix over python-version and pass mypy --python-version=X on each leg so mypy analyzes as if it were that release. Set fail-fast: false so every version reports independently. This surfaces new-union syntax, PEP 695 generics, and stdlib stub drift that a one-version pipeline never sees.

If your library advertises support for Python 3.9 through 3.13, then “mypy passes” is only true for the one interpreter your CI happened to install. The X | Y union operator, type statements from PEP 695, and even the shape of typeshed stubs all shift between releases. A matrix run is the type-checking analogue of a test matrix: it proves the annotations hold on every version you ship.

The matrix workflow

Two knobs work together. matrix.python-version controls which interpreter actions/setup-python installs — that matters for anything mypy resolves at import time, such as which typeshed stubs ship. The --python-version flag tells mypy which target language level to analyze against, independent of the interpreter actually running the tool. Set both to the same value on each leg and the analysis is honest end to end.

# .github/workflows/typecheck.yml — GitHub Actions, mypy 1.13
name: typecheck
on:
  pull_request:
  push:
    branches: [main]

jobs:
  mypy:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false                       # every version reports independently
      matrix:
        python-version: ["3.9", "3.11", "3.12", "3.13"]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
      - run: pip install -e ".[dev]"          # mypy pinned in pyproject
      - name: Run mypy for this target
        run: >
          mypy
          --python-version ${{ matrix.python-version }}
          --show-error-codes
          src/

fail-fast: false is the load-bearing line. The default (true) cancels every sibling leg the instant one fails, so a [syntax] error on 3.9 would hide whatever 3.13 was about to report. You want the full picture: is a failure universal, or specific to one release?

The matrix block is a Cartesian product. List a second axis and GitHub expands every combination into its own leg — python-version: ["3.9", "3.12"] crossed with os: [ubuntu-latest, windows-latest] becomes four jobs. For a type matrix you usually want only the python-version axis, but include: bolts on a single extra leg — a 3.13 leg that additionally runs pyright, say — without multiplying the whole grid, and exclude: drops a combination that can never work. Keep the version list defined once and reference ${{ matrix.python-version }} from both the mypy job and the pytest job so the type matrix and the test matrix cannot drift out of sync.

Cost scales linearly with the number of legs, but each leg is cheap once the dependency install is cached (see caching mypy and pyright). Because --python-version only swaps the analysis target and never runs your code, mypy still executes under whatever single interpreter setup-python installed on that leg — a four-version type matrix therefore finishes in roughly the wall-clock time of its slowest leg, since the legs run concurrently and none blocks another when fail-fast is off.

pyright has the same two-knob shape, so the matrix pattern transfers directly. Its target-version equivalent is pythonVersion in pyrightconfig.json (or [tool.pyright] in pyproject.toml), overridable per invocation with pyright --pythonversion 3.9. If you leave it unset, pyright infers the target from the active interpreter — which, inside a matrix leg, is matrix.python-version, so simply installing the right interpreter often gives pyright the correct target for free. Where mypy needs the explicit flag to override its default, pyright mostly needs you not to pin pythonVersion in config so the per-leg interpreter can drive it. Running both checkers across the same matrix catches the widest set of version-specific issues, at the cost of reconciling their occasionally divergent diagnostics.

What a single run misses

Version-specific type errors fall into three broad classes: runtime constructs that only evaluate on newer releases, grammar that only parses on newer releases, and stub or import differences that only resolve on a matching target. A newest-version-only run is blind to all three, because every one of them is, by construction, valid on the newest version. The matrix is what forces each class into the open on the release where it actually breaks.

Consider the modern union operator. Written as a runtime type it only evaluates on 3.10+, but as an annotation under from __future__ import annotations it is a string mypy still validates against the target.

# src/config.py — checked with mypy 1.13
from __future__ import annotations

def build_dsn(port: int | None) -> str:   # ok on --python-version 3.12
    ...

# src/registry.py — mypy 1.13, evaluated at runtime (no __future__)
IntOrStr = int | str                        # runtime: TypeError on 3.9
# mypy --python-version 3.9 flags: unsupported operand [operator]

PEP 695 is the sharper case. The type statement and class Repo[T] syntax are hard syntax errors before 3.12, and only a matrix that includes an old target catches it.

# src/repo.py — PEP 695 generics, mypy 1.13
type OrderId = int                          # 3.12+ only
class OrderRepository[T]:                    # 3.12+ only
    def get(self, key: OrderId) -> T: ...
# mypy --python-version 3.11 flags: [syntax]  "type parameter syntax requires 3.12"

The new-union rewrite is exactly the kind of change ruff’s UP007 / UP045 rules will suggest; the matrix is what proves the rewrite is safe on your lowest supported version before you apply it. PEP 695 details live in the PEP 695 type parameter syntax guide, and the union forms in Union and Optional types.

The two failures above are fundamentally different, and the distinction is worth internalizing. A runtime union like IntOrStr = int | str is executable code — the | operator runs at import time, so on 3.9 it raises TypeError: unsupported operand type(s) for |. Adding from __future__ import annotations does not save it, because that import only defers annotations to strings; a bare assignment at module scope is not an annotation. mypy reproduces this with --python-version 3.9 and reports [operator]. PEP 695’s type X = int and class Repo[T] are a sharper failure still: they are new grammar, so the interpreter cannot even parse the file before 3.12. No __future__ import defers syntax — the only fixes are to raise your minimum version or guard the construct behind a runtime sys.version_info branch. mypy flags the pre-3.12 target with [misc]/[syntax] and a message about type-parameter syntax requiring 3.12.

The same trap hides in imports of typing symbols that were backported. from typing import Self is valid syntax on any version but only resolves at runtime on 3.11+; on 3.10 it raises ImportError, and the portable form is from typing_extensions import Self. mypy is happy with either against a modern --python-version, so a static-only check never sees the problem — but the matrix’s paired interpreter (installed by setup-python) runs your import and surfaces the ImportError the moment a leg targets an older release. The same applies to override (3.12), TypeAlias (3.10), ParamSpec (3.10), and assert_type. Other grammar-level features a matrix protects — match statements and parenthesized context managers (3.10+), except* groups (3.11+) — fail identically to PEP 695: a hard parse error your newest-version-only job will never reach.

Stub differences across versions

typeshed — the stub set mypy bundles — ships version-conditional annotations gated by sys.version_info checks inside the stub files themselves. A stdlib function whose signature changed (a parameter that became keyword-only, a return type narrowed from Any), a module that did not exist yet, or a method added in a later release is all annotated differently per target. mypy reads the stub branch matching your --python-version, so a call that is clean on 3.12 can raise [call-overload], [arg-type], [attr-defined], or [import-not-found] on 3.9 where the older stub branch applies. Matrix legs are the only way to see both.

Concrete cases make this tangible. tomllib landed in 3.11, so import tomllib type-checks on --python-version 3.11 but reports [import-not-found] (mypy) / reportMissingImports (pyright) on 3.10. str.removeprefix/removesuffix arrived in 3.9, so "log_".removeprefix("log_") is [attr-defined] on a 3.8 target. datetime.UTC (an alias for datetime.timezone.utc) is 3.11-only in the stubs. These are not hypothetical — they are the routine reasons a green newest-version build breaks for a user on an older interpreter:

# src/settings.py — mypy 1.13
import tomllib                      # ok on --python-version 3.11+

def load(path: str) -> dict[str, object]:
    with open(path, "rb") as f:
        return tomllib.load(f)
# mypy --python-version 3.10 flags: Cannot find implementation or library
# stub for module named "tomllib"  [import-not-found]

The distinction from third-party stubs matters: packages like types-requests target one library API and are not branched by your --python-version the way stdlib typeshed is. So version-conditional surprises are overwhelmingly a stdlib phenomenon, which is precisely what matrix legs over --python-version are built to surface. Keep the interpreter and the target equal per leg (covered in the callout above) so a stub-branch error is never masked by a mismatched runtime.

The same target awareness applies to sys.version_info guards in your own code. Both checkers treat if sys.version_info >= (3, 11): as a statically evaluable condition and prune the unreachable branch for the current target — so code inside a 3.11+ guard is only analyzed on legs whose --python-version is 3.11 or newer, and code in the else fallback is only analyzed on older legs. A matrix is therefore the only way to type-check both arms of a version guard; a single newest-version run silently skips the fallback path your older users actually execute. One caveat on the pyright side: pyright bundles its own copy of typeshed pinned to the pyright release, so the exact stub branch it reads depends on the pyright version as well as the target — another reason to pin the checker version (as the caching guide argues) alongside the interpreter.

Common mistakes

Most broken matrices fail in one of four ways, and each has a single corrective. Walk the questions below before trusting a green matrix: if any answer lands on the left, the matrix is quietly testing nothing.

• Omitting --python-version entirely. Every leg then analyzes against mypy’s default target (the version mypy itself runs under) and the matrix tests nothing but which interpreter installed the tool. You get four identical result sets and a false sense of coverage — the grid burns CI minutes proving one thing four times.
• Leaving fail-fast: true (the default). The first red leg cancels the rest, hiding whether a [syntax] or [operator] error is version-specific or universal. You re-run, fix the one you saw, and the next hidden leg fails on the following push. Always set fail-fast: false for a type matrix so every leg reports in one pass.
• Testing only the newest version. Modern syntax passes cleanly, so the [syntax] errors that only fire on your oldest supported release never appear until a user on 3.9 files the bug. If your pyproject.toml says requires-python = ">=3.9", the first entry in the matrix must be "3.9" — the floor is the leg that matters most.
• Interpreter and target drifting apart. Installing 3.12 but passing --python-version 3.9 checks 3.9 semantics against 3.12’s runtime, so a genuine ImportError (e.g. from typing import Self on 3.10) never fires while [attr-defined] stub errors can be masked. Keep matrix.python-version and --python-version equal on every leg — that equality is the whole diagram in “The matrix workflow” above.

A green matrix is only as honest as these four settings. When all of them hold, each leg is a genuine proof that your annotations — and the syntax and imports they depend on — hold on that shipped interpreter, which is exactly the guarantee a single-version run cannot give.

FAQ

Do I need both setup-python and --python-version? Yes, for full fidelity. --python-version alone changes the analysis target but leaves you on whatever interpreter is installed, so a genuine runtime TypeError on an old release stays invisible. Pinning setup-python to the same value makes each leg both analyze and (via your test job) run under that version.

Can I list versions once and reuse them for tests and mypy? Yes. Define the matrix at the workflow level and reference ${{ matrix.python-version }} in both your pytest and mypy jobs, or use a shared include list, so the type matrix and test matrix never drift out of sync.

Related

• GitHub Actions Type Checking
• PEP 695 Type Parameter Syntax
• Union and Optional Types
• Ruff Linter Integration
• mypy Configuration & Strictness

Back to GitHub Actions Type Checking