Skip to content

[ty] Fix edge-case bugs when narrowing tagged unions in match statements#22870

Merged
AlexWaygood merged 3 commits intomainfrom
alex/unwrap-or-default
Jan 30, 2026
Merged

[ty] Fix edge-case bugs when narrowing tagged unions in match statements#22870
AlexWaygood merged 3 commits intomainfrom
alex/unwrap-or-default

Conversation

@AlexWaygood
Copy link
Member

@AlexWaygood AlexWaygood commented Jan 26, 2026

Summary

Fix two edge-case bugs in our narrowing/reachability logic that could cause issues when narrowing tagged unions of TypedDicts or tuples:

  1. An incorrect early return in narrow.rs meant that we were apply incorrect positive-case narrowing to tagged unions for OR patterns where one element in the pattern had a Literal type and another did not:

    from typing import Literal
    class Config:
        MODE: str = "default"
    
    def _(u: tuple[Literal["foo"], int] | tuple[Literal["bar"], str]):
        match u[0]:
            case Config.MODE | "foo":
                reveal_type(u)  # on `main`: revealed: tuple[Literal["foo"], int]
  2. pattern_kind_to_type in reachability_constraints.rs was incorrectly returning the inferred type of all VALUE patterns. But that's only correct if the VALUE pattern has a single-valued type[^1]. If the VALUE pattern does not have a single-valued type, the set of values that would definitely lead to that match branch being taken is the empty set -- so returning Never here is more appropriate. This bug led to incorrect inferences of Never in subsequent branches of match statements involving TypedDicts and tuples:

    from typing import Literal
    class Config:
        MODE: str = "default"
    
    def _(u: tuple[Literal["foo"], int] | tuple[Literal["bar"], str]):
        match u[0]:
            case Config.MODE | "foo":
                pass
            case _:
                reveal_type(u)  # revealed: Never

Test plan

I've added new mdtests.

It's sort-of interesting that this PR has no primer impact at all -- I guess that just shows that pattern-matching still is pretty rarely used across the Python ecosystem. It takes a while for new syntax features to gain widespread adoption!

@AlexWaygood AlexWaygood added the ty Multi-file analysis & type inference label Jan 26, 2026
@AlexWaygood AlexWaygood changed the title . [experimenting] Jan 26, 2026
@astral-sh-bot
Copy link

astral-sh-bot bot commented Jan 26, 2026

Typing conformance results

No changes detected ✅

@astral-sh-bot
Copy link

astral-sh-bot bot commented Jan 26, 2026

mypy_primer results

Changes were detected when running on open source projects
prefect (https://github.com/PrefectHQ/prefect)
+ src/integrations/prefect-dbt/prefect_dbt/core/settings.py:94:28: error[invalid-assignment] Object of type `dict[Any, Any] | int | dict[str, Any] | ... omitted 4 union elements` is not assignable to `dict[str, Any]`
+ src/integrations/prefect-dbt/prefect_dbt/core/settings.py:99:28: error[invalid-assignment] Object of type `int | dict[Any, Any] | float | ... omitted 3 union elements` is not assignable to `dict[str, Any]`
+ src/prefect/cli/deploy/_core.py:86:21: error[invalid-assignment] Object of type `dict[Any, Any] | int | dict[str, Any] | ... omitted 4 union elements` is not assignable to `dict[str, Any]`
+ src/prefect/cli/deploy/_core.py:87:21: error[invalid-assignment] Object of type `int | dict[Any, Any] | float | ... omitted 3 union elements` is not assignable to `dict[str, Any]`
- src/prefect/deployments/runner.py:997:70: warning[possibly-missing-attribute] Attribute `__name__` may be missing on object of type `Unknown | (((...) -> Any) & ((*args: object, **kwargs: object) -> object))`
+ src/prefect/deployments/runner.py:997:70: warning[possibly-missing-attribute] Attribute `__name__` may be missing on object of type `Unknown | ((...) -> Any)`
+ src/prefect/deployments/steps/core.py:137:38: error[invalid-argument-type] Argument is incorrect: Argument type `dict[Any, Any] | int | dict[str, Any] | ... omitted 4 union elements` does not satisfy constraints (`str`, `int`, `int | float`, `bool`, `dict[Any, Any]`, `list[Any]`, `None`) of type variable `T`
+ src/prefect/flow_engine.py:989:32: error[invalid-await] `Unknown | R@FlowRunEngine | Coroutine[Any, Any, R@FlowRunEngine]` is not awaitable
+ src/prefect/flow_engine.py:1580:24: error[invalid-await] `Unknown | R@AsyncFlowRunEngine | Coroutine[Any, Any, R@AsyncFlowRunEngine]` is not awaitable
+ src/prefect/flow_engine.py:1661:43: error[invalid-argument-type] Argument to function `next` is incorrect: Expected `SupportsNext[Unknown]`, found `Unknown | R@run_generator_flow_sync`
+ src/prefect/flow_engine.py:1669:21: warning[possibly-missing-attribute] Attribute `throw` may be missing on object of type `Unknown | R@run_generator_flow_sync`
+ src/prefect/flow_engine.py:1703:44: warning[possibly-missing-attribute] Attribute `__anext__` may be missing on object of type `Unknown | R@run_generator_flow_async`
+ src/prefect/flow_engine.py:1710:25: warning[possibly-missing-attribute] Attribute `throw` may be missing on object of type `Unknown | R@run_generator_flow_async`
- src/prefect/flows.py:285:34: error[unresolved-attribute] Object of type `((**P@Flow) -> R@Flow) & ((*args: object, **kwargs: object) -> object)` has no attribute `__name__`
+ src/prefect/flows.py:285:34: error[unresolved-attribute] Object of type `(**P@Flow) -> R@Flow` has no attribute `__name__`
- src/prefect/flows.py:403:68: error[unresolved-attribute] Object of type `((**P@Flow) -> R@Flow) & ((*args: object, **kwargs: object) -> object)` has no attribute `__name__`
+ src/prefect/flows.py:403:68: error[unresolved-attribute] Object of type `(**P@Flow) -> R@Flow` has no attribute `__name__`
- src/prefect/flows.py:1877:53: warning[unused-type-ignore-comment] Unused blanket `type: ignore` directive
+ src/prefect/flows.py:1937:21: error[no-matching-overload] No overload of function `run_coro_as_sync` matches arguments
- src/prefect/utilities/templating.py:320:13: error[invalid-assignment] Invalid subscript assignment with key of type `object` and value of type `Unknown | dict[str, Any]` on object of type `dict[str, Any]`
+ src/prefect/utilities/templating.py:320:13: error[invalid-assignment] Invalid subscript assignment with key of type `object` and value of type `Unknown | int | dict[str, Any] | ... omitted 4 union elements` on object of type `dict[str, Any]`
- src/prefect/utilities/templating.py:323:16: error[invalid-return-type] Return type does not match returned value: expected `T@resolve_block_document_references | dict[str, Any]`, found `list[Unknown | dict[str, Any]]`
+ src/prefect/utilities/templating.py:323:16: error[invalid-return-type] Return type does not match returned value: expected `T@resolve_block_document_references | dict[str, Any]`, found `list[Unknown | int | dict[str, Any] | ... omitted 4 union elements]`
- src/prefect/utilities/templating.py:437:16: error[invalid-return-type] Return type does not match returned value: expected `T@resolve_variables`, found `dict[object, Unknown]`
+ src/prefect/utilities/templating.py:437:16: error[invalid-return-type] Return type does not match returned value: expected `T@resolve_variables`, found `dict[object, Unknown | int | float | ... omitted 4 union elements]`
- src/prefect/utilities/templating.py:442:16: error[invalid-return-type] Return type does not match returned value: expected `T@resolve_variables`, found `list[Unknown]`
+ src/prefect/utilities/templating.py:442:16: error[invalid-return-type] Return type does not match returned value: expected `T@resolve_variables`, found `list[Unknown | int | float | ... omitted 4 union elements]`
- src/prefect/workers/base.py:232:13: error[invalid-argument-type] Argument is incorrect: Argument type `str | dict[str, Any]` does not satisfy constraints (`str`, `int`, `int | float`, `bool`, `dict[Any, Any]`, `list[Any]`, `None`) of type variable `T`
+ src/prefect/workers/base.py:232:13: error[invalid-argument-type] Argument is incorrect: Argument type `str | int | dict[str, Any] | ... omitted 3 union elements` does not satisfy constraints (`str`, `int`, `int | float`, `bool`, `dict[Any, Any]`, `list[Any]`, `None`) of type variable `T`
+ src/prefect/workers/base.py:234:22: error[invalid-argument-type] Argument expression after ** must be a mapping type: Found `int | Unknown | float | ... omitted 4 union elements`
- Found 5362 diagnostics
+ Found 5374 diagnostics

scikit-build-core (https://github.com/scikit-build/scikit-build-core)
- src/scikit_build_core/build/wheel.py:99:20: error[no-matching-overload] No overload of bound method `__init__` matches arguments
- Found 47 diagnostics
+ Found 46 diagnostics

static-frame (https://github.com/static-frame/static-frame)
+ static_frame/core/bus.py:645:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemLocReduces[Bus[Any], object_]`, found `InterGetItemLocReduces[Bottom[Bus[Any]] | Bottom[Series[Any, Any]] | TypeBlocks | ... omitted 6 union elements, object_]`
- static_frame/core/bus.py:649:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemILocReduces[Bus[Any], object_]`, found `InterGetItemILocReduces[Self@iloc, Self@iloc]`
+ static_frame/core/bus.py:649:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemILocReduces[Bus[Any], object_]`, found `InterGetItemILocReduces[Bottom[Bus[Any]] | IndexHierarchy | TypeBlocks | ... omitted 7 union elements, Self@iloc]`
+ static_frame/core/series.py:772:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemILocReduces[Series[Any, Any], TVDtype@Series]`, found `InterGetItemILocReduces[Bottom[Series[Any, Any]] | Bottom[Index[Any]] | ndarray[Never, Never] | ... omitted 7 union elements, TVDtype@Series]`
+ static_frame/core/series.py:4072:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemILocReduces[SeriesHE[Any, Any], TVDtype@SeriesHE]`, found `InterGetItemILocReduces[Bottom[Series[Any, Any]] | Bottom[Index[Any]] | TypeBlocks | ... omitted 7 union elements, TVDtype@SeriesHE]`
+ static_frame/core/yarn.py:418:16: error[invalid-return-type] Return type does not match returned value: expected `InterGetItemILocReduces[Yarn[Any], object_]`, found `InterGetItemILocReduces[Bottom[Yarn[Any]] | IndexHierarchy | TypeBlocks | ... omitted 7 union elements, object_]`
- Found 1827 diagnostics
+ Found 1831 diagnostics

pandas-stubs (https://github.com/pandas-dev/pandas-stubs)
+ tests/frame/test_groupby.py:229:15: error[type-assertion-failure] Type `Series[Any]` does not match asserted type `Series[str | bytes | int | ... omitted 12 union elements]`
+ tests/frame/test_groupby.py:625:15: error[type-assertion-failure] Type `Series[Any]` does not match asserted type `Series[str | bytes | int | ... omitted 12 union elements]`
- Found 4411 diagnostics
+ Found 4413 diagnostics

No memory usage changes detected ✅

@AlexWaygood

This comment was marked as resolved.

AlexWaygood pushed a commit that referenced this pull request Jan 30, 2026
…b-patterns

This test demonstrates a bug in match statement narrowing where an OR pattern
containing a non-literal value type (like a class attribute with type `str`)
incorrectly narrows the TypedDict union.

The bug causes:
- `case Config.mode | "foo":` to narrow to `Foo` instead of `Foo | Bar`
- Subsequent `case "bar":` to see `Never` instead of `Bar`

This test should fail on main but pass with PR #22870, which changes
`evaluate_match_pattern_value` to use `.unwrap_or_default()` instead of `?`
when `evaluate_expr_compare_op` returns `None`.

https://claude.ai/code/session_01NAVrriYXEB5X7KJcmNyeWn
@AlexWaygood AlexWaygood force-pushed the alex/unwrap-or-default branch from a9fec0e to 8a5642e Compare January 30, 2026 13:41
AlexWaygood pushed a commit that referenced this pull request Jan 30, 2026
When an OR pattern in a match statement contains a sub-pattern with a
non-single-valued type (like `Config.mode: str`), we cannot definitively
exclude any specific type from subsequent patterns. This is because the
non-literal pattern could match any value of its type at runtime.

For example, in `case Config.mode | "foo":`, if `Config.mode` has type
`str`, the pattern could match any string value. We cannot narrow based
on `"foo"` alone because the actual match might have been against
`Config.mode`.

This fix makes two changes:
1. In `pattern_kind_to_type`, for `Value` patterns, only return the type
   if it's single-valued. For non-single-valued types, return `Never` to
   indicate we can't exclude any specific type.
2. For `Or` patterns, if any sub-pattern returns `Never` (can't exclude
   anything), return `Never` for the whole OR pattern.

Also includes the PR #22870 fix to use `.unwrap_or_default()` instead
of `?` in `evaluate_match_pattern_value`, which allows TypedDict/tuple
narrowing to continue even when the compare op returns `None`.

https://claude.ai/code/session_01NAVrriYXEB5X7KJcmNyeWn
@AlexWaygood AlexWaygood changed the title [experimenting] [ty] Fix edge-case bugs when narrowing tagged unions in match statements Jan 30, 2026
@AlexWaygood AlexWaygood added the bug Something isn't working label Jan 30, 2026
@AlexWaygood AlexWaygood marked this pull request as ready for review January 30, 2026 14:59
@AlexWaygood
Copy link
Member Author

This PR had an interesting development process:

  • I saw the early return in narrow.rs while working on something else and thought it looked wrong
  • I put up a PR changing just that, hoping primer would tell me what bugs it fixed (and therefore help me write a regression test for it). But primer showed no changes on the ecosystem.
  • I then asked Claude to write me a test, and it did so successfully, but the expected types weren't revealed on subsequent match branches in the test
  • Claude then successfully found the other latent bug (in reachability_constraints.rs) that fixed the inferred types in the subsquent match branches

Copy link
Contributor

@carljm carljm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

@AlexWaygood AlexWaygood merged commit 9ebdd85 into main Jan 30, 2026
50 checks passed
@AlexWaygood AlexWaygood deleted the alex/unwrap-or-default branch January 30, 2026 23:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants