Unexpected return type

[danpascu]

I'm trying to implement a function that receives either a simple type (from a list of acceptable types) or a composite type between the simple type and None and returns the base type, i.e. either the simple type or extract the base type from the composite type. Note that the function doesn't deal with values of those types, but with the types themselves.

As an example get_base_type(int) will return int while get_base_type(str | None) will return str

For this I have the following code:

from types import NoneType, UnionType
from typing import cast, reveal_type

type XMLData = str | int | float | bool


def get_base_type[D: XMLData](data_type: type[D] | UnionType) -> type[D]:
    if isinstance(data_type, UnionType):
        if len(data_type.__args__) != 2 or NoneType not in data_type.__args__:
            raise TypeError(f'Unsupported data type: {data_type!r}')
        try:
            base_type = cast(type[D], next(t for t in data_type.__args__ if issubclass(t, XMLData.__value__)))
        except StopIteration:
            raise TypeError(f'Unsupported data type: {data_type!r}') from None
        else:
            return reveal_type(base_type)  # Type of "base_type" is "type[D@get_base_type]"
    return data_type


t1 = reveal_type(get_base_type(int))  # Type of "get_base_type(int)" is "type[int]"
t2 = reveal_type(get_base_type(str | None))  # Type of "get_base_type(str | None)" is "Unknown"
t3: type[str] = reveal_type(get_base_type(str | None))  # Type of "get_base_type(str | None)" is "type[str]"

I added comments on the relevant lines with what pyright reports about the types via the call to reveal_type.

Now I understand I have to add that cast for base_type because I don't expect pyright to be able to understand my runtime introspection into the composite type. Without that cast base_type appears of being of type Any.

What surprises me is that even with the cast and despite the fact that pyright correctly reports the type of base_type as type[D@get_base_type] right before returning it, after it is returned from the function and assigned to t2 it appears as "Unknown".

I'm not sure if this is a bug or I'm misunderstanding something here, but how can the type before return be something and right after return become something else?
Also note that if I add a type annotation as with t3, then the type is reported correctly.

Pyright doesn't report any errors for this code, which makes it even more confusing, because despite the function being annotated to return type[D], pyright returns type Unknown but doesn't complain that the return types don't match.

I also checked with mypy and mypy reports something similar: it says that I need a type annotation for t2 and it claims that the type for t2 is type[Never] which is even more confusing, because it seems to suggest that the function never returns in that case.

[danpascu]

I now realize that this may be because the UnionType branch doesn't involve D on input, so it has no idea what type[D] means. But I don't know how to parameterize that UnionType with D.

[erictraut]

When a static type checker evaluates a call to a function, it uses only the function's signature (i.e. the parameter and return type annotations supplied in the function declaration) to evaluate the type returned by the function. It does not pay attention to the body of the function. In fact, you can omit the entire function body and replace it with ..., as is often done in type stub files. The types of the local variables and expressions within a function body (e.g. where you are calling reveal_type) have no bearing on the evaluation of calls to get_base_type.

One of the problems here is that you're using type forms that look like type expressions (such as str | None), but you're using them as value expressions, not as type expressions. When these forms are used as value expressions, they are not treated as types by a static type checker. Instead, they're treated as regular runtime objects like UnionType.

You may be interested in TypeForm[T], an extension to the type system that is being proposed. You can read about it here. In particular, you may find this post helpful in understanding the difference between type expressions and value expressions.

[danpascu]

Thanks for the pointer. It looks promising, but still needs to be accepted and even if it is, is still ways away. For now I guess I'll just have to provide a manual type annotation.