TypeVar "Ts" appears only once in generic function signature

[adam-grant-hendry]

Can someone please explain to me what this error means?

For reference, I am attempting to type hint *args as suggested in PEP 646 and am getting this error on the pyright playground at line 7.

from __future__ import annotations

from typing_extensions import TypeVarTuple, Unpack

Ts = TypeVarTuple('Ts')

def foo(*args: Unpack[tuple[Unpack[Ts]]]):  # Error occurs here
    for ndx, arg in enumerate(args):
        print(f'Arg{ndx}: {arg}, {type(arg)}')

if __name__ == '__main__':
    foo(1, 2.0, ('buckle my shoe'))

[hmc-cs-mdrissi]

We can simplify this. This code,

from typing import TypeVar

T = TypeVar('T')

def foo(x: T):
  print('hi')

produces same kind of error. The main value of generic type is when you will use it in multiple places. Typically if it's used as both input type and part of return type. It could also just appear several times in input type. It can also be fine if type variable is under class scope.

But a bare function with a single type variable is rarely useful. If you don't care what type is you should just use object. For your code,

def foo(*args: object): 
    for ndx, arg in enumerate(args):
        print(f'Arg{ndx}: {arg}, {type(arg)}')

is recommended way to type it.

[erictraut]

@sassanh, OK I agree that's a legit use. Thanks for the example. I've found a way to adjust the heuristic so it allows this case while still flagging the common misuses with a warning. I think this is a reasonable compromise.

[Azureblade3808]

@erictraut Your new algorithm may be too permissive IMHO.

When I changed the related test sample from

def callable(f: Callable[P, R]):
    def inner(*args: P.args, **kwargs: P.kwargs) -> list[R]:
        return [f(*args, **kwargs)]

    return inner

into

def callable(f: Callable[P, R]):
    def inner(*args: P.args, **kwargs: P.kwargs):
        pass

    return inner

There is still no warning even when R is totally meaningless here.

While I still haven't made sure how to implement it, I think an alternative way could be scanning if a type-var is used only once through the function in both its interface and implementation. This should have a result like:

# Good. `T` is used twice.
def f0[T](x: T) -> None:
    y: T = x  # pyright: ignore[reportUnusedVariable]

# Bad. `T` is used only once, when `y` doesn't have an explicit type.
def f1[T](x: T) -> None:
    y = x  # pyright: ignore[reportUnusedVariable]

# Good. Both `P` and `R` are used more than once.
def callable[**P, R](f: Callable[P, R]):
    def inner(*args: P.args, **kwargs: P.kwargs) -> list[R]:
        return [f(*args, **kwargs)]

    return inner

# Bad. `R` is used only once.
def callable[**P, R](f: Callable[P, R]):
    def inner(*args: P.args, **kwargs: P.kwargs):
        pass

    return inner

# Also bad. `R` is used only once, when neither `callable` nor `inner` has an explicit return type.
def callable[**P, R](f: Callable[P, R]):
    def inner(*args: P.args, **kwargs: P.kwargs):
        return [f(*args, **kwargs)]

    return inner

[erictraut]

Yes, the new algorithm is more permissive. I'm trying to strike a good balance. The important thing is to cover the most common misuses hit by developers who are just starting to use TypeVars. Someone who is using a ParamSpec is far beyond the beginning phase.

[sassanh]

Thanks for the quick action! I guess I found another case when single appearance of a TypeVar is legit:

While this already works without TypeVar and inheritance is enough (note that A is the return value type of the callable test takes as input):

from typing import Callable
class A:
    ...
class B(A):
    ...
def test(fn: Callable[[], A]) -> None:
    ...
def f() -> B:
    return B()
test(f)

This doesn't work with inheritance (note than A is in the arguments of the callable test takes as input)

from typing import Callable
class A:
    ...
class B(A):
    ...
def test(fn: Callable[[A], None]) -> None:
    ...
def f(value: B) -> None:
    ...
test(f)

I think I understand the rationale behind this behavior: test shouldn't be able to call fn with an A instance as it needs a B instance and B may have more features than A, but it's ok if test calls fn and get a B instance while it is expecting an A instance since B has all the features of A.

So I think for the second scenario the signature of test should be defined with a TypeVar bound to A. I wanted to share this with you.
Let me know if a real world example is needed.

[erictraut]

Input parameter types are contravariant. How are you proposing to implement test in your second example above? How would it know what type the callback function can accept if you used a type variable in the Callable annotation?

def test[T: A](fn: Callable[[T], None]) -> None:
    fn(A()) # Not type safe!
    fn(B()) # Not type safe!

The only way this would make sense is if you passed a value of type T to test as well.

def test[T: A](fn: Callable[[T], None], arg: T) -> None:
    fn(arg) # Type safe!

test(f, B())

[gh-andre]

This warning is effectively saying "you are using a feature of the type system in a way that indicates you don't understand something; the results may not be what you intend".

You might as well use this phrase as the error message - it's just as informative, but far more honest. Certainly works better than suggesting to use object or Any.

[Azureblade3808]

We are not in the position to judge the user.

Such warnings can be caused by misunderstanding of how TypeVar works, or typos, or unfinished code, or other reasons. It could be very insulting to say "you don't understand something" or similar words.

[sytranvn]

I want to use TypeVar to declare a generic type that would be extended in subclass like a config class with extra fields.

# foo.py
class Foo:
    a: int
    b: int

# bar.py
from abc import ABC, abstractmethod
from typing import TypeVar

from foo import Foo

FooBase = TypeVar("FooBase", bound=Foo)


class Bar(ABC):
    @abstractmethod
    @classmethod
    def init_foo(cls, foo: FooBase):
        #                  ~~~~~~~
        # TypeVar "FooBase" appears only once in generic function signature
        #  Use "Foo" instead [reportInvalidTypeVarUse]
        pass

# subbar.py
from bar import Bar
from foo import Foo


class SubFoo(Foo):
    c: str


class SubBar(Bar):
    @classmethod
    def init_foo(cls, foo: SubFoo):
        return super().init_foo(foo)

But if I replace FooBase with Foo in Bar.init_foo, I get error message in override method of subclass.

Method "init_foo" overrides class "Bar" in an incompatible manner
     Parameter 2 type mismatch: base parameter is type "Foo", override parameter is type "SubFoo"
       "Foo" is not assignable to "SubFoo" [reportIncompatibleMethodOverride]

[erictraut]

I don't have the full context for the problem you're trying to solve, but it looks from your code sample that you want a class-scoped type variable rather than a method-scoped type variable.

class Bar[FooBase: Foo](ABC):
    @abstractmethod
    @classmethod
    def init_foo(cls, foo: FooBase):
        pass

class SubBar(Bar[SubFoo]):
    @classmethod
    def init_foo(cls, foo: SubFoo):
        return super().init_foo(foo)

[sytranvn]

Yes, this is what I want to achieve. But this syntax only works with python>=3.12 but unfortunately our codebase is still python3.10.

[erictraut]

You can use the old syntax to achieve the same thing. The important point is that it should be a class-scoped TypeVar, not a method-scoped TypeVar.

[sytranvn]

from abc import abstractmethod
from typing import Generic, TypeVar


class Foo:
    a: int
    b: int


FooBase = TypeVar("FooBase", bound=Foo)


class BarBase:
    pass


class Bar(BarBase, Generic[FooBase]):
    @abstractmethod
    @classmethod
    def init_foo(cls, foo: FooBase):
        pass


class SubFoo(Foo):
    c: str


class SubBar(Bar):
    @classmethod
    def init_foo(cls, foo: SubFoo):
        return super().init_foo(foo)

I put the solution to my problem here in case anyone ever needs it.