Typing kwargs when passed a TypedDict

[oliverbeagley-pgg]

I'm aware that v1.1.380 brought some changes to TypedDict:

• Changed the behavior of the dictionary expansion operator (**) when used in an argument expression for a call and the operand is a TypedDict. It now takes into account the fact that a (non-closed) TypedDict can contain additional keys with object values. This new behavior is consistent with mypy.

However I am having issues with pyright complaining about using it and I am unsure whether this is my understanding of the type system is lacking or if it is a bug.

from typing import TypeAlias
from typing import TypedDict

OptionalScalarType: TypeAlias = bool | float | int | str | None


class Person(TypedDict):
    name: str
    age: int


def foo(**kwargs: OptionalScalarType) -> None:
    print(kwargs)


person: Person = {"name": "John", "age": 30}


foo(**person)  # error

The last line has the error

error: Argument of type "object" cannot be assigned to parameter "kwargs" of type "OptionalScalarType" in function "foo"
    Type "object" is not assignable to type "OptionalScalarType"
      "object" is not assignable to "bool"
      "object" is not assignable to "float"
      "object" is not assignable to "int"
      "object" is not assignable to "str"
      "object" is not assignable to "None" (reportArgumentType)

This doesn't complain on v1.1.379, but I don't understand why it does on v1.1.380+.

If this behaviour is intended, I'm in the situation where I am not in control of function foo so would like to know how I can do something like foo(**person) and not have the type checker complain.

(problem in particular I'm facing is with the .options method on DataFramerReader/Writer in pyspark, I've pulled out some boilerplate into a dict and now pyright isn't happy about it)

[erictraut]

Pyright's current behavior is correct, and it conforms to behavior mandated by the Python typing spec.

The issue is that TypedDict classes are structural and are not "closed", which means that a dict value can be compatible with a TypedDict definition even if it has extra items that are not defined in that TypedDict. Static type checkers must therefore assume that extra items are present and their values are of type object.

There is a draft PEP 728 that proposes to add the concept of "closed" TypedDicts to the type system. Pyright has provisional support for this PEP which can be enabled with the enableExperimentalFeatures setting. However, the runtime doesn't yet support this PEP, so you won't be able to use it in your code until this PEP gets further along.

Here's how it would look if you were to use a closed TypedDict.

Code sample in pyright playground

from typing import TypeAlias, TypedDict

OptionalScalarType: TypeAlias = bool | float | int | str | None

class Person(TypedDict, closed=True):
    name: str
    age: int

def foo(**kwargs: OptionalScalarType) -> None:
    print(kwargs)

person: Person = {"name": "John", "age": 30}

foo(**person)

In the meantime, you have a few options:

1. Don't use dictionary expansion when calling your function. Pass the keyword arguments by name, as in foo(name="John", age=30).
2. Use a cast: foo(**cast(dict[str, OptionalScalarType], person))
3. Suppress the error using a # type: ignore or # pyright: ignore comment

[oliverbeagley-pgg]

In the Supported and Unsupported Operations section it mentions around type checkers ensuring extra and missing keys are caught during construction, does that not mean that person: Person = {"name": "John", "age": 30} must only have the keys as in Person (subject to NotRequired etc)?

I understand from a function signature point of view that giving an argument a hint of a Person does not mean that only those keys exist as a subtype could inherit from Person and add new keys which the function should still accept but anticipate potential new keys as object. But is it not the case that by the time that I have done something like person: Person that nothing but the keys defined in the class Person can exist in the variable person? And as such if I pass person to foo there are no additional keys potentially being object for it to worry about?

[erictraut]

You are correct that the spec indicates that if a TypedDict object is constructed using a dictionary literal expression, then type checkers must enforce that no other keys are present. This was a controversial part of the spec because it's arguably inconsistent with the rest of the spec. It was justified as a way to catch key misspellings, which is a frequent source of bugs. There are other ways that TypedDicts can be constructed besides dictionary literal expressions, and they do not enforce that no other keys are present.

As I said, pyright is working correctly here and in conformance with the typing spec. If I were to make a change like the one you're proposing, it would make pyright non-conformant and would break assumptions in other typed code.