How does Pylance/Pyright get the documentation string for a NamedTuple's field?

[bmitc]

Let's say that I have a NamedTuple:

from typing import NamedTuple

class LogItem(NamedTuple):
    """Represents a log item"""

    message: str
    """The message sent"""

    response: str
    """The response received"""

And when I hover over the fields, the documentation string given shows up, which is extremely useful.

However, in Python itself, I don't know how to access the documentation strings for the fields. For example:

>>> LogItem.__doc__
'Represents a log item'
>>> LogItem.message.__doc__
'Alias for field number 0'

How is Pyright getting the documentation string? Is it using a super custom parser, or is there a simple method that I could use in pure Python to get each field's documentation string?

If dataclasses are an alternative for this, I'd be open to that as well. Thank you for any help!

[erictraut]

Pyright implements its own parser (written in TypeScript). I don't think the CPython parser retains variable docstrings in the AST it produces. @JelleZijlstra could probably confirm.

[bmitc]

Thank you both!

This is now probably beyond the scope of Pyright, but do either of you know a clever way to do this, maybe with decorators or some clever use of default values?

The idea is that the NamedTuples could act as a schema such that the fields' values and descriptions could be written to a file at runtime, while the fields' types are validated by MyPy. Of course, the field descriptions could be attached to the named tuple using standard methods, but I was looking for a lightweight approach.

[JelleZijlstra]

You could do something like PEP 727 (https://peps.python.org/pep-0727/). Though that PEP is unlikely to be accepted, you can put whatever you want in Annotated[] and use it in custom tooling.

[bmitc]

Thanks all for the help! Going to mark this as answered, since it is from the Pylance/Pyright side of things.

[Viicos]

@bmitc If you're interested, I implemented something similar in Pydantic: pydantic/pydantic#6563.

We use the AST to extract the variable docstrings. There's a lot of history in the linked PR as getting the source code of the class can be tricky, but for most cases you'll be fine using inspect.getsourcelines.

[bmitc]

I'll take a look. Thanks for the heads up!

I'm wondering if CPython would be open to this change. I'm curious if that's been proposed in such a limited manner for NamedTuple and dataclasses. At least for NamedTuple, CPython already keeps around a string for fields' __doc__ string, it's just the "wrong" string and not useful.

>>> from typing import NamedTuple
>>> class Test(NamedTuple):
...     FieldA: int
...     """This is the A field"""
...
>>> Test.FieldA.__doc__
'Alias for field number 0'