feat: Use TypeAliasType to define aliases

[Viicos]

See langchain-ai/langchain-google#610 (comment) for more context. The TL;DR is that it is very hard (if not impossible) for Pydantic to resolve the forward references inside these aliases. Taking the following simplified example:

# module1.py

from typing import List

Alias = List['SomeType']

SomeType = int

# module2.py

from module1 import Alias
from pydantic import BaseModel

class Model(BaseModel):
    a: Alias

It is strictly equivalent to:

class Model(BaseModel):
    a: List['SomeType']

because Alias is just a simple variable.

By using TypeAliasType instead (or the new type syntax once support for 3.11 is dropped), we can access the __module__ attribute of the TypeAliasType instance and grab the SomeType alias from there.

Moreover, due to how Python caches the evaluated value of forward references, this led to a bug in https://github.com/langchain-ai/langchain-google, see my comment for more details.

---

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

• Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
• Ensure the tests and linter pass
• Code coverage does not decrease (if any source code was changed)
• Appropriate docs were updated (if necessary)
• Get the necessary approvals
• Once the last commit on the PR has been approved, add the "ready to pull" label to the Pull Request

Note: do not merge your PR from GitHub. Adding the "ready to pull" label is the final step in the review process.
After approvals, the changes in your PR will be committed to the main branch and this PR will be closed.

Fixes #<issue_number_goes_here> 🦕

[Ark-kun]

Hello. To be honest, I'm not sure we wanted these type aliased to be exported and used outside. We wanted the type aliases to be inlined so that function signatures are fully resolved to actually importable types, not union aliases.
If I understand correctly, this PR moves to an opposite direction (formalization of type aliases).

Since, AFAIK, this does not make any SDK surface changes and is not breaking backwards compatibility, I think I can approve this change. However I think it might be best to find a better and more robust long-term workaround (e.g. not using these internal type aliases from other libraries).

[Viicos]

Hello,

indeed it seems that these aliases are only used for function annotations in this project (and not as Pydantic fields annotations). It might be best getting in touch with the maintainers at https://github.com/langchain-ai/langchain-google to discuss this.

[matthew29tang]

The unit test infra builds are failing with the following error (due to bumping the lower bound of typing_extensions):

The conflict is caused by:
google-cloud-aiplatform 1.73.0 depends on typing_extensions>4.6.0
google-cloud-aiplatform[testing] 1.73.0 depends on typing_extensions>4.6.0
tensorflow 2.13.0 depends on typing-extensions<4.6.0 and >=3.6.6

Can you adjust the bounds and verify that the google-cloud-aiplatform[testing] extras installation works?

[Viicos]

I'm not sure what's reason to have the different versions here:

python-aiplatform/setup.py

Lines 208 to 210 in dfe6d6c

	# Lazy import requires > 2.12.0
	"tensorflow == 2.13.0; python_version<='3.11'",
	"tensorflow == 2.16.1; python_version>'3.11'",

2.13.0 was the last tensorflow version to support Python 3.8, and also the last one having an upper bound on typing-extensions.

So I see two ways to go forward:

• this project drops support for Py3.8, so that tensorflow can be bumped.
• I remove the typing-extensions constraint, and we hope that users are going to have a recent version of typing-extensions installed.

[pedroallenrevez]

Any news on this? Highly sought after

[matthew29tang]

We aren't planning to drop python 3.8 support at this time. But perhaps we can see if we can mitigate this issue for python >3.8 users. Maybe there is a combination of dependencies for >python 3.8 that we can have typing_extensions>4.6.0?

[pedroallenrevez]

https://devguide.python.org/versions/

Python 3.8 reached EOL @matthew29tang .

Why keep supporting legacy?

[yeesian]

Hi @Viicos thank you for the PR (and thank you everyone else for the feedback) -- sorry for my slow response here -- can you make the following changes:

1. wrap the changes in a try-except block,
2. remove the typing-extensions constraint and lift the pydantic upperbounds for other parts of the SDK (that are correlated with generative_models) across the other sites like in feat: Use TypeAliasType to define aliases for union types in generative models #4761
3. (optional) drop the unit tests for Langchain with Python 3.8 (being done in chore: Remove unit testing for LangChain in Python 3.8 #4762)

(If you might prefer -- I'm also happy to take over from here in #4761. Otherwise I'd prefer to preserve your authorship of changes and full context in this PR.)

[Viicos]

Hi @yeesian,

Feel free to pick this up. Maybe it could be worth fixing the issue at https://github.com/langchain-ai/langchain-google instead, as it is importing the private SafetySettingsType type alias?

Otherwise, the TypeAliasType fix here should still work. If you feel like it, you can add myself as a co-author as Victorien <[email protected]>.

[prousso]

Any news on this ?

[yeesian]

Sorry for the slowness from me here, this change is on the main branch now 2224c83, and will be available in the next release (i.e. v1.76.0). @Viicos, I wasn't able to set the authorship due to the way https://github.com/copybara-github works, really sorry about it.

[Viicos]

No worries. Should I close this one then?

[pedroallenrevez]

Can I help somehow? this doesn't allow using new pydantic-ai which is an absolute banger. This issue has been like this for 1 month

[ostefano]

@yeesian what is the status of this? Is v1.76.0 being released?

Update: Released, thank you 🙏