Cache models in a class attribute to avoid one network request per provider

[julien-nc]

As OpenAI apparently has rate limit on the models endpoint, page loading and requests to /ocs/v2.php/apps/assistant/api/v1/task-types can be slowed down quite a bit because each provider gets the list of models to populate the enum value in their optional input shapes.

Thankfully they all use the same instance of OpenAiAPIService so the model request response can be simply cached in a class attribute.

[kyteinsky]

This cache will be lost for different requests though, no? Different request -> new instance of the service. Would be better to use the local cache with userid in the key, what say you?
Also, with a timeout.

[julien-nc]

This cache will be lost for different requests though, no?

Yes

Would be better to use the local cache with userid in the key, what say you?

With a local cache we can get outdated values. Even if the model list should not change very often, we can't be sure.

Also, with a timeout.

Timeout on what? The request to the models endpoint?