You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In the current re-design of the API we added Generator objects that accept a model and an output type as parameters:
fromoutlinesimportGenerator, modelsfrompydanticimportBaseModelclassFoo(BaseModel):
bar: strmodel=models.openai("gpt-4o")
generator=Generator(model, Foo)
result=generator("prompt")
The rationale behind this design is that, for open source models, the index for structured generation can take some time to compile and we hold it in this object so it can be re-used. There are a few issues with this design:
API models do not have this problem
This adds an extra step in the happy pip install outlines -> text generated path
It does not correspond to most people's mental model. Other libraries make you call the model instance with the prompt to generate text.
I suggest that we modify the original design to make simple use cases straightforward, but still allows flexibility in more complex cases. We can add a __call__ method to model instances so the previous workflow can be re-written as:
fromoutlinesimportGenerator, modelsfrompydanticimportBaseModelclassFoo(BaseModel):
bar: strmodel=models.openai("gpt-4o")
result=model("prompt", Foo)
While keeping the Generator pattern for cases where a user may want to call the same model repeatedly with the same structure:
fromoutlinesimportGenerator, modelsfrompydanticimportBaseModelclassFoo(BaseModel):
bar: strmodel=models.openai("gpt-4o")
generator=Generator(model, Foo)
result_1=generator("prompt_1")
result_2=generator("prompt_2")
The text was updated successfully, but these errors were encountered:
In the current re-design of the API we added
Generator
objects that accept a model and an output type as parameters:The rationale behind this design is that, for open source models, the index for structured generation can take some time to compile and we hold it in this object so it can be re-used. There are a few issues with this design:
pip install outlines -> text generated path
I suggest that we modify the original design to make simple use cases straightforward, but still allows flexibility in more complex cases. We can add a
__call__
method to model instances so the previous workflow can be re-written as:While keeping the
Generator
pattern for cases where a user may want to call the same model repeatedly with the same structure:The text was updated successfully, but these errors were encountered: