Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add change log #303

Merged
merged 1 commit into from
Dec 14, 2024
Merged

add change log #303

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion adalflow/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,10 @@

## [0.2.7] - TO Be Released
### Added
- `Memory` is completed with `call` and `add_dialog_turn` methods.
- Integrated `LanceDB` in the `Retriever`
### Improved
- `BedrockAPIClient` added more details on setup, yet it is still in experimental stage.
- `AzureAPIClient` added more details on setup, yet it is still in experimental stage.
## [0.2.6] - 2024-11-25
### Improved
- Add default `max_tokens=512` to the `AnthropicAPIClient` to avoid the error when the user does not provide the `max_tokens` in the prompt.
Expand Down
4 changes: 4 additions & 0 deletions adalflow/adalflow/components/model_client/bedrock_client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,10 @@ def get_first_message_content(completion: Dict) -> str:
class BedrockAPIClient(ModelClient):
__doc__ = r"""A component wrapper for the Bedrock API client.

Note:

This api is in experimental and is not fully tested and validated yet.

Support:
1. AWS Titan
2. Claude
Expand Down
4 changes: 2 additions & 2 deletions notebooks/adalflow_colab_template.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,8 +87,8 @@
"\n",
"\n",
"# Set environment variables\n",
"os.environ['OPENAI_API_KEY'] = openai_api_key\n",
"os.environ['GROQ_API_KEY'] = groq_api_key\n",
"os.environ[\"OPENAI_API_KEY\"] = openai_api_key\n",
"os.environ[\"GROQ_API_KEY\"] = groq_api_key\n",
"\n",
"print(\"API keys have been set.\")"
]
Expand Down
14 changes: 9 additions & 5 deletions notebooks/evaluation/adalflow_llm_eval.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@
"\n",
"\n",
"# Set environment variables\n",
"os.environ['OPENAI_API_KEY'] = openai_api_key\n",
"os.environ[\"OPENAI_API_KEY\"] = openai_api_key\n",
"\n",
"print(\"API keys have been set.\")"
]
Expand Down Expand Up @@ -301,11 +301,11 @@
"source": [
"# without questions, and with customized judgement query\n",
"\n",
"\n",
"def compute_llm_as_judge_wo_questions():\n",
" from adalflow.eval.llm_as_judge import LLMasJudge, DefaultLLMJudge\n",
" from adalflow.components.model_client import OpenAIClient\n",
"\n",
"\n",
" llm_judge = DefaultLLMJudge(\n",
" model_client=OpenAIClient(),\n",
" model_kwargs={\n",
Expand Down Expand Up @@ -413,7 +413,7 @@
" eval_rslt = llm_evaluator.compute(\n",
" questions=questions, gt_answers=gt_answers, pred_answers=pred_answers\n",
" )\n",
" print(eval_rslt)\n"
" print(eval_rslt)"
]
},
{
Expand Down Expand Up @@ -569,8 +569,12 @@
}
],
"source": [
"source=\"Paul Merson has restarted his row with Andros Townsend after the Tottenham midfielder was brought on with only seven minutes remaining in his team 's 0-0 draw with Burnley on Sunday . 'Just been watching the game , did you miss the coach ? # RubberDub # 7minutes , ' Merson put on Twitter . Merson initially angered Townsend for writing in his Sky Sports column that 'if Andros Townsend can get in ( the England team ) then it opens it up to anybody . ' Paul Merson had another dig at Andros Townsend after his appearance for Tottenham against Burnley Townsend was brought on in the 83rd minute for Tottenham as they drew 0-0 against Burnley Andros Townsend scores England 's equaliser in their 1-1 friendly draw with Italy in Turin on Tuesday night The former Arsenal man was proven wrong when Townsend hit a stunning equaliser for England against Italy and he duly admitted his mistake . 'It 's not as though I was watching hoping he would n't score for England , I 'm genuinely pleased for him and fair play to him \\u00e2\\u20ac\\u201c it was a great goal , ' Merson said . 'It 's just a matter of opinion , and my opinion was that he got pulled off after half an hour at Manchester United in front of Roy Hodgson , so he should n't have been in the squad . 'When I 'm wrong , I hold my hands up . I do n't have a problem with doing that - I 'll always be the first to admit when I 'm wrong . ' Townsend hit back at Merson on Twitter after scoring for England against Italy Sky Sports pundit Merson ( centre ) criticised Townsend 's call-up to the England squad last week Townsend hit back at Merson after netting for England in Turin on Wednesday , saying 'Not bad for a player that should be 'nowhere near the squad ' ay @ PaulMerse ? ' Any bad feeling between the pair seemed to have passed but Merson was unable to resist having another dig at Townsend after Tottenham drew at Turf Moor .\",\n",
"summary=\"Paul merson was brought on with only seven minutes remaining in his team 's 0-0 draw with burnley . Andros townsend scored the tottenham midfielder in the 89th minute . Paul merson had another dig at andros townsend after his appearance . The midfielder had been brought on to the england squad last week . Click here for all the latest arsenal news news .\",\n",
"source = (\n",
" \"Paul Merson has restarted his row with Andros Townsend after the Tottenham midfielder was brought on with only seven minutes remaining in his team 's 0-0 draw with Burnley on Sunday . 'Just been watching the game , did you miss the coach ? # RubberDub # 7minutes , ' Merson put on Twitter . Merson initially angered Townsend for writing in his Sky Sports column that 'if Andros Townsend can get in ( the England team ) then it opens it up to anybody . ' Paul Merson had another dig at Andros Townsend after his appearance for Tottenham against Burnley Townsend was brought on in the 83rd minute for Tottenham as they drew 0-0 against Burnley Andros Townsend scores England 's equaliser in their 1-1 friendly draw with Italy in Turin on Tuesday night The former Arsenal man was proven wrong when Townsend hit a stunning equaliser for England against Italy and he duly admitted his mistake . 'It 's not as though I was watching hoping he would n't score for England , I 'm genuinely pleased for him and fair play to him \\u00e2\\u20ac\\u201c it was a great goal , ' Merson said . 'It 's just a matter of opinion , and my opinion was that he got pulled off after half an hour at Manchester United in front of Roy Hodgson , so he should n't have been in the squad . 'When I 'm wrong , I hold my hands up . I do n't have a problem with doing that - I 'll always be the first to admit when I 'm wrong . ' Townsend hit back at Merson on Twitter after scoring for England against Italy Sky Sports pundit Merson ( centre ) criticised Townsend 's call-up to the England squad last week Townsend hit back at Merson after netting for England in Turin on Wednesday , saying 'Not bad for a player that should be 'nowhere near the squad ' ay @ PaulMerse ? ' Any bad feeling between the pair seemed to have passed but Merson was unable to resist having another dig at Townsend after Tottenham drew at Turf Moor .\",\n",
")\n",
"summary = (\n",
" \"Paul merson was brought on with only seven minutes remaining in his team 's 0-0 draw with burnley . Andros townsend scored the tottenham midfielder in the 89th minute . Paul merson had another dig at andros townsend after his appearance . The midfielder had been brought on to the england squad last week . Click here for all the latest arsenal news news .\",\n",
")\n",
"\n",
"compute_g_eval_summarization(source=source, summary=summary)"
]
Expand Down
143 changes: 77 additions & 66 deletions notebooks/qas/adalflow_object_count_auto_optimization.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -137,12 +137,14 @@
"\n",
"# Prompt user to enter their API keys securely\n",
"openai_api_key = getpass(\"Please enter your OpenAI API key: \")\n",
"groq_api_key = getpass(\"Please enter your GROQ API key, simplly press Enter if you don't have one: \")\n",
"groq_api_key = getpass(\n",
" \"Please enter your GROQ API key, simplly press Enter if you don't have one: \"\n",
")\n",
"\n",
"\n",
"# Set environment variables\n",
"os.environ['OPENAI_API_KEY'] = openai_api_key\n",
"os.environ['GROQ_API_KEY'] = groq_api_key\n",
"os.environ[\"OPENAI_API_KEY\"] = openai_api_key\n",
"os.environ[\"GROQ_API_KEY\"] = groq_api_key\n",
"\n",
"print(\"API keys have been set.\")"
]
Expand Down Expand Up @@ -209,6 +211,7 @@
"<END_OF_USER>\n",
"\"\"\"\n",
"\n",
"\n",
"class ObjectCountTaskPipeline(adal.Component):\n",
" def __init__(self, model_client: adal.ModelClient, model_kwargs: Dict):\n",
" super().__init__()\n",
Expand Down Expand Up @@ -242,9 +245,7 @@
" self, question: str, id: str = None\n",
" ) -> Union[adal.GeneratorOutput, adal.Parameter]:\n",
" output = self.llm_counter(prompt_kwargs={\"input_str\": question}, id=id)\n",
" return output\n",
"\n",
"\n"
" return output"
]
},
{
Expand Down Expand Up @@ -329,44 +330,42 @@
"from adalflow.components.model_client.groq_client import GroqAPIClient\n",
"\n",
"\n",
"if len(os.environ['OPENAI_API_KEY']) > 1:\n",
" gpt_3_model = {\n",
" \"model_client\": OpenAIClient(),\n",
" \"model_kwargs\": {\n",
" \"model\": \"gpt-3.5-turbo\",\n",
" \"max_tokens\": 2000,\n",
" \"temperature\": 0.0,\n",
" \"top_p\": 0.99,\n",
" \"frequency_penalty\": 0,\n",
" \"presence_penalty\": 0,\n",
" \"stop\": None,\n",
" },\n",
" }\n",
" gpt_4o_model = {\n",
" \"model_client\": OpenAIClient(),\n",
" \"model_kwargs\": {\n",
" \"model\": \"gpt-4o\",\n",
" \"max_tokens\": 4000,\n",
" \"temperature\": 0.0,\n",
" \"top_p\": 0.99,\n",
" \"frequency_penalty\": 0,\n",
" \"presence_penalty\": 0,\n",
" \"stop\": None,\n",
" },\n",
" }\n",
"if len(os.environ[\"OPENAI_API_KEY\"]) > 1:\n",
" gpt_3_model = {\n",
" \"model_client\": OpenAIClient(),\n",
" \"model_kwargs\": {\n",
" \"model\": \"gpt-3.5-turbo\",\n",
" \"max_tokens\": 2000,\n",
" \"temperature\": 0.0,\n",
" \"top_p\": 0.99,\n",
" \"frequency_penalty\": 0,\n",
" \"presence_penalty\": 0,\n",
" \"stop\": None,\n",
" },\n",
" }\n",
" gpt_4o_model = {\n",
" \"model_client\": OpenAIClient(),\n",
" \"model_kwargs\": {\n",
" \"model\": \"gpt-4o\",\n",
" \"max_tokens\": 4000,\n",
" \"temperature\": 0.0,\n",
" \"top_p\": 0.99,\n",
" \"frequency_penalty\": 0,\n",
" \"presence_penalty\": 0,\n",
" \"stop\": None,\n",
" },\n",
" }\n",
"\n",
"if len(os.environ['GROQ_API_KEY']) > 1:\n",
" llama_3_1_model ={\n",
" \"model_client\": GroqAPIClient(),\n",
" \"model_kwargs\": {\n",
" \"model\": \"llama-3.1-8b-instant\"\n",
" }\n",
" }\n",
"if len(os.environ[\"GROQ_API_KEY\"]) > 1:\n",
" llama_3_1_model = {\n",
" \"model_client\": GroqAPIClient(),\n",
" \"model_kwargs\": {\"model\": \"llama-3.1-8b-instant\"},\n",
" }\n",
"\n",
"\n",
"question = \"I have a flute, a piano, a trombone, four stoves, a violin, an accordion, a clarinet, a drum, two lamps, and a trumpet. How many musical instruments do I have?\"\n",
"task_pipeline = ObjectCountTaskPipeline(**gpt_3_model)\n",
"print(task_pipeline)\n"
"print(task_pipeline)"
]
},
{
Expand Down Expand Up @@ -467,6 +466,7 @@
"from adalflow.datasets.big_bench_hard import BigBenchHard\n",
"from adalflow.utils.data import subset_dataset\n",
"\n",
"\n",
"def load_datasets(max_samples: int = None):\n",
" \"\"\"Load the dataset\"\"\"\n",
" train_data = BigBenchHard(split=\"train\")\n",
Expand All @@ -479,7 +479,7 @@
" val_data = subset_dataset(val_data, max_samples)\n",
" test_data = subset_dataset(test_data, max_samples)\n",
"\n",
" return train_data, val_data, test_data\n"
" return train_data, val_data, test_data"
]
},
{
Expand Down Expand Up @@ -583,11 +583,11 @@
" def prepare_task(self, sample: Example):\n",
" return self.task.call, {\"question\": sample.question, \"id\": sample.id}\n",
"\n",
" def prepare_eval(\n",
" self, sample: Example, y_pred: adal.GeneratorOutput\n",
" ) -> float:\n",
" def prepare_eval(self, sample: Example, y_pred: adal.GeneratorOutput) -> float:\n",
" y_label = -1\n",
" if (y_pred is not None and y_pred.data is not None): # if y_pred and y_pred.data: might introduce bug when the data is 0\n",
" if (\n",
" y_pred is not None and y_pred.data is not None\n",
" ): # if y_pred and y_pred.data: might introduce bug when the data is 0\n",
" y_label = y_pred.data\n",
" return self.eval_fn, {\"y\": y_label, \"y_gt\": sample.answer}"
]
Expand Down Expand Up @@ -820,7 +820,7 @@
"from adalflow.datasets.types import Example\n",
"\n",
"\n",
"class ObjectCountAdalComponent(adal.AdalComponent):# noqa: F811\n",
"class ObjectCountAdalComponent(adal.AdalComponent): # noqa: F811\n",
" def __init__(\n",
" self,\n",
" model_client: adal.ModelClient,\n",
Expand All @@ -844,12 +844,11 @@
" def prepare_task(self, sample: Example):\n",
" return self.task.call, {\"question\": sample.question, \"id\": sample.id}\n",
"\n",
"\n",
" def prepare_eval(\n",
" self, sample: Example, y_pred: adal.GeneratorOutput\n",
" ) -> float:\n",
" def prepare_eval(self, sample: Example, y_pred: adal.GeneratorOutput) -> float:\n",
" y_label = -1\n",
" if (y_pred is not None and y_pred.data is not None): # if y_pred and y_pred.data: might introduce bug when the data is 0\n",
" if (\n",
" y_pred is not None and y_pred.data is not None\n",
" ): # if y_pred and y_pred.data: might introduce bug when the data is 0\n",
" y_label = y_pred.data\n",
" return self.eval_fn, {\"y\": y_label, \"y_gt\": sample.answer}\n",
"\n",
Expand Down Expand Up @@ -891,7 +890,7 @@
" **gpt_3_model,\n",
" teacher_model_config=gpt_4o_model,\n",
" text_optimizer_model_config=gpt_4o_model,\n",
" backward_engine_model_config=gpt_4o_model\n",
" backward_engine_model_config=gpt_4o_model,\n",
" )\n",
" print(adal_component)\n",
" trainer = adal.Trainer(\n",
Expand All @@ -916,7 +915,7 @@
" test_dataset=test_dataset,\n",
" debug=debug,\n",
" resume_from_ckpt=resume_from_ckpt,\n",
" )\n"
" )"
]
},
{
Expand Down Expand Up @@ -3255,10 +3254,14 @@
}
],
"source": [
"train(debug=False, max_steps=12, strategy=\"constrained\",\n",
" raw_shots=0, bootstrap_shots=1,\n",
" exclude_input_fields_from_bootstrap_demos=True\n",
" )"
"train(\n",
" debug=False,\n",
" max_steps=12,\n",
" strategy=\"constrained\",\n",
" raw_shots=0,\n",
" bootstrap_shots=1,\n",
" exclude_input_fields_from_bootstrap_demos=True,\n",
")"
]
},
{
Expand Down Expand Up @@ -6015,13 +6018,17 @@
}
],
"source": [
"\n",
"ckpt_path = \"/content/adalflow/ckpt/ObjectCountAdalComponent/constrained_max_steps_12_4e8a1_run_1.json\"\n",
"\n",
"train(debug=False, max_steps=12, strategy=\"constrained\",\n",
" raw_shots=0, bootstrap_shots=1,\n",
" resume_from_ckpt=ckpt_path,\n",
" exclude_input_fields_from_bootstrap_demos=True)"
"train(\n",
" debug=False,\n",
" max_steps=12,\n",
" strategy=\"constrained\",\n",
" raw_shots=0,\n",
" bootstrap_shots=1,\n",
" resume_from_ckpt=ckpt_path,\n",
" exclude_input_fields_from_bootstrap_demos=True,\n",
")"
]
},
{
Expand Down Expand Up @@ -8038,11 +8045,15 @@
}
],
"source": [
"\n",
"train(debug=False, max_steps=12, strategy=\"random\",\n",
" raw_shots=0, bootstrap_shots=1,\n",
" resume_from_ckpt=ckpt_path,\n",
" exclude_input_fields_from_bootstrap_demos=False)"
"train(\n",
" debug=False,\n",
" max_steps=12,\n",
" strategy=\"random\",\n",
" raw_shots=0,\n",
" bootstrap_shots=1,\n",
" resume_from_ckpt=ckpt_path,\n",
" exclude_input_fields_from_bootstrap_demos=False,\n",
")"
]
},
{
Expand Down
912 changes: 455 additions & 457 deletions notebooks/tutorials/adalflow_classification_optimization.ipynb
View file
Open in desktop

Large diffs are not rendered by default.

Loading
Loading