Adding declarative HTTP tools to autogen ext

[EItanya]

Why are these changes needed?

These changes are needed because currently there's no generic way to add tools to autogen studio workflows using the existing DSL and schema other than inline python.

This API will be quite verbose, and lacks a discovery mechanism, but it unlocks a lot of programmatic use-cases.

Related issue number

#5170

Checks

• I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
• I've added tests (if relevant) corresponding to the changes introduced in this PR.
• I've made sure all auto checks have passed.

[jackgerrits]

There is already a community extension for MCP here: https://github.com/richard-gyiko/autogen-ext-mcp

Would be consider adding component config support there if the owner is willing? Then things will work in studio.

@richard-gyiko would you be willing to support component config for mcp tools? Once implemented, MCP tools will be usable in studio.

[EItanya]

I understand there's a community extension, this impl is clearly almost entirely based on it, but as I've noted in other issues and PRs in which I've been having discussions around this, autogenstudio not having a way to call generic tools hugely limits its usefulness. I do think switching over to ComponentConfig makes sense, and I was actually about to do that. However, I think this functionality needs to exist in core autogenstudio in some way. Not necessarily for me but for the tool as a whole. Also the component config is still blocked on this issue, which seems to be quite a bit of work unless I'm missing something.

The context for this draft can be found:

1. External tool calling API for autogen studio #5170 (comment)
2. Make FunctionTools Serializable (Declarative) #5052 (comment)
3. External tool calling API for autogen studio #5170

[jackgerrits]

Given the importance of MCP we should consider making it part of AutoGen itself, I agree.

Yes, that #5172 is a big issue, but it's top pri right now. Once this is implemented, studio will be able to use any component that follows the component config framework (whether it's within the AutoGen-* packages or some other pypi package). This means that MCP community extension (provided it adds support too) will just work in studio. It's going to unlock a lot of opportunities and is a big deal for studio. It's just not quite ready yet.

So, overall MCP and API based tools are critical for AutoGen and studio. Just need to decide if it should be a part of this repo or rely on community extensions.

@victordibia @ekzhu thoughts?

[EItanya]

That all makes total sense to me.

We're very interested in unlocking the use-cases #5172 will bring and I'm happy to help with the implementation if @victordibia thinks it can be split up in any way.

As far as whether or not MCP is or isn't a part of the core repo I agree with you that it's probably a discussion worth having AFTER #5172 is done rather than before because it does change the nature of extension.

[victordibia]

This is great! I agree with all the conversation so far!
I'll try to wire up an initial PR for #5172 and will update here.
I'll ping you @EItanya on that PR too and we can pair on it to test and get it merged. And then we can test this PR also.

[richard-gyiko]

There is already a community extension for MCP here: https://github.com/richard-gyiko/autogen-ext-mcp

Would be consider adding component config support there if the owner is willing? Then things will work in studio.

@richard-gyiko would you be willing to support component config for mcp tools? Once implemented, MCP tools will be usable in studio.

@jackgerrits I've created a PR if needed: richard-gyiko/autogen-ext-mcp#5
Hope this implements what you were talking about.

[ekzhu]

@richard-gyiko what do you think about making the MCP extension an official extension as part of autogen-ext?

[richard-gyiko]

@richard-gyiko what do you think about making the MCP extension an official extension as part of autogen-ext?

I think it's a good idea! I feel like having the ability to use MCP server tools has at least the same weight as having the ability to use Langchain ones. Since the autogen-ext already has LangchainToolAdapter it just feels natural to have the MCPToolAdapter in there as well.

[victordibia]

As far as whether or not MCP is or isn't a part of the core repo I agree with you that it's probably a discussion worth having AFTER #5172 is done rather than before because it does change the nature of extension.

After thinking on this a bit more, this PR is trying to do two things

• 1. introduce two types of tools in extensions (http tool and mcp tool)
• 2. integrate these tools in AutoGen Studio

Given that there is work to make tools work "automatically" in AGS, I would say we should refocus this PR on just introducing the tools (with some cordination with @richard-gyiko ). It would also be good to update the description with usage example. e.g, something like

http_tool = HttpTool(....)  # a concrete tool instance 
http_tool_config = HttpTool.dump_component()

print(http_tool_config)

# result of dumped component

[ekzhu]

I think it's a good idea! I feel like having the ability to use MCP server tools has at least the same weight as having the ability to use Langchain ones. Since the autogen-ext already has LangchainToolAdapter it just feels natural to have the MCPToolAdapter in there as well.

This is great! Given what @victordibia already said, let's refocus to just the extension. My suggestion is:

1. @richard-gyiko to create a separate PR for the MCP adapter to the autogen_ext.tools.mcp namespace
2. @EItanya to create another PR to the autogen_ext.tools.http namespace for the HTTP tool, the naming is not final.

For the http tool, it seems it is only for POST requests? Do you think it should support both GET and POST? Also, what is the reason for choosing httpx over a more mature library like requests?

[EItanya]

I think it's a good idea! I feel like having the ability to use MCP server tools has at least the same weight as having the ability to use Langchain ones. Since the autogen-ext already has LangchainToolAdapter it just feels natural to have the MCPToolAdapter in there as well.

This is great! Given what @victordibia already said, let's refocus to just the extension. My suggestion is:

1. @richard-gyiko to create a separate PR for the MCP adapter to the autogen_ext.tools.mcp namespace
2. @EItanya to create another PR to the autogen_ext.tools.http namespace for the HTTP tool, the naming is not final.

For the http tool, it seems it is only for POST requests? Do you think it should support both GET and POST? Also, what is the reason for choosing httpx over a more mature library like requests?

This plan sounds great to me. I will add the HTTP tool, name pending. I will remove all MCP related code from this PR.

I do think as far as adding MCP tools as an extension it's worth having a discussion about the interface for it given that it is serializable now, and I don't think it necessarily makes sense for the user to know the Tool schema ahead of time since it's already available via the server. @richard-gyiko we can follow up about this in your PR.

As far as GET vs POST, I was going to add method later I was just playing around with stuff here, it's definitely still missing a lot of things. In terms of httpx vs requests, requests doesn't support async so basically everyone is moving at this point. openai, etc.

In addition @victordibia I had a lot of free time and went ahead and started on the code to update the server to use the new loading style 🙈. You can find it here: #5200. It's definitely still in super rough shape, and it absolutely won't work with the UI, but I managed to run the autogenstudio:serve endpoint with the following config and it worked!

{"provider":"autogen_agentchat.teams.RoundRobinGroupChat","component_type":"team","version":1,"component_version":1,"description":null,"config":{"participants":[{"provider":"autogen_agentchat.agents.AssistantAgent","component_type":"agent","version":1,"component_version":1,"config":{"name":"assistant","model_client":{"provider":"autogen_ext.models.openai.OpenAIChatCompletionClient","component_type":"model","version":1,"component_version":1,"config":{"model":"gpt-4o","api_key":"YOUR_API_KEY"}},"handoffs":[{"target":"flights_refunder","description":"Handoff to flights_refunder.","name":"transfer_to_flights_refunder","message":"Transferred to flights_refunder, adopting the role of flights_refunder immediately."},{"target":"user","description":"Handoff to user.","name":"transfer_to_user","message":"Transferred to user, adopting the role of user immediately."}],"model_context":{"provider":"autogen_core.model_context.UnboundedChatCompletionContext","component_type":"chat_completion_context","version":1,"component_version":1,"config":{}},"description":"An agent that provides assistance with ability to use tools.","system_message":"Use tools to solve tasks.","reflect_on_tool_use":false,"tool_call_summary_format":"{result}"}}],"termination_condition":{"provider":"autogen_agentchat.conditions.MaxMessageTermination","component_type":"termination","version":1,"component_version":1,"config":{"max_messages":2}}}}

[EItanya]

@microsoft-github-policy-service agree

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 50 lines in your changes missing coverage. Please review.

Project coverage is 69.78%. Comparing base (b6597fd) to head (9c4a00b).

Files with missing lines	Patch %	Lines
...togen-ext/src/autogen_ext/tools/http/_http_tool.py	0.00%	48 Missing ⚠️
...autogen-ext/src/autogen_ext/tools/http/__init__.py	0.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5181      +/-   ##
==========================================
- Coverage   70.08%   69.78%   -0.31%     
==========================================
  Files         179      181       +2     
  Lines       11620    11670      +50     
==========================================
  Hits         8144     8144              
- Misses       3476     3526      +50     

Flag	Coverage Δ
unittests	69.78% <0.00%> (-0.31%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.