Support MCP tools with models other than Anthropic

[prd-tuong-nguyen]

Validations

• I believe this is a way to improve. I'll try to join the Continue Discord for questions
• I'm not able to find an open issue that requests the same enhancement

Problem

We can currently only use the tools with Anthropic

Solution

Permit all LLM to use MCP's tools.

[xiaoshyang]

I am using an openai-api compatible interface calling method to call the claude deployed by AWS. Please provide support as well. thank you

[VikashLoomba]

+1, no real reason to limit the tools to the Anthropic provider and Claude only. Plenty of open source models support tool use

[shermanhuman]

Question: Is this something we can accomplish in a generic way? Can we write one thing, and then possibly add a configuration parameter to turn it on for each model - or maintain a database of models / providers that have that capability?

Or is this something we need to implement on a provider by provider basis? If that's the case instead of asking for "all LLM" we might want to start compiling a list of precisely what is wanted. @prd-tuong-nguyen @xiaoshyang @VikashLoomba could you elaborate a bit on what you are trying to do with tools and what providers and APIs and models you would like tool support on?

[prd-tuong-nguyen]

@shermanhuman I think we should prioritize support for the OpenAI Compatible format, as nearly all LLM serving frameworks and providers support this format. In other words, if we support OpenAI Compatible, we support nearly "all LLM"

[VikashLoomba]

I agree with what @prd-tuong-nguyen said. If we can enable OpenAI based tool calling (perhaps by a per-model setting "tools_enabled" in config.json) first, that would be most useful.