[Feature] Exposing a ChatController

[DominikStarke]

I think it'd be great if the chat exposed a controller for more flexible handling of the following scenarios:

• Attaching files (For example ClipboardData) - right now this is possible with a custom messageSender, but the files aren't visible in the chat before submission
• Call sendMessage programmatically. For example there might be predefined prompts, which should be executed on click.
• Manage focus of the Chats Input. Especially on Desktop there are Scenarios, where manual focus management might be required (Also the current approach behaves kinda weird)

[csells]

• Attaching files (For example ClipboardData) - right now this is possible with a custom messageSender, but the files aren't visible in the chat before submission

Can you give me an example of what you'd like to accomplish here?

• Call sendMessage programmatically. For example there might be predefined prompts, which should be executed on click.

You can do that today with the suggestions parameter of the LlmChatView constructor.

• Manage focus of the Chats Input. Especially on Desktop there are Scenarios, where manual focus management might be required (Also the current approach behaves kinda weird)

Please log any bugs along with repro samples against any issues you find with the current approach to focus.

What kind of scenario would you like to handle with manual focus management?

[DominikStarke]

Can you give me an example of what you'd like to accomplish here?

Well I'm filling in files from the Clipboard: https://github.com/DominikStarke/ogre
Right now I'm tracking them outside the chat window.

You can do that today with the suggestions parameter of the LlmChatView constructor.

That might work. What I want to achieve is if the user has something in his clipboard, he can invoke some "quick-actions", like writing a summary, or similar.

Please log any bugs along with repro samples against any issues you find with the current approach to focus.

I'll upload a video in a different issue. Soft-keyboards can't be hidden on mobile and on navigation once the keyboard loses focus and regains focus, just to be then dropped again, because the route is now out of focus. It makes for a very unpleasent route transition. That's on ios at least.

What kind of scenario would you like to handle with manual focus management?

When running on desktop my app has a simple hotkey, which brings it to front. When this happens I want to focus the input field (like in the video of the repo above). However right now it won't do that if the input isn't focused before the application is hidden.

[csells]

Can you give me an example of what you'd like to accomplish here?

Well I'm filling in files from the Clipboard: DominikStarke/ogre Right now I'm tracking them outside the chat window.

I'm sorry. I don't understand. Can you frame it in user actions, the results you'd like to provide to them and how you'd like the AIT to help with that?

What kind of scenario would you like to handle with manual focus management?

When running on desktop my app has a simple hotkey, which brings it to front. When this happens I want to focus the input field (like in the video of the repo above). However right now it won't do that if the input isn't focused before the application is hidden.

Great. If you could log a feature request for that functionality, that would be great.

[DominikStarke]

Let's say the user goes to a website, this page right here for example.
He sees there's already quite a lot of conversation going on and instead of reading everything he then:

• Selects all the text on the website
• Copies the selection to his clipboard
• Opens my app
• Now the app has the copied text as Context for the chat.

So far so good, that works already.
However I've had to create my own widget to visualize these kind of attachments.

Ideally I'd like them in the same area as the attachments opened with the file dialog (right over the chat input).

I hope this makes it more clear.

Here's a short Video:

freecompress-Bildschirmaufnahme.2024-12-26.um.22.44.36.mp4

[csells]

so, so your request to add support for the clipboard as an attachment, either manually or automatically? or is it for the app to be able to define custom attachment types?

[DominikStarke]

I'm lookiong for a way to add these attachments to the chat windows attached files, like normal attachments:

I don't expect the package itself to handle clipboard data.

[csells]

have you tried doing this with a messageSender?

[DominikStarke]

Yes, I mentioned that earlier. Submission with messageSender works (like a charm) and the attachments are displayed in the chat history correctly.
My issue is rather... reducing visual clutter:

[csells]

understood. how should this work programatically?

[DominikStarke]

Easiest solution

Toggles to enable / disable the different buttons....

LlmChatView(
  provider: EchoProvider(),
  showFileUploadButton: true | false,
  showImageUploadButton: true | false,
  ...
);

Then it's possible to customize file handling...

Easy solution

Probably an attachments parameter to LlmChatView?

final controller = LlmChatViewController();
LlmChatView(
  attachments: [...],
  provider: EchoProvider()
);

Harder solution

I wanted to avoid calling setState in an LlmChatView parent however, so that's where the "controller" idea comes from...

final controller = LlmChatViewController();
LlmChatView(
  controller: controller, // Optional
  provider: EchoProvider()
);

We could then have something like controller.addAttachment(...), which would internally notify listening Widgets.
Same for requesting focus controller.requestFocus().
I also think controller.addSuggestions(...), controller.setHistory(...) would make sense, because otherwise its always necessary to call setState on the parent?

Or am I missing something?