New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Overhaul inheritance - LLM #21

Open

Tracked by

daniel-z-kaplan opened this issue Jan 24, 2024 · 1 comment

Open

Tracked by

Overhaul inheritance - LLM #21

daniel-z-kaplan opened this issue Jan 24, 2024 · 1 comment

Labels

reduce technical debt

Collaborator

daniel-z-kaplan commented Jan 24, 2024

No description provided.

Alexis-BX added the reduce technical debt label

Member

Alexis-BX commented Jan 26, 2024 •

edited

Loading

Right now LLMs (in robin/model/language_model/) inherit from LlavaMetaForCausalLM (robin/model/llava_arch.py)

move llava_arch to LM folder
clean children classes (move as much as possible to parent)
Remove all calls to children in other files (eg train.py, train()), replace with single call to parent and parent instantiates the proper child (see multimodal_encoder/builder.py for example
clean up imports

PrateekHumane mentioned this issue

Separate vision encoder and LLM #42

Open

2 tasks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment