Added falcon model converter #2040
Conversation
| class TestTask(TestCase): | ||
| @pytest.mark.large | ||
| def test_convert_tiny_preset(self): | ||
| model = FalconCausalLM.from_preset("hf://tiiuae/falcon-7b") |
There was a problem hiding this comment.
I don't think we can afford to download this ~15gb file in our testing setup. You could try the 1b model? Or create a small test model on hf, as was done for llama and others.
There was a problem hiding this comment.
@mattdangerw - I'll create small test with 1b falcon model and commit again.
|
@SamanehSaadat can you take a look for the falcon conversions options here? I remember there were some annoying gotchas (e.g. different tokenizer types), that this might not conver. |
|
|
||
| @pytest.mark.large | ||
| def test_class_detection(self): | ||
| model = FalconCausalLM.from_preset("hf://tiiuae/falcon-7b") |
There was a problem hiding this comment.
Does this work? I think we only have Falcon-1b support! 7b model has a different attention mechanism which hasn't been added!
There was a problem hiding this comment.
We should probably also attach a colab verifying that output from the huggingface and KerasHub versions align. And sound like that might actually run into differences here due to what @SamanehSaadat is saying.
@SamanehSaadat how much work is needed of the architecture code to support the 7 and other variants? Is it something that could be added here or a ton to do?
There was a problem hiding this comment.
@mattdangerw I think adding support for the 7b is non-trivial. There are some major architectural differences like alibi, GQA vs. MHA, and rotary embedding (to me, it's almost like adding a new architecture!).
There was a problem hiding this comment.
Thanks! Sounds like we will need to either throw in the converter if we encounter the falcon huggingface options we don't currently support, or add them in (on a separate pr?).
@mehtamansi29 we'd probably need a colab verifying that the output matches for some subset of falcon checkpoints on huggingface, and ideally that we throw for falcon checkpoints that needs arch options we don't yet support.
Falcon model converter is missing. Added the same. Fixes #1988