Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

llama model linked from howto does not load #5

Open
mdegans opened this issue Jun 4, 2024 · 0 comments
Open

llama model linked from howto does not load #5

mdegans opened this issue Jun 4, 2024 · 0 comments
Assignees

Comments

@mdegans
Copy link
Owner

mdegans commented Jun 4, 2024

Because the tokenizer in the metadata is wrong. My solution to this was to convert from safetensors myself using the latest llama.cpp, but the docs should be fixed. There are models that do work out of the box on HF.

Issue is it has to be a recent conversion since GGUF format changes every 5 minutes. I will probably upload my conversion but with 30mbps cable internet it's going to be a while.

@mdegans mdegans self-assigned this Jun 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant