Offline environment tabby freezes without any output

[magiccaptain]

Describe the bug
I would like to use Tabby in an offline environment. However, when I disconnect from the internet and run the command TABBY_ROOT=/tabby_home ./tabby serve --model StarCoder-1B --chat-model Qwen2-1.5B-Instruct, the program freezes without any output. But once I reconnect to the internet, the program starts up normally. Is it necessary for the Tabby program to access the network every time it is launched? And is there any way to prevent this from happening? I have already downloaded the Tabby models files in the tabby_home directory.

Information about your version
tabby 0.21.1

Information about your GPU
CPU

[zwpaper]

Hi @magiccaptain, Tabby can operate in an offline environment. However, by default, when you specify a model name in the command arguments, Tabby will attempt to fetch the model from the Tabby Registry online.

Here are two options to consider:

1. Follow the instructions at https://tabby.tabbyml.com/docs/references/models-http-api/llama.cpp/ to use an HTTP model, preventing Tabby from downloading one.

2. Copy the existing ~/.tabby/models/TabbyML/models.json and the three specified model directories ~/.tabby/models/TabbyML/${MODEL_NAME} to the offline environment.

After completing one of these steps, you can run Tabby offline.

[topcheer]

try to blacklist github.com and raw.githubusercontent.com in your /etc/hosts by adding following line before you run tabby
0.0.0.0 github.com raw.githubusercontent.com

[zwpaper]

Hi @topcheer, I've conducted the test you mentioned, and it works on my end:

I also performed a test on Linux with the ethernet disabled, and it worked as well.

[xiaoshidetaikuai]

I installed Tabby in an online environment, and it worked properly. Then I copied the Tabby executable and the .tabby folder to an offline Linux system. However, it reported an "Invalid 'model_id'" error. The error message is as follows:
The application panicked(crashed).
Message: Invalid model_id <TabbyML/starCode-1B>;please consult https://github.com/TabbyML/registry-tabby for the correct model_id
Locaition: crates/tabby-common/sc/registry.rs:168
Information about your version
tabby v0.24.0-rc.1 tabby_x86_64-manylinux_2_28.tar.gz

Information about your GPU
CPU