Error : Invalid model file when runing scripts/local/run-dev

[abdshomad]

I have successfully install the dependencies, but when i try to Run Locally, by running scripts/local/run-dev. There is error below:

Found model file at  llm/local/ggml-gpt4all-j-v1.3-groovy.bin
Invalid model file
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/demo/miniconda3/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/demo/learn/GPT/rag-stack/server/server/main.py", line 38, in <module>
    llm = get_selected_llm()
  File "/home/demo/learn/GPT/rag-stack/server/llm/llm.py", line 24, in get_selected_llm
    return Gpt4AllLLM()
  File "/home/demo/learn/GPT/rag-stack/server/llm/llm.py", line 40, in __init__
    self.llm = GPT4All(
  File "/home/demo/.cache/pypoetry/virtualenvs/rag-stack-5gekgb59-py3.10/lib/python3.10/site-packages/langchain/load/serializable.py", line 74, in __init__
    super().__init__(**kwargs)
  File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for GPT4All
__root__
  Unable to instantiate model (type=value_error)
Stopping Docker containers...

Further information:
~ cat /etc/os-release

PRETTY_NAME="Ubuntu 22.04.3 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.3 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

~ python --version
Python 3.10.9

~ nvidia-smi

Sun Aug  6 11:28:10 2023       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.125.06   Driver Version: 525.125.06   CUDA Version: 12.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA A30          On   | 00000000:00:10.0 Off |                    0 |
| N/A   32C    P0    29W / 165W |   6255MiB / 24576MiB |      0%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+
|   1  NVIDIA A30          On   | 00000000:00:11.0 Off |                    0 |
| N/A   31C    P0    29W / 165W |      7MiB / 24576MiB |      0%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A     12237      G   /usr/lib/xorg/Xorg                  4MiB |
|    0   N/A  N/A    638035      C   /usr/bin/python3                 3196MiB |
|    0   N/A  N/A   2951384      C   ...keai/.venv/bin/python3.10     3052MiB |
|    1   N/A  N/A     12237      G   /usr/lib/xorg/Xorg                  4MiB |
+-----------------------------------------------------------------------------+

~ pip list

Package                   Version
------------------------- ----------
absl-py                   1.4.0
accelerate                0.20.3
aiofiles                  23.1.0
aiohttp                   3.8.5
aiosignal                 1.3.1
altair                    5.0.1
antlr4-python3-runtime    4.9.3
anyio                     3.7.1
appdirs                   1.4.4
APScheduler               3.10.1
arrow                     1.2.3
async-timeout             4.0.2
attrs                     23.1.0
Authlib                   1.2.1
auto-gptq                 0.2.2
backoff                   2.2.1
beautifulsoup4            4.12.2
bioc                      2.0
bitsandbytes              0.39.0
blinker                   1.6.2
boltons                   23.0.0
boto3                     1.26.101
botocore                  1.29.101
bravado                   11.0.3
bravado-core              6.1.0
brotlipy                  0.7.0
cachetools                5.3.0
certifi                   2022.12.7
cffi                      1.15.1
chardet                   5.2.0
charset-normalizer        3.2.0
chromadb                  0.3.25
click                     8.1.6
clickhouse-connect        0.6.8
cmake                     3.27.0
colorama                  0.4.6
coloredlogs               15.0.1
conda                     23.3.1
conda-content-trust       0.1.3
conda-package-handling    2.0.2
conda_package_streaming   0.7.0
contourpy                 1.1.0
cryptography              41.0.3
cycler                    0.11.0
dataclasses-json          0.5.14
datasets                  2.13.0
decorator                 5.1.1
dill                      0.3.6
diskcache                 5.6.1
docutils                  0.20.1
docx2txt                  0.8
duckdb                    0.8.1
effdet                    0.4.1
einops                    0.6.1
et-xmlfile                1.1.0
evaluate                  0.4.0
exceptiongroup            1.1.2
execnet                   2.0.2
faiss-cpu                 1.7.4
fastapi                   0.101.0
ffmpy                     0.3.1
filelock                  3.12.2
filetype                  1.2.0
fire                      0.5.0
Flask                     2.3.2
Flask-SQLAlchemy          3.0.5
flatbuffers               23.5.26
fonttools                 4.42.0
fqdn                      1.5.1
frozenlist                1.4.0
fsspec                    2023.6.0
future                    0.18.3
gitdb                     4.0.10
GitPython                 3.1.32
google-auth               2.17.2
google-auth-oauthlib      1.0.0
gpt4all                   1.0.5
gradio                    3.39.0
gradio_client             0.3.0
greenlet                  2.0.2
grpcio                    1.53.0
h11                       0.14.0
hnswlib                   0.7.0
httpcore                  0.17.3
httptools                 0.6.0
httpx                     0.24.1
huggingface-hub           0.16.4
humanfriendly             10.0
idna                      3.4
importlib-metadata        6.8.0
iniconfig                 2.0.0
InstructorEmbedding       1.0.1
intervaltree              3.1.0
iopath                    0.1.10
isoduration               20.11.0
itsdangerous              2.1.2
Jinja2                    3.1.2
jmespath                  1.0.1
joblib                    1.3.1
jsonlines                 3.1.0
jsonpatch                 1.32
jsonpointer               2.1
jsonref                   1.1.0
jsonschema                4.18.6
jsonschema-specifications 2023.7.1
kiwisolver                1.4.4
langchain                 0.0.250
langsmith                 0.0.19
layoutparser              0.3.4
linkify-it-py             2.0.2
lit                       16.0.6
llama-cpp-python          0.1.73
lm-dataformat             0.0.20
loralib                   0.1.1
lxml                      4.9.3
lz4                       4.3.2
Markdown                  3.4.3
markdown-it-py            2.2.0
MarkupSafe                2.1.2
marshmallow               3.20.1
matplotlib                3.7.1
mdit-py-plugins           0.3.3
mdurl                     0.1.2
monotonic                 1.6
mpmath                    1.3.0
msg-parser                1.2.0
msgpack                   1.0.5
multidict                 6.0.4
multiprocess              0.70.14
mypy-extensions           1.0.0
neptune                   1.2.0
networkx                  3.1
nltk                      3.8.1
numexpr                   2.8.4
numpy                     1.24.3
nvidia-cublas-cu11        11.10.3.66
nvidia-cuda-cupti-cu11    11.7.101
nvidia-cuda-nvrtc-cu11    11.7.99
nvidia-cuda-runtime-cu11  11.7.99
nvidia-cudnn-cu11         8.5.0.96
nvidia-cufft-cu11         10.9.0.58
nvidia-curand-cu11        10.2.10.91
nvidia-cusolver-cu11      11.4.0.1
nvidia-cusparse-cu11      11.7.4.91
nvidia-nccl-cu11          2.14.3
nvidia-nvtx-cu11          11.7.91
oauthlib                  3.2.2
olefile                   0.46
omegaconf                 2.3.0
onnxruntime               1.15.1
openai                    0.27.8
openapi-schema-pydantic   1.2.4
opencv-python             4.8.0.74
openpyxl                  3.1.2
orjson                    3.9.2
overrides                 7.3.1
packaging                 23.1
pandas                    2.0.2
pdf2image                 1.16.3
pdfminer.six              20221105
pdfplumber                0.10.2
peft                      0.4.0
Pillow                    9.5.0
pip                       22.3.1
pip-licenses              4.3.0
pluggy                    1.0.0
portalocker               2.7.0
posthog                   3.0.1
prettytable               3.8.0
protobuf                  3.20.0
psutil                    5.9.5
pyarrow                   12.0.1
pyasn1                    0.4.8
pyasn1-modules            0.2.8
pycocotools               2.0.6
pycosat                   0.6.4
pycparser                 2.21
pydantic                  1.10.12
pydub                     0.25.1
PyJWT                     2.8.0
pynvml                    11.5.0
pyOpenSSL                 23.0.0
pypandoc                  1.11
pypandoc-binary           1.11
pyparsing                 3.1.1
pypdf                     3.14.0
pypdfium2                 4.18.0
pyphen                    0.14.0
PySocks                   1.7.1
pytesseract               0.3.10
pytest                    7.2.2
pytest-xdist              3.2.1
python-dateutil           2.8.2
python-docx               0.8.11
python-dotenv             1.0.0
python-magic              0.4.27
python-multipart          0.0.6
python-pptx               0.6.21
pytz                      2023.3
PyYAML                    6.0.1
referencing               0.30.2
regex                     2023.6.3
replicate                 0.10.0
requests                  2.31.0
requests-file             1.5.1
requests-oauthlib         1.3.1
responses                 0.18.0
rfc3339-validator         0.1.4
rfc3987                   1.3.8
rouge                     1.0.1
rouge-score               0.1.2
rpds-py                   0.9.2
rsa                       4.9
ruamel.yaml               0.17.21
ruamel.yaml.clib          0.2.6
s3transfer                0.6.1
sacrebleu                 2.3.1
safetensors               0.3.1
scikit-learn              1.2.2
scipy                     1.11.1
semantic-version          2.10.0
sentence-transformers     2.2.2
sentencepiece             0.1.99
setuptools                65.6.3
simplejson                3.19.1
six                       1.16.0
smmap                     5.0.0
sniffio                   1.3.0
sortedcontainers          2.4.0
soupsieve                 2.4.1
SQLAlchemy                2.0.19
starlette                 0.27.0
swagger-spec-validator    3.0.3
sympy                     1.12
tabulate                  0.9.0
tenacity                  8.2.2
tensorboard               2.13.0
tensorboard-data-server   0.7.0
tensorboard-plugin-wit    1.8.1
tensorboardX              2.6
termcolor                 2.3.0
text-generation           0.6.0
textstat                  0.7.3
threadpoolctl             3.2.0
tiktoken                  0.4.0
timm                      0.9.5
tokenizers                0.13.3
tomli                     2.0.1
toolz                     0.12.0
torch                     2.0.1
torchvision               0.15.2
tqdm                      4.65.0
transformers              4.30.2
triton                    2.0.0
typing_extensions         4.7.1
typing-inspect            0.9.0
tzdata                    2023.3
tzlocal                   5.0.1
uc-micro-py               1.0.2
ujson                     5.8.0
unstructured              0.9.0
unstructured-inference    0.5.7
uri-template              1.3.0
urllib3                   1.26.16
uvicorn                   0.23.2
uvloop                    0.17.0
validators                0.20.0
watchfiles                0.19.0
wcwidth                   0.2.6
weaviate-client           3.22.1
webcolors                 1.13
websocket-client          1.6.1
websockets                11.0.3
Werkzeug                  2.3.6
wheel                     0.37.1
xlrd                      2.0.1
XlsxWriter                3.1.2
xxhash                    3.3.0
yarl                      1.9.2
zipp                      3.16.2
zstandard                 0.21.0

• I have tried to delete ggml-gpt4all-j-v1.3-groovy.bin and re-download it, but the above error still appeared.
• I have checked that the file llm/local/ggml-gpt4all-j-v1.3-groovy.bin exists
• My Python version is Python 3.10, using Miniconda

Now trying to check model compatibility, environment settings, dependency versions

[Victorivus]

Hi @abdshomad, downgrading gpt4all to 1.0.0 makes the file valid.

I fixec the gpt4all and the torch versions in the server/pyproject.toml to gpt4all = "1.0.0" torch = ">=2.0.0, !=2.0.1, !=2.1.0" and rerunning in the server folder:

poetry lock

Then, bash scripts/local/run-dev.shshould work