From 05d9e57fb3945b10c861046c1e6749e2bfc258e3 Mon Sep 17 00:00:00 2001 From: Ronghang Hu Date: Mon, 30 Sep 2024 09:55:58 -0700 Subject: [PATCH] [docs] add a release note and new installation instructions for SAM 2.1 (#338) --- INSTALL.md | 22 ++++++++++++++++++++++ README.md | 13 +++++++++++++ 2 files changed, 35 insertions(+) diff --git a/INSTALL.md b/INSTALL.md index fe2a9cd14..00f622e80 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -66,6 +66,28 @@ to manually add `sam2_configs` into your Python's `sys.path`. +
+ +I got `RuntimeError: Error(s) in loading state_dict for SAM2Base` when loading the new SAM 2.1 checkpoints + +
+ +This is likely because you have installed a previous version of this repo, which doesn't have the new modules to support the SAM 2.1 checkpoints yet. Please try the following steps: + +1. pull the latest code from the `main` branch of this repo +2. run `pip uninstall -y SAM-2` to uninstall any previous installations +3. then install the latest repo again using `pip install -e ".[notebooks]"` + +In case the steps above still don't resolve the error, please try running in your Python environment the following +```python +from sam2.modeling import sam2_base + +print(sam2_base.__file__) +``` +and check whether the content in the printed local path of `sam2/modeling/sam2_base.py` matches the latest one in https://github.com/facebookresearch/segment-anything-2/blob/main/sam2/modeling/sam2_base.py (e.g. whether your local file has `no_obj_embed_spatial`) to indentify if you're still using a previous installation. + +
+
My installation failed with `CUDA_HOME environment variable is not set` diff --git a/README.md b/README.md index 4a7ca7079..4ed294c20 100644 --- a/README.md +++ b/README.md @@ -12,6 +12,15 @@ ![SA-V dataset](assets/sa_v_dataset.jpg?raw=true) +## Latest updates + +**09/30/2024 -- SAM 2.1 Developer Suite (new checkpoints, training code, web demo) is released** + +- A new suite of improved model checkpoints (denoted as **SAM 2.1**) are released. See [Model Description](#model-description) for details. + * To use the new SAM 2.1 checkpoints, you need the latest model code from this repo. If you have installed an earlier version of this repo, please first uninstall the previous version via `pip uninstall SAM-2`, pull the latest code from this repo (with `git pull`), and then reinstall the repo following [Installation](#installation) below. +- The training (and fine-tuning) code has been released. See [`training/README.md`](training/README.md) on how to get started. +- The frontend + backend code for the SAM 2 web demo has been released. See [`demo/README.md`](demo/README.md) for details. + ## Installation SAM 2 needs to be installed first before use. The code requires `python>=3.10`, as well as `torch>=2.3.1` and `torchvision>=0.18.1`. Please follow the instructions [here](https://pytorch.org/get-started/locally/) to install both PyTorch and TorchVision dependencies. You can install SAM 2 on a GPU machine using: @@ -175,6 +184,10 @@ See [sav_dataset/README.md](sav_dataset/README.md) for details. You can train or fine-tune SAM 2 on custom datasets of images, videos, or both. Please check the training [README](training/README.md) on how to get started. +## Web demo for SAM 2 + +We have released the frontend + backend code for the SAM 2 web demo (a locally deployable version similar to https://sam2.metademolab.com/demo). Please see the web demo [README](demo/README.md) for details. + ## License The SAM 2 model checkpoints, SAM 2 demo code (front-end and back-end), and SAM 2 training code are licensed under [Apache 2.0](./LICENSE), however the [Inter Font](https://github.com/rsms/inter?tab=OFL-1.1-1-ov-file) and [Noto Color Emoji](https://github.com/googlefonts/noto-emoji) used in the SAM 2 demo code are made available under the [SIL Open Font License, version 1.1](https://openfontlicense.org/open-font-license-official-text/).