From e1bcae5e796a863444460fcb0a8f32c83c61c237 Mon Sep 17 00:00:00 2001 From: "[[ -z $EMAIL ]] && read -e -p \"Enter your email (for git configuration): \" EMAIL" Date: Tue, 9 Jul 2024 09:03:12 -0400 Subject: [PATCH 1/3] Document HF_TRANSFER --- docs/hub/models-downloading.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/docs/hub/models-downloading.md b/docs/hub/models-downloading.md index de8914d80..14247f1b7 100644 --- a/docs/hub/models-downloading.md +++ b/docs/hub/models-downloading.md @@ -42,3 +42,16 @@ git clone git@hf.co: # example: git clone git@hf.co:bigscience/bloom If you have write-access to the particular model repo, you'll also have the ability to commit and push revisions to the model. Add your SSH public key to [your user settings](https://huggingface.co/settings/keys) to push changes and/or access private repos. + +## Faster downloads + +You can also use the `hf_transfer` library which might give you the ability to download data faster from the Hub. This is +mainly relevent if you have a machine with high bandwidth. + +```bash +pip install huggingface_hub[hf_transfer] +HF_HUB_ENABLE_HF_TRANSFER=1 huggingface-cli download ... +``` +To learn more, check out the documentation [here](https://huggingface.co/docs/huggingface_hub/guides/download#faster-downloads) + + From 563a584992755a2bf7193fd1160fd27532632092 Mon Sep 17 00:00:00 2001 From: Zach Mueller Date: Tue, 9 Jul 2024 10:04:21 -0400 Subject: [PATCH 2/3] Update docs/hub/models-downloading.md Co-authored-by: Lucain --- docs/hub/models-downloading.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/docs/hub/models-downloading.md b/docs/hub/models-downloading.md index 14247f1b7..fd02c5e32 100644 --- a/docs/hub/models-downloading.md +++ b/docs/hub/models-downloading.md @@ -52,6 +52,14 @@ mainly relevent if you have a machine with high bandwidth. pip install huggingface_hub[hf_transfer] HF_HUB_ENABLE_HF_TRANSFER=1 huggingface-cli download ... ``` -To learn more, check out the documentation [here](https://huggingface.co/docs/huggingface_hub/guides/download#faster-downloads) + + + +`hf_transfer` is a power user tool! +It is tested and production-ready, +but it lacks user-friendly features like advanced error handling or proxies. +For more details, please take a look at this [guide](https://huggingface.co/docs/huggingface_hub/hf_transfer). + + From 76f603d7e83bccb5336a8eb4364324b32d76f84f Mon Sep 17 00:00:00 2001 From: Zach Mueller Date: Tue, 9 Jul 2024 10:04:28 -0400 Subject: [PATCH 3/3] Update docs/hub/models-downloading.md Co-authored-by: Lucain --- docs/hub/models-downloading.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/hub/models-downloading.md b/docs/hub/models-downloading.md index fd02c5e32..0c429520f 100644 --- a/docs/hub/models-downloading.md +++ b/docs/hub/models-downloading.md @@ -45,8 +45,9 @@ Add your SSH public key to [your user settings](https://huggingface.co/settings/ ## Faster downloads -You can also use the `hf_transfer` library which might give you the ability to download data faster from the Hub. This is -mainly relevent if you have a machine with high bandwidth. +If you are running on a machine with high bandwidth, +you can increase your download speed with [`hf_transfer`](https://github.com/huggingface/hf_transfer), +a Rust-based library developed to speed up file transfers with the Hub. ```bash pip install huggingface_hub[hf_transfer]