diff --git a/Docs/_static/aimet-furo.css b/Docs/_static/aimet-furo.css index ff8a085f8b5..624bcdf024d 100644 --- a/Docs/_static/aimet-furo.css +++ b/Docs/_static/aimet-furo.css @@ -1,4 +1,17 @@ - +/* + * AIMET color palette + */ +:root { + --aimet-blue: #3253dc; + --aimet-dark-blue: #0000ff; + --aimet-white: #ffffff; + --aimet-border-grey: #e0e0e0; + --aimet-menu-hover: #e3efff; + --aimet-menu-font-active: #0058ff; + --aimet-code-grey: #fafafa; + --aimet-light-blue: #e7f2fa; + } + aside.toc-drawer ul ul ul ul { display: none; } @@ -35,9 +48,51 @@ max-height: 25em; } +*.highlight pre, pre.literal-block { + background-color: var(--aimet-code-grey); +} + /* * Glossary formatting */ [id^="term-"] { font-weight: bold; } + +/* + * Heading formatting + */ + +h1 { + font-size:2.5em; + margin-top:1.75rem; + margin-bottom:1rem; +} + +h2 { + font-size:2em; + margin-top:1.75rem; + margin-bottom:1.25rem; + font-weight: normal; +} + +h3 { + font-size:1.5em; + margin-top:1.25rem; + margin-bottom:1.25rem; + font-weight: normal; +} + +h4 { + font-size:1.25em; + margin-top:1.25rem; + margin-bottom:1.25rem; + font-weight: normal; +} + +h5 { + font-size:1.125em; + margin-top:1.25rem; + margin-bottom:1.25rem; + font-weight: normal; +} \ No newline at end of file diff --git a/Docs/index.rst b/Docs/index.rst index c686725f5e5..48ced65ebd8 100644 --- a/Docs/index.rst +++ b/Docs/index.rst @@ -19,18 +19,16 @@ AIMET Documentation API Reference Release Notes -The AI Model Efficiency Toolkit (AIMET) is a software toolkit designed for quantizing and compressing trained models. +The AI Model Efficiency Toolkit (AIMET) is a software toolkit for quantizing and compressing trained models. The primary objective of model optimization is to facilitate its deployment on edge devices like mobile phones or laptops. -The primary objective of model optimization is to facilitate its deployment on edge devices like mobile phones or laptops. +AIMET employs post-training and fine-tuning techniques to minimize accuracy loss during quantization and compression. -AIMET employs post-training and fine-tuning techniques to optimize trained models, aiming to minimize accuracy loss during quantization or compression. - -AIMET supports models from PyTorch, TensorFlow/Keras, and ONNX frameworks. +AIMET supports models from the PyTorch, TensorFlow/Keras, and ONNX frameworks. Quick Start =========== -To quickly install and begin using AIMET with PyTorch, see the :doc:`Quick start guide <../install/quick-start>`. +To quickly install and begin using AIMET with PyTorch, see the :doc:`Quick Start Guide <../install/quick-start>`. Installation ============ @@ -40,32 +38,37 @@ For other install options, including for TensorFlow and ONNX platforms or to run User Guide ========== -For a high-level explanation of how to use AIMET to optimize a model, see the :doc:`Quantization user guide <../userguide/index>`. +For a high-level explanation of how to use AIMET to optimize a model, see the :doc:`Quantization User Guide <../userguide/index>`. Quantization Simulation Guide ============================= -For a guide to quantization simulation and its related techniques, see the doc :doc:`Quantization simulation guide <../quantsim/index>`. +Quantization simulation (QuantSim) emulates the behavior of quantized hardware in a model running on floating point hardware. For a guide to quantization simulation and its related techniques, see the :doc:`Quantization Simulation Guide <../quantsim/index>`. Feature Guide ============= -For instructions on applying individual AIMET features, see the :doc:`Feature guide <../featureguide/index>`. +For instructions on applying individual AIMET features, see the :doc:`Feature Guide <../featureguide/index>`. Example Notebooks ================= -To view end-to-end examples of model quantization and compression, and to download the examples in Jupyter notebook format, see :doc:`Example notebooks <../examples/index>`. +To view end-to-end examples of model quantization and compression, and to download the examples in Jupyter notebook format, see the :doc:`Example Notebooks <../examples/index>` page. API Reference ============= -For a detailed look at the AIMET API documentation, see the :doc:`API reference <../apiref/index>`. +For a detailed look at the AIMET API, see the :doc:`API Reference <../apiref/index>`. Release Notes ============= -For information specific to this release, see :doc:`Release notes `. +For information about new features in this release, see the :doc:`Release Notes `. + +Glossary +======== + +See the :doc:`glossary <../glossary>` for explanations of terms and acronyms used on this website. | | diff --git a/Docs/install/index.rst b/Docs/install/index.rst index 287d036ff84..c0b0cb50ad9 100644 --- a/Docs/install/index.rst +++ b/Docs/install/index.rst @@ -4,13 +4,37 @@ Installation ############ -Install the latest version of AIMET pacakge for all framework variants and compute platform from -the **.whl** files hosted at https://github.com/quic/aimet/releases. +This page describes instructions for installing the latest version of AIMET across all frameworks +(PyTorch, TensorFlow, and ONNX) and compute platforms. Please choose **ONE** of the available +installation options based on your needs and preferences. + +- :ref:`PyPI ` +- :ref:`Alternative packages ` +- :ref:`Host install from scratch ` +- :ref:`Docker install ` +- :ref:`Building from source ` + +.. _default-package: + +PyPI +==== + +To install the latest version of AIMET for the PyTorch framework, which supports the new **aimet_torch 2** +interface, use the :ref:`Quick start ` instructions. + +.. _alternative-packages: + +Alternative packages +==================== + +Install the latest version of AIMET for supported framework variants and compute platforms including +TensorFlow, ONNX and PyTorch (legacy `aimet_torch.v1` interface) from the .whl files hosted at +https://github.com/quic/aimet/releases. Prerequisites -============= +------------- -The AIMET package requires the following host platform setup. Following prerequisites apply +The AIMET package requires the following host platform setup. The following prerequisites apply to all frameworks variants. * 64-bit Intel x86-compatible processor @@ -21,15 +45,15 @@ to all frameworks variants. * Nvidia driver version 455 or later (using the latest driver is recommended; both CUDA and cuDNN are supported) .. note:: - Starting with the AIMET 2 release, there is no longer a dependency on ``liblapacke``. Therefore, - you should only install the following debian package if you are still using AIMET 1.x. + Starting with the AIMET 2 release, there is no longer a dependency on ``liblapacke``. + Install the following Debian package if (and only if) you are still using AIMET 1.x. .. code-block:: bash apt-get install liblapacke Choose and install a package -============================ +---------------------------- Use one of the following commands to install AIMET based on your choice of framework and compute platform. @@ -39,19 +63,22 @@ Use one of the following commands to install AIMET based on your choice of frame .. tab-item:: PyTorch :sync: torch + .. important:: + Legacy `aimet_torch.v1` necessitates x86_64 architecture, Python 3.10 and PyTorch version 2.1. + **PyTorch 2.1** With CUDA 12.x: .. parsed-literal:: - python3 -m pip install |download_url|\ |version|/aimet_torch-|version|\+cu121\ |whl_suffix| -f |torch_pkg_url| + python3 -m pip install |download_url|\ |version|/aimet_torch-|version|\+cu121\ |torch_whl_suffix| -f |torch_pkg_url| With CPU only: .. parsed-literal:: - python3 -m pip install |download_url|\ |version|/aimet_torch-|version|\+cpu\ |whl_suffix| -f |torch_pkg_url| + python3 -m pip install |download_url|\ |version|/aimet_torch-|version|\+cpu\ |torch_whl_suffix| -f |torch_pkg_url| .. tab-item:: TensorFlow @@ -88,25 +115,113 @@ Use one of the following commands to install AIMET based on your choice of frame python3 -m pip install |download_url|\ |version|/aimet_onnx-|version|\+cpu\ |whl_suffix| -f |torch_pkg_url| -.. |whl_suffix| replace:: \-cp310-cp310-manylinux_2_34_x86_64.whl -.. |download_url| replace:: \https://github.com/quic/aimet/releases/download/ -.. |torch_pkg_url| replace:: \https://download.pytorch.org/whl/torch_stable.html +Verifying the installation +-------------------------- + +Verify your installation using the following instructions. + +**Step 1:** Handle imports and other setup. + +.. code-block:: python + + import numpy as np + from aimet_common import libpymo + + x = np.random.randn(100) + + quant_scheme = libpymo.QuantizationMode.QUANTIZATION_TF + analyzer = libpymo.EncodingAnalyzerForPython(quant_scheme) + +**Step 2:** Compute scale and offset. + +.. code-block:: python + + bitwidth = 8 + is_symmetric, strict_symmetric, unsigned_symmetric = True, False, True + use_cuda = False + analyzer.updateStats(x, use_cuda) + encoding, _ = analyzer.computeEncoding(bitwidth, is_symmetric, strict_symmetric, unsigned_symmetric) -Advanced installation instructions (optional) -============================================= + print(f'Min: {encoding.min}, Max: {encoding.max}, Scale(delta): {encoding.delta}, Offset: {encoding.offset}') -Following are two ways to setup, including prerequisites and dependencies. +The encodings values should be similar to the one shown below. -* :ref:`On your host machine ` -* :ref:`Using our pre-built or locally built Docker images ` +.. rst-class:: script-output -Installing an older version -=========================== + .. code-block:: none -View the release notes for older versions at https://github.com/quic/aimet/releases. Follow the -documentation corresponding to that release to select and install the appropriate AIMET package. + Min: -3.3734087606114667, Max: 3.3470540046691895, Scale(delta): 0.026354755942277083, Offset: -128.0 + +**Step 3:** Perform quantize-dequantize. + +.. code-block:: python + + quantizer = libpymo.TensorQuantizationSimForPython() + out = quantizer.quantizeDequantize(x, + encoding, + libpymo.RoundingMode.ROUND_NEAREST, + bitwidth, + use_cuda) + print(out) + +The quantized-dequantized output should be similar to the one shown below. + +.. rst-class:: script-output + + .. code-block:: none + + [-1.291383 0.36896658 1.0541903 -1.2123188 -2.2137995 1.2650282 + -0.23719281 0.10541902 0.50074035 -0.05270951 -0.94877124 0. + 0.10541902 0.52709514 -0.7115784 2.2401543 -0.34261182 2.0293162 + 0.34261182 -0.6061594 -0.36896658 -0.6588689 -1.5022211 -0.10541902 + -1.4758663 -0.8433522 0.7115784 -0.23719281 0.44803086 -0.94877124 + 0.18448329 -1.0014807 0.55344987 -0.13177378 0.15812853 -0.7115784 + -0.4216761 1.1068997 -0.07906426 1.6603496 0.55344987 -0.47438562 + -0.6325141 0.4216761 -1.4495116 1.5549306 -0.6325141 -1.2123188 + 0.50074035 1.291383 0.07906426 -1.2123188 -2.0820258 1.0014807 + -0.18448329 -0.4216761 1.0278355 -0.21083805 0.52709514 1.6867044 + -0.68522364 1.0278355 -0.55344987 -0.26354757 0.10541902 -0.02635476 + 0.6588689 -0.34261182 -0.05270951 3.347054 0.07906426 -1.080545 + -0.57980466 1.4231569 -0.6588689 1.291383 -0.13177378 0.31625706 + -0.36896658 0.05270951 -0.81699747 -1.4231569 -1.1068997 -0.68522364 + 0.7115784 -1.2650282 -0.7115784 0.50074035 0.28990233 -0.73793316 + 0.21083805 2.4246376 -0.15812853 0.52709514 -0.02635476 -0.13177378 + -1.8711877 0.4216761 -0.55344987 -0.76428795] + +Old versions +------------ + +You can also view the release notes for older AIMET versions at https://github.com/quic/aimet/releases. +Follow the documentation corresponding to that release to select and install the appropriate AIMET package. + +.. _host-install-from-scratch: + +Host install from scratch +========================= + +The :ref:`Host install from scratch ` page contains the procedure to prepare the environment +and manually install and setup AIMET (including prerequisites and dependencies for all framework and +variants) on a fresh Linux host machine. Use it if you experienced problems installing and/or +using AIMET via any of the previous installation procedures. + +.. _docker-install: + +Docker install +============== + +The :ref:`Docker install ` page describes how to install AIMET in a Docker +container using pre-built or locally built Docker images. + +.. _building-from-source: Building from source ==================== -To build the latest AIMET code from the source, see `build AIMET from source `_. +For most users, installing the pre-built AIMET package via the pip package manager offers the best +experience. However, if you want to use the latest code or contribute to AIMET, you need to build it +from source. To build the latest AIMET code from the source, see `build AIMET from source `_. + +.. |torch_whl_suffix| replace:: \-cp310-none-any.whl +.. |whl_suffix| replace:: \-cp310-cp310-manylinux_2_34_x86_64.whl +.. |download_url| replace:: \https://github.com/quic/aimet/releases/download/ +.. |torch_pkg_url| replace:: \https://download.pytorch.org/whl/torch_stable.html diff --git a/Docs/install/install_docker.rst b/Docs/install/install_docker.rst index a0e00efe555..8cd00ab1a0a 100644 --- a/Docs/install/install_docker.rst +++ b/Docs/install/install_docker.rst @@ -31,10 +31,10 @@ Follow the instructions below to install AIMET within a Docker container. Depend installation choices, you will skip various sections on this page. -1. Choose your AIMET variant ----------------------------- +Choose your AIMET variant +------------------------- -**1.1 Choose a variant.** +**Step 1:** Choose a variant. Choose a variant (a combination of framework and runtime environment) from the following table. Copy the ****. @@ -65,7 +65,7 @@ Copy the ****. - CPU - `onnx-cpu` -**1.2 Set the AIMET_VARIANT environment variable.** +**Step 2:** Set the AIMET_VARIANT environment variable. Set the ``AIMET_VARIANT`` shell variable to your chosen variant string. @@ -74,21 +74,24 @@ Set the ``AIMET_VARIANT`` shell variable to your chosen variant string. export AIMET_VARIANT= -2. Choose to download or build an image ---------------------------------------- +Choose to download or build an image +------------------------------------ -Choose one of the following options. We recommend using a prebuilt Docker image unless your +**Step 3:** Choose one of the following options. We recommend using a prebuilt Docker image unless your installation requires custom dependencies. -- :ref:`Download a prebuilt Docker image ` -- :ref:`Build a Docker image ` +Continue to :ref:`Download a prebuilt Docker image (step 4)` + +or + +Skip to :ref:`Build a Docker image (step 5)`. .. _docker-install-download: -3. Download a prebuilt Docker image ------------------------------------ +Download a prebuilt Docker image +-------------------------------- -**3.1 Set environment variables.** +**Step 4:** Set environment variables. Set the following shell variables to define the Docker image installation. @@ -109,15 +112,15 @@ where: When you start the Docker container, it will be downloaded from the image library located at **docker_image_name**. -**3.2 Skip to** :ref:`Start the Docker container `. +Skip to :ref:`Start the Docker container (step 7)`. .. _docker-install-build: -4. Build a Docker image ------------------------ +Build a Docker image +-------------------- -**4.1 Set environment variables.** +**Step 5:** Set environment variables. Set the following shell variables to define the Docker image installation. @@ -138,7 +141,7 @@ where: is whatever name you want to assign the AIMET Docker container. -**4.2 Build the Docker image from code in the the AIMET repo.** +**Step 6:** Build the Docker image from code in the the AIMET repo. .. code-block:: bash @@ -146,16 +149,16 @@ where: .. _docker-install-start: -5. Start the docker container ------------------------------ +Start the docker container +-------------------------- -**5.1 Check that a Docker container named $docker_container_name is not already running. Remove the container if it is.** +**Step 7:** Check that a Docker container named $docker_container_name is not already running. Remove the container if it is. .. code-block:: bash docker ps -a | grep ${docker_container_name} && docker kill ${docker_container_name} -**5.2 Specify a port to use for port forwarding if you plan to run the Visualization APIs (optional).** +**Step 8:** (optional) Specify a port to use for port forwarding if you plan to run the Visualization APIs. .. code-block:: bash @@ -163,7 +166,7 @@ where: where **** is any unused port on the host. -**5.3 Run the Docker container.** +**Step 9:** Run the Docker container. .. code-block:: bash @@ -202,35 +205,37 @@ As a convenience, the following block contains the *first line* of the Docker ru nvidia-docker run --rm -it -u $(id -u ${USER}):$(id -g ${USER}) \ # CPU only, without port forwarding: - docker run --rm -it -u $(id -u ${USER}):$(id -g ${USER}) + docker run --rm -it -u $(id -u ${USER}):$(id -g ${USER}) \ -6. Install AIMET packages -------------------------- +Install AIMET packages +---------------------- **Choose an option to install the AIMET package on the Docker container.** 1. From PyPI (PyTorch only) 2. Any framework variant (hosted **.whl** files) -**6.1 To install the most recent PyTorch AIMET package with GPU support (the most common option) from PyPI, type the following commands in the Docker container.** +**Step 10:** To install the most recent PyTorch AIMET package with GPU support (the most common option) from PyPI, type the following commands in the Docker container. .. code-block:: bash python3 -m pip install aimet-torch -**6.2 To install the latest version of any AIMET variant from the.whl files.** +Skip to :ref:`Environment setup (step 12) `. -6.2.1 Select the release tag for the version you want to install, for example, "|version|". +**Step 11:** To install the latest version of any AIMET variant from the .whl files, follow the substeps below. + +**Step 11.1:** Select the release tag for the version you want to install, for example, "|version|". Releases are listed at: https://github.com/quic/aimet/releases - Identify the .whl file corresponding to the package variant that you want to install - Continue with the instructions below to install AIMET from the .whl file -6.2.2 Set the package details. +**Step 11.2:** Set the package details. .. parsed-literal:: - + # Set the release tag, for example "|version|" export release_tag="" @@ -238,13 +243,13 @@ Releases are listed at: https://github.com/quic/aimet/releases export download_url="\https://github.com/quic/aimet/releases/download/${release_tag}" # Set the wheel file name with extension, - # for example "aimet_torch-|version|\+cu121-cp310-cp310-manylinux_2_34_x86_64.whl" + # for example "aimet_torch-|version|\+cu121\ |torch_whl_suffix|" export wheel_file_name="" # NOTE: Do the following only for the PyTorch and ONNX variant packages! export find_pkg_url_str="-f https://download.pytorch.org/whl/torch_stable.html" -6.2.3 Install the selected AIMET package. +**Step 11.3:** Install the selected AIMET package. .. note:: @@ -258,12 +263,13 @@ Releases are listed at: https://github.com/quic/aimet/releases .. _docker-install-setup: Environment setup -================= +----------------- -**Run the environment setup script to set common environment variables.** +**Step 12:** Run the environment setup script to set common environment variables. .. code-block:: bash source /usr/local/lib/python3.10/dist-packages/aimet_common/bin/envsetup.sh -.. _docker-install-next: + +.. |torch_whl_suffix| replace:: \-cp310-none-any.whl diff --git a/Docs/install/install_host.rst b/Docs/install/install_host.rst index 9cf664647ef..58469d5f704 100644 --- a/Docs/install/install_host.rst +++ b/Docs/install/install_host.rst @@ -23,14 +23,14 @@ Prerequisites Ensure that you have the following prerequisites installed: -1. Python and pip. -2. The CUDA toolkit, if using GPUs. +- Python and pip. +- The CUDA toolkit, if using GPUs. Instructions follow. -**1. Install Python and pip.** +**Step 1:** Install Python and pip. -1.1 Install the latest build of Python 3.10. +Step 1.1: Install the latest build of Python 3.10. .. code-block:: bash @@ -40,7 +40,7 @@ Instructions follow. apt-get install --assume-yes wget gnupg2 -1.2 If you have multiple Python versions installed, set the default version. +Step 1.2: If you have multiple Python versions installed, set the default version. .. code-block:: bash @@ -48,7 +48,7 @@ Instructions follow. update-alternatives --set python3 /usr/bin/python3.10 -**2. Install the CUDA toolkit (optional).** +**Step 2:** Install the CUDA toolkit (optional). .. note:: @@ -60,7 +60,7 @@ The released AIMET GPU packages have been tested with the following CUDA toolkit - TensorFlow GPU variant: `CUDA Toolkit 11.8.0 `_ - ONNX GPU variant: `CUDA Toolkit 11.8.0 `_ -2.1 Visit the CUDA Toolkit link above for the version corresponding to your AIMET GPU package and download the tested version of the CUDA toolkit for your environment. +Step 2.1: Visit the CUDA Toolkit link above for the version corresponding to your AIMET GPU package and download the tested version of the CUDA toolkit for your environment. All versions of the CUDA toolkit are also listed at https://developer.nvidia.com/cuda-toolkit-archive. @@ -68,9 +68,9 @@ All versions of the CUDA toolkit are also listed at https://developer.nvidia.com In the next step, do not execute the final command, **sudo apt-get install cuda**, in the install instructions. -2.2 Follow the command-line instructions on the developer.nvidia.com download page to install the CUDA toolkit, but do *not* execute the final command, **sudo apt-get install cuda**. +Step 2.2: Follow the command-line instructions on the developer.nvidia.com download page to install the CUDA toolkit, but do *not* execute the final command, **sudo apt-get install cuda**. -2.3 Execute the following to update the CUDA repository key. +Step 2.3: Execute the following to update the CUDA repository key. .. code-block:: bash @@ -85,263 +85,264 @@ Installing AIMET **Choose your AIMET variant.** -Based on your machine learning framework and GPU preference, choose one of the install procedures below. +Based on your machine learning framework, choose one of the install procedures below. -:ref:`1. Installing AIMET for PyTorch ` +.. tab-set:: + :sync-group: platform -:ref:`2. Installing AIMET for TensorFlow ` + .. tab-item:: PyTorch + :sync: torch -:ref:`3. Installing AIMET for ONNX ` + **Installing AIMET for PyTorch** -.. _man-install-torch: + **Step 1:** Select the release tag for the version you want to install. -1. Installing AIMET for PyTorch -------------------------------- + For example, "|version|". Releases are listed at: https://github.com/quic/aimet/releases -**1.1 Select the release tag for the version you want to install.** + - Identify the .whl file corresponding to the package variant that you want to install. + - Continue with the instructions below to install AIMET from the .whl file. -For example, "|version|". Releases are listed at: https://github.com/quic/aimet/releases + **Step 2:** Set the package details. -- Identify the .whl file corresponding to the package variant that you want to install. -- Continue with the instructions below to install AIMET from the .whl file. + .. parsed-literal:: -**1.2 Set the package details.** + # Set the release tag, for example "|version|" + export release_tag="" -.. parsed-literal:: + # Construct the download root URL + export download_url="\https://github.com/quic/aimet/releases/download/${release_tag}" - # Set the release tag, for example "|version|" - export release_tag="" + # Set the wheel file name with extension, + # for example "aimet_torch-|version|\+cu121\ |torch_whl_suffix|" + export wheel_file_name="" - # Construct the download root URL - export download_url="\https://github.com/quic/aimet/releases/download/${release_tag}" + # NOTE: Do the following ONLY for the PyTorch and ONNX variant packages! + export find_pkg_url_str="-f https://download.pytorch.org/whl/torch_stable.html" - # Set the wheel file name with extension, - # for example "aimet_torch-|version|\+cu121-cp310-cp310-manylinux_2_34_x86_64.whl" - export wheel_file_name="" - # NOTE: Do the following ONLY for the PyTorch and ONNX variant packages! - export find_pkg_url_str="-f https://download.pytorch.org/whl/torch_stable.html" + **Step 3:** Install the selected AIMET package. + .. note:: -**1.3 Install the selected AIMET package.** + Python dependencies are automatically installed. -.. note:: + .. code-block:: bash - Python dependencies are automatically installed. + # Install the wheel package + python3 -m pip install ${download_url}/${wheel_file_name} ${find_pkg_url_str} -.. code-block:: bash + **Step 4:** Install the common Debian packages. - # Install the wheel package - python3 -m pip install ${download_url}/${wheel_file_name} ${find_pkg_url_str} + .. code-block:: bash -**1.4 Install the common Debian packages.** + cat /usr/local/lib/python3.10/dist-packages/aimet_common/bin/reqs_deb_common.txt | xargs apt-get --assume-yes install -.. code-block:: bash + **Step 1.5:** Install the Torch Debian packages. - cat /usr/local/lib/python3.10/dist-packages/aimet_common/bin/reqs_deb_common.txt | xargs apt-get --assume-yes install + .. code-block:: bash -**1.5 Install the Torch Debian packages.** + cat /usr/local/lib/python3.10/dist-packages/aimet_onnx/bin/reqs_deb_torch_common.txt | xargs apt-get --assume-yes install -.. code-block:: bash + **Step 6:** Install the Torch GPU Debian packages. - cat /usr/local/lib/python3.10/dist-packages/aimet_onnx/bin/reqs_deb_torch_common.txt | xargs apt-get --assume-yes install + .. code-block:: bash -**1.6 Install the Torch GPU Debian packages.** + cat /usr/local/lib/python3.10/dist-packages/aimet_torch/bin/reqs_deb_torch_gpu.txt | xargs apt-get --assume-yes install -.. code-block:: bash + **Step 7:** Replace Pillow with Pillow-SIMD (optional). - cat /usr/local/lib/python3.10/dist-packages/aimet_torch/bin/reqs_deb_torch_gpu.txt | xargs apt-get --assume-yes install + Pillow-SIMD is an optimized version of the Pillow Python Imaging Library. It can improve image processing performance on x86 architecture machines. -**1.7 Replace Pillow with Pillow-SIMD (optional).** + .. code-block:: bash -Pillow-SIMD is an optimized version of the Pillow Python Imaging Library. It can improve image processing performance on x86 architecture machines. + python3 -m pip uninstall -y pillow + python3 -m pip install --no-cache-dir Pillow-SIMD==9.0.0.post1 -.. code-block:: bash + **Step 8:** Link to executable paths. - python3 -m pip uninstall -y pillow - python3 -m pip install --no-cache-dir Pillow-SIMD==9.0.0.post1 + .. code-block:: bash -**1.8 Link to executable paths.** + ln -s /usr/lib/x86_64-linux-gnu/libjpeg.so /usr/lib + ln -s /usr/local/cuda- /usr/local/cuda -.. code-block:: bash + where **** is the version of CUDA that you installed in the :ref:`Prerequisite section <_installation-prereq>`, for example **12.1.0**. - ln -s /usr/lib/x86_64-linux-gnu/libjpeg.so /usr/lib - ln -s /usr/local/cuda- /usr/local/cuda + **Step 9:** Run the environment setup script to set common environment variables. -where **** is the version of CUDA that you installed in the :ref:`Prerequisite section <_installation-prereq>`, for example **12.1.0**. + .. code-block:: bash -**1.9 Run the environment setup script to set common environment variables.** + source /usr/local/lib/python3.10/dist-packages/aimet_common/bin/envsetup.sh -.. code-block:: bash + **Installation is complete.** Proceed to :ref:`Next steps `. - source /usr/local/lib/python3.10/dist-packages/aimet_common/bin/envsetup.sh -**Installation is complete.** Proceed to :ref:`Next steps `. + .. tab-item:: TensorFlow + :sync: tf + **Installing AIMET for TensorFlow** -.. _man-install-tf: + **Step 1:** Select the release tag for the version you want to install. -2. Installing AIMET for TensorFlow ----------------------------------- + For example, "|version|". Releases are listed at: https://github.com/quic/aimet/releases -**2.1 Select the release tag for the version you want to install.** + - Identify the .whl file corresponding to the package variant that you want to install + - Continue with the instructions below to install AIMET from the .whl file -For example, "|version|". Releases are listed at: https://github.com/quic/aimet/releases + **Step 2:** Set the package details. -- Identify the .whl file corresponding to the package variant that you want to install -- Continue with the instructions below to install AIMET from the .whl file + .. parsed-literal:: -**2.2 Set the package details.** + # Set the release tag, for example "|version|" + export release_tag="" -.. parsed-literal:: + # Construct the download root URL + export download_url="\https://github.com/quic/aimet/releases/download/${release_tag}" - # Set the release tag, for example "|version|" - export release_tag="" + # Set the wheel file name with extension, + # for example "aimet_tensorflow-|version|\+cu118-cp310-cp310-manylinux_2_34_x86_64.whl" + export wheel_file_name="" - # Construct the download root URL - export download_url="\https://github.com/quic/aimet/releases/download/${release_tag}" + **Step 3:** Install the selected AIMET package. - # Set the wheel file name with extension, - # for example "aimet_tensorflow-|version|\+cu118-cp310-cp310-manylinux_2_34_x86_64.whl" - export wheel_file_name="" + .. note:: -**2.3 Install the selected AIMET package.** + Python dependencies are automatically installed. -.. note:: + .. code-block:: bash - Python dependencies are automatically installed. + # Install the wheel package + python3 -m pip install ${download_url}/${wheel_file_name} -.. code-block:: bash - # Install the wheel package - python3 -m pip install ${download_url}/${wheel_file_name} + **Step 4:** Install the common Debian packages. + .. code-block:: bash -**2.4 Install the common Debian packages.** + cat /usr/local/lib/python3.10/dist-packages/aimet_common/bin/reqs_deb_common.txt | xargs apt-get --assume-yes install -.. code-block:: bash + **Step 5:** Install the tensorflow GPU debian packages. - cat /usr/local/lib/python3.10/dist-packages/aimet_common/bin/reqs_deb_common.txt | xargs apt-get --assume-yes install + .. code-block:: bash -**2.5 Install the tensorflow GPU debian packages.** + cat /usr/local/lib/python3.10/dist-packages/aimet_tensorflow/bin/reqs_deb_tf_gpu.txt | xargs apt-get --assume-yes install -.. code-block:: bash + **Step 6:** Replace Pillow with Pillow-SIMD (optional). - cat /usr/local/lib/python3.10/dist-packages/aimet_tensorflow/bin/reqs_deb_tf_gpu.txt | xargs apt-get --assume-yes install + Pillow-SIMD is an optimized version of the Pillow Python Imaging Library. It can improve image processing performance on x86 architecture machines. -**2.6 Replace Pillow with Pillow-SIMD (optional).** + .. code-block:: bash -Pillow-SIMD is an optimized version of the Pillow Python Imaging Library. It can improve image processing performance on x86 architecture machines. + python3 -m pip uninstall -y pillow + python3 -m pip install --no-cache-dir Pillow-SIMD==9.0.0.post1 -.. code-block:: bash + **Step 7:** Link to executable paths. - python3 -m pip uninstall -y pillow - python3 -m pip install --no-cache-dir Pillow-SIMD==9.0.0.post1 + .. code-block:: bash -**2.7 Link to executable paths.** + ln -s /usr/lib/x86_64-linux-gnu/libjpeg.so /usr/lib + ln -s /usr/local/cuda- /usr/local/cuda -.. code-block:: bash + where **** is the version of CUDA that you installed in the :ref:`Prerequisite section <_installation-prereq>`, for example **11.8.0**. - ln -s /usr/lib/x86_64-linux-gnu/libjpeg.so /usr/lib - ln -s /usr/local/cuda- /usr/local/cuda + **Step 8:** Run the environment setup script to set common environment variables. -where **** is the version of CUDA that you installed in the :ref:`Prerequisite section <_installation-prereq>`, for example **11.8.0**. + .. code-block:: bash -**2.8 Run the environment setup script to set common environment variables.** + source /usr/local/lib/python3.10/dist-packages/aimet_common/bin/envsetup.sh -.. code-block:: bash + **Installation is complete.** Proceed to :ref:`Next steps `from PyPI. - source /usr/local/lib/python3.10/dist-packages/aimet_common/bin/envsetup.sh + .. tab-item:: ONNX + :sync: onnx -**Installation is complete.** Proceed to :ref:`Next steps `from PyPI. + **Installing AIMET for ONNX** -.. _man-install-onnx: + **Step 1:** Select the release tag for the version you want to install. -3. Installing AIMET for ONNX ----------------------------- + For example, "|version|". Releases are listed at: https://github.com/quic/aimet/releases -**3.1 Select the release tag for the version you want to install.** + - Identify the .whl file corresponding to the package variant that you want to install + - Continue with the instructions below to install AIMET from the .whl file -For example, "|version|". Releases are listed at: https://github.com/quic/aimet/releases + **Step 2:** Set the package details. -- Identify the .whl file corresponding to the package variant that you want to install -- Continue with the instructions below to install AIMET from the .whl file + .. parsed-literal:: -**3.2 Set the package details.** + # Set the release tag, for example "|version|" + export release_tag="" -.. parsed-literal:: + # Construct the download root URL + export download_url="\https://github.com/quic/aimet/releases/download/${release_tag}" - # Set the release tag, for example "|version|" - export release_tag="" + # Set the wheel file name with extension, + # for example "aimet_onnx-|version|\+cu118-cp310-cp310-manylinux_2_34_x86_64.whl" + export wheel_file_name="" - # Construct the download root URL - export download_url="\https://github.com/quic/aimet/releases/download/${release_tag}" + # NOTE: Do the following ONLY for the PyTorch and ONNX variant packages! + export find_pkg_url_str="-f https://download.pytorch.org/whl/torch_stable.html" - # Set the wheel file name with extension, - # for example "aimet_onnx-|version|\+cu118-cp310-cp310-manylinux_2_34_x86_64.whl" - export wheel_file_name="" + **Step 3:** Install the selected AIMET package. - # NOTE: Do the following ONLY for the PyTorch and ONNX variant packages! - export find_pkg_url_str="-f https://download.pytorch.org/whl/torch_stable.html" + .. note:: -**3.3 Install the selected AIMET package.** + Python dependencies are automatically installed. -.. note:: + .. code-block:: bash - Python dependencies are automatically installed. + # Install the wheel package + python3 -m pip install ${download_url}/${wheel_file_name} ${find_pkg_url_str}| -.. code-block:: bash + **Step 4:** Install the common Debian packages. - # Install the wheel package - python3 -m pip install ${download_url}/${wheel_file_name} ${find_pkg_url_str}| + .. code-block:: bash -**3.4 Install the common Debian packages.** + cat /usr/local/lib/python3.10/dist-packages/aimet_common/bin/reqs_deb_common.txt | xargs apt-get --assume-yes install -.. code-block:: bash + **Step 5:** Install the ONNX Debian packages. - cat /usr/local/lib/python3.10/dist-packages/aimet_common/bin/reqs_deb_common.txt | xargs apt-get --assume-yes install + .. code-block:: bash -**3.5 Install the ONNX Debian packages.** + cat /usr/local/lib/python3.10/dist-packages/aimet_onnx/bin/reqs_deb_onnx_common.txt | xargs apt-get --assume-yes install -.. code-block:: bash + **Step 6:** Install the ONNX GPU debian packages. - cat /usr/local/lib/python3.10/dist-packages/aimet_onnx/bin/reqs_deb_onnx_common.txt | xargs apt-get --assume-yes install + .. code-block:: bash -**3.6 Install the ONNX GPU debian packages.** + cat /usr/local/lib/python3.10/dist-packages/aimet_onnx/bin/reqs_deb_onnx_gpu.txt | xargs apt-get --assume-yes install -.. code-block:: bash - cat /usr/local/lib/python3.10/dist-packages/aimet_onnx/bin/reqs_deb_onnx_gpu.txt | xargs apt-get --assume-yes install + **Step 7:** Replace Pillow with Pillow-SIMD (optional). + Pillow-SIMD is an optimized version of the Pillow Python Imaging Library. It can improve image processing performance on x86 architecture machines. -**3.7 Replace Pillow with Pillow-SIMD (optional).** + .. code-block:: bash -Pillow-SIMD is an optimized version of the Pillow Python Imaging Library. It can improve image processing performance on x86 architecture machines. + python3 -m pip uninstall -y pillow + python3 -m pip install --no-cache-dir Pillow-SIMD==9.0.0.post1 -.. code-block:: bash - python3 -m pip uninstall -y pillow - python3 -m pip install --no-cache-dir Pillow-SIMD==9.0.0.post1 + **Step 8:** Replace onnxruntime with onnxruntime-gpu. + .. code-block:: bash -**3.8 Replace onnxruntime with onnxruntime-gpu.** + export ONNXRUNTIME_VER=$(python3 -c 'import onnxruntime; print(onnxruntime.__version__)') + python3 -m pip uninstall -y onnxruntime + python3 -m pip install --no-cache-dir onnxruntime-gpu==$ONNXRUNTIME_VER -.. code-block:: bash - export ONNXRUNTIME_VER=$(python3 -c 'import onnxruntime; print(onnxruntime.__version__)') - python3 -m pip uninstall -y onnxruntime - python3 -m pip install --no-cache-dir onnxruntime-gpu==$ONNXRUNTIME_VER + **Step 9:** Link to executable paths. + .. code-block:: bash -**3.9 Link to executable paths.** + ln -s /usr/lib/x86_64-linux-gnu/libjpeg.so /usr/lib -.. code-block:: bash - ln -s /usr/lib/x86_64-linux-gnu/libjpeg.so /usr/lib + **Step 10:** Run the environment setup script to set common environment variables. + .. code-block:: bash + + source /usr/local/lib/python3.10/dist-packages/aimet_common/bin/envsetup.sh -**3.10 Run the environment setup script to set common environment variables.** -.. code-block:: bash - source /usr/local/lib/python3.10/dist-packages/aimet_common/bin/envsetup.sh +.. |torch_whl_suffix| replace:: \-cp310-none-any.whl \ No newline at end of file diff --git a/Docs/install/quick-start.rst b/Docs/install/quick-start.rst index 2f233da32ed..80f3a98d3ab 100644 --- a/Docs/install/quick-start.rst +++ b/Docs/install/quick-start.rst @@ -1,36 +1,47 @@ .. _install-quick-start: -########### -Quick Start -########### +##################### +Quick Start (PyTorch) +##################### -This page describes how to quickly install the latest version of AIMET for PyTorch framework. +This page describes how to quickly install the latest version of AIMET for the PyTorch framework. -For all the framework variants and compute platform, see :ref:`Installation `. +For all the framework variants and compute platforms, see :ref:`Installation `. -Installation -============ +.. _install-quick-start-platform: -Type the following command to install AIMET for PyTorch framework using pip package manager. +Tested platform +=============== -.. code-block:: bash - - python3 -m pip install aimet-torch - -AIMET PyTorch package has been tested using the following recommended host platform configuration: +aimet_torch 2 has been tested using the following host platform configuration: * 64-bit Intel x86-compatible processor -* Python 3.8–3.12 +* Python 3.10 * Ubuntu 22.04 * For GPU variants: * Nvidia GPU card (Compute capability 5.2 or later) * Nvidia driver version 455 or later (using the latest driver is recommended; both CUDA and cuDNN are supported) -Verification -============ +Installing AIMET +================ + +.. note:: + aimet_torch 2 should run on any platform that supports PyTorch using Python 3.8 or later. + + See :ref:`tested platform ` for information about tested host platform configuration. + +Type the following command to install AIMET for the PyTorch framework using the pip package manager. + +.. code-block:: bash + + python3 -m pip install aimet-torch + +Verifying the installation +========================== + +To confirm that AIMET PyTorch is installed correctly, do the following. -To confirm that AIMET PyTorch is installed correctly, you can verify the installation by executing the -following sample code snippet. +**Step 1:** Verify torch by executing the following sample code snippet. .. code-block:: python @@ -38,7 +49,7 @@ following sample code snippet. x = torch.randn(100) -Then instantiate 8-bit symmetric affine quantizer. +**Step 2:** Verify AIMET PyTorch by instantiating an 8-bit symmetric quantizer. .. code-block:: python @@ -48,7 +59,7 @@ Then instantiate 8-bit symmetric affine quantizer. out = Q.affine.quantize(x, scale, offset, qmin=-128, qmax=127) print(out) -The quantized output should be something similar to the one shown below. +The quantized output should be similar to the one shown below. .. rst-class:: script-output @@ -65,20 +76,20 @@ The quantized output should be something similar to the one shown below. -116., -128., 42., -64., -58., -88., -72., 127., -35., 68., -63., -28., 14., -1., -128., -27., -91., -77., 56., 127.]) -Quick example -============= +Running a quick example +======================= Next, using :mod:`aimet_torch` for the MobileNetV2 network, create a :class:`QuantizationSimModel`, perform calibration, -and then evaluate it: +and then evaluate it. -**Step 1**: Let's handle necessary imports and other setup. +**Step 1:** Handle imports and other setup. .. literalinclude:: ../snippets/torch/installation_verification.py :language: python :start-after: # [step_1] :end-before: # End of [step_1] -**Step 2**: We will create :class:`QuantizationSimModel` and ensure the model contains quantization operations. +**Step 2:** Create a :class:`QuantizationSimModel` and ensure the model contains quantization operations. .. literalinclude:: ../snippets/torch/installation_verification.py :language: python @@ -90,7 +101,7 @@ The model should be composed of Quantized :class:`nn.Modules`, similar to the ou .. rst-class:: script-output .. code-block:: none - + MobileNetV2( (features): Sequential( (0): Conv2dNormActivation( @@ -110,14 +121,14 @@ The model should be composed of Quantized :class:`nn.Modules`, similar to the ou ... ) -**Step 3**: We perform calibration. As a proof of concept, random input is being passed in. However, calibration should be performed using a representative dataset in real world cases. +**Step 3:** Calibrate the model. This example uses random values as input. In real-world cases, calibration should be performed using a representative dataset. .. literalinclude:: ../snippets/torch/installation_verification.py :language: python :start-after: # [step_3] :end-before: # End of [step_3] -**Step 4**: We perform evaluation. +**Step 4:** Evaluate the model. .. literalinclude:: ../snippets/torch/installation_verification.py :language: python diff --git a/Docs/release_notes.rst b/Docs/release_notes.rst index 4a5bc665557..974e33d73b4 100644 --- a/Docs/release_notes.rst +++ b/Docs/release_notes.rst @@ -1,9 +1,41 @@ +.. include:: abbreviation.txt + .. _rn-index: ############# Release notes ############# +2.0.0 +===== + +* New Features + * Common + * Reorganized the documentation to more clearly explain AIMET procedures + * Redesigned the documentation using the `Furo theme `_ + * Added post-AIMET procedures on how to take AIMET quantized model to |qnn| and |qai_hub| + * PyTorch + * BREAKING CHANGE: :mod:`aimet_torch.v2` has become the default API. All the legacy APIs are migrated to :mod:`aimet_torch.v1` subpackage, for example from :mod:`aimet_torch.qc_quantize_op` to :mod:`aimet_torch.v1.qc_quantize_op` + * Added Manual Mixed Precision Configurator (Beta) to make it easy to configure a model in Mixed Precision. + * ONNX + * Optimized :func:`QuantizationSimModel.__init__` latency + * Align :mod:`ConnectedGraph` representation with onnx graph + +* Bug Fixes + * ONNX + * Bug fixes for Adaround + * Bug fixes for BN fold + +* Upgrading + * PyTorch + * aimet_torch 2 is fully backward compatible with all the public APIs of aimet_torch 1.x. If you are using low-level components of :class:`QuantizationSimModel`, please see :doc:`Migrate to aimet_torch 2 `. + +1.35.1 +====== + +* PyTorch + * Fixed package versioning for compatibility with latest pip version + 1.35.0 ====== diff --git a/Docs/versions.rst b/Docs/versions.rst index 7cc4b70f850..fb6003d90d3 100644 --- a/Docs/versions.rst +++ b/Docs/versions.rst @@ -4,26 +4,27 @@ AIMET documentation versions ############################ -* `2.00 (current) `_ -* `1.35 `_ -* `1.34 `_ -* `1.33 `_ -* `1.32 `_ -* `1.31 `_ -* `1.30 `_ -* `1.29 `_ -* `1.28 `_ -* `1.27 `_ -* `1.26 `_ -* `1.25 `_ -* `1.24 `_ -* `1.23 `_ -* `1.22 `_ -* `1.21 `_ -* `1.20 `_ -* `1.19 `_ -* `1.18 `_ -* `1.17 `_ -* `1.16 `_ -* `1.14 `_ -* `1.13 `_ +* `2.0.0 (current) `_ +* `1.35.1 `_ +* `1.35.0 `_ +* `1.34.0 `_ +* `1.33.0 `_ +* `1.32.0 `_ +* `1.31.0 `_ +* `1.30.0 `_ +* `1.29.0 `_ +* `1.28.0 `_ +* `1.27.0 `_ +* `1.26.0 `_ +* `1.25.0 `_ +* `1.24.0 `_ +* `1.23.0 `_ +* `1.22.0 `_ +* `1.21.0 `_ +* `1.20.0 `_ +* `1.19.1 `_ +* `1.18.0 `_ +* `1.17.0 `_ +* `1.16.0 `_ +* `1.14.0 `_ +* `1.13.0 `_