deploy: be71a38

_modules/components/data_process/data_components.html

@@ -440,7 +440,7 @@ <h1>Source code for components.data_process.data_components</h1><div class="high
     <span class="n">retriever_output</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="n">RetrieverOutput</span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="n">RetrieverOutput</span><span class="p">]],</span>
     <span class="n">deduplicate</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span>
 <span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">str</span><span class="p">:</span>
-<span class="w">    </span><span class="sa">r</span><span class="sd">&quot;&quot;&quot;The retrieved documents from one or mulitple queries.</span>
+<span class="w">    </span><span class="sa">r</span><span class="sd">&quot;&quot;&quot;The retrieved documents from one or multiple queries.</span>
 <span class="sd">    Deduplicate is especially helpful when you used query expansion.</span>
 <span class="sd">    &quot;&quot;&quot;</span>
 <span class="w">    </span><span class="sd">&quot;&quot;&quot;</span>

_sources/contributor/contribute_to_code.rst.txt

@@ -0,0 +1,96 @@
+Contribute to Code
+======================================
+This document will cover how you can contribute to lightRAG codebase.
+
+Set Up
+^^^^^^^^^^^^^^^^^^^
+The current ``LightRAG`` code contribution supports `poetry <https://python-poetry.org/>`_ setup only. The team is working on optimizing the library and will get back to support more environment soon.
+If you are only interested in using ``LightRAG`` as a package, please check our `installation guide <https://lightrag.sylph.ai/get_started/installation.html#install-lightrag>`_.
+
+To set up ``poetry`` and contribute, please check the following steps:
+
+1. **Clone the Repository:**
+
+   .. code-block:: bash
+
+        git clone https://github.com/SylphAI-Inc/LightRAG
+        cd LightRAG
+
+2. **Configure API Keys:**
+
+   Copy the example environment file and add your API keys:
+
+   .. code-block:: bash
+
+        cp .env.example .env
+        # example API keys:
+        # OPENAI_API_KEY=YOUR_API_KEY_IF_YOU_USE_OPENAI
+        # GROQ_API_KEY=YOUR_API_KEY_IF_YOU_USE_GROQ
+        # ANTHROPIC_API_KEY=YOUR_API_KEY_IF_YOU_USE_ANTHROPIC
+        # GOOGLE_API_KEY=YOUR_API_KEY_IF_YOU_USE_GOOGLE
+        # COHERE_API_KEY=YOUR_API_KEY_IF_YOU_USE_COHERE
+        # HF_TOKEN=YOUR_API_KEY_IF_YOU_USE_HF
+
+3. **Install Dependencies:**
+
+    The ``./lightrag/pyproject.toml`` controls the dependencies for the ``LightRAG`` package.
+    Use Poetry to install the dependencies and set up the virtual environment:
+
+   .. code-block:: bash
+        cd lightrag
+        poetry install
+        poetry shell
+
+Codebase Structure
+^^^^^^^^^^^^^^^^^^^
+It is recommended to check our `LightRAG codebase structure <https://lightrag.sylph.ai/developer_notes/index.html>`_ and current `API references <https://lightrag.sylph.ai/apis/index.html>`_ to familiarize yourself with the directories and paths before contributing.
+
+Code Examples
+^^^^^^^^^^^^^^^^^^^
+We want to support you with our best. We have included code samples in the `tutorial <https://lightrag.sylph.ai/developer_notes/index.html>`_ for you to refer to.
+
+We inlcude a list of potential samples(`We are working in progress to add more`):
+
+- `ModelClient integration <https://lightrag.sylph.ai/developer_notes/model_client.html#model-inference-sdks>`_. This document will help if you want to add new models not included in our codebase.
+- `Retriever Integration <https://lightrag.sylph.ai/developer_notes/retriever.html#retriever-in-action>`_. We provide different retrivers but you can create more.
+
+Code Tips
+^^^^^^^^^^^^^^^^^^^
+* When writing code, it is appreciated to include any important docstrings and comments. Please refer to `documentation contribution guidelines <./contribute_to_document.html>`_ for standard docstrings.
+* LightRAG is a Python library and if you could follow the `Google Python Style Guide <https://google.github.io/styleguide/pyguide.html>`_, the codebase will be more consistent.
+
+Dependencies
+^^^^^^^^^^^^^^^^^^^
+If you want to add any new dependencies to the package, please include them in your PR description to inform us.
+Since we have already set up the testing automatic workflow in GitHub, please also set your new dependencies in 
+``./lightrag/pyproject.toml`` file ``[tool.poetry.group.test.dependencies]`` section to avoid dependency errors in our CI/CD workflow.
+
+In order to correctly add the dependency using ``poetry``, please run 
+
+.. code-block:: bash
+
+        poetry add --group test <package-name>
+
+Testing
+^^^^^^^^^^^^^^^^^^^
+After you update the code, please make sure your code is well tested before making a pull request. 
+There is a ``./lightrag/tests`` folder in the project directory to host your unit testing cases.
+
+You might need to install the testing packages using ``poetry``:
+
+For example:
+
+.. code-block:: bash
+
+        poetry add --group test unittest
+        poetry add --group test pytest
+        poetry add --group test mypy
+
+
+All the test scripts should start with ``test_``. For example, run the individual test for ``components`` with:
+
+.. code-block:: bash
+
+    python lightrag/tests/test_components.py
+
+

_sources/contributor/contribute_to_document.rst.txt

@@ -1,14 +1,11 @@
-Contribute to Document
+Contribute to Documentation
 ===============================================
 
 .. contents::
    :local:
    :depth: 2
 
-.. _Writing documentation:
-
-Writing Documentation
----------------------------
+.. _Documentation Contribution:
 
 - **User-Facing Documentation**: Found on the main docs site. These include tutorials, guides, and usage documentation meant for end users.
 - **Developer Documentation**: Located within the repository's READMEs and the ``docs/`` directory. These documents are more technical and intended for contributors and maintainers.
@@ -35,14 +32,14 @@ Setup & Build Documentation
 
 **2. Install Necessary Packages**
 
-LightRAG's documentation style is `pydata_sphinx_theme <https://pydata-sphinx-theme.readthedocs.io/en/stable/>`_ (version: 0.15.3).
+LightRAG's documentation style is `pydata_sphinx_theme <https://pydata-sphinx-theme.readthedocs.io/en/stable/>`_.
 
-Install by ``pip``:
+.. Install by ``pip``:
 
-.. code-block:: bash
+.. .. code-block:: bash
 
-    cd docs
-    pip install -r requirements.txt 
+..     cd docs
+..     pip install -r requirements.txt 
 
 Install by ``poetry`` along with all other dependencies for LightRAG:
 
@@ -107,6 +104,9 @@ To manually run these tests, run:
     make doctest
 
 
+Documentation Dependencies
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+If your documentation requires any new dependencies, please include it in the ``pyproject.toml`` under the root directory, include it in your PR description and let us know.
 
 Commit Changes
 ~~~~~~~~~~~~~~~~~~~~~~~~~

_sources/contributor/contribution.rst.txt

@@ -0,0 +1,63 @@
+LightRAG Contribution Guide
+=======================================
+``LightRAG``'s contribution process is similar to most open source projects on GitHub. We encourage new project ideas and the communication between ``LightRAG`` team, developers and the broader community.
+Please don't forget to join us on `Discord <https://discord.com/invite/ezzszrRZvT>`_.
+
+Contribution Process
+----------------------------
+You are always welcomed to contribute even if you've never participated in open source project before. 
+Here is the basic contribution process:
+
+Environment
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+When contributing, please note:
+LightRAG separates the source code environment and documentation environment. 
+
+* To activate the code environment, you should run ``poetry install`` and ``poetry shell`` under ``./lightrag``. The ``./lightrag/pyproject.toml`` contains the dependencies for the ``LightRAG`` package.
+
+* To activate the documentation environment, you can run ``poetry install`` and ``poetry shell`` under ``.``. The ``./pyproject.toml`` controls documentation dependencies.
+
+Find a direction to work on
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+The team builds ``LightRAG`` based on latest researches and product cases. But you might have your own task to apply ``LightRAG``.
+Therefore, you can extend ``LightRAG`` and add any new features you believe will solve yours or others' problems.
+If you don't have any idea yet, you can:
+
+* Check the `existing issues <https://github.com/SylphAI-Inc/LightRAG/issues>`_ and see if there is anyone you know how to fix or you'd love to fix.
+
+* Join us on `Discord <https://discord.com/invite/ezzszrRZvT>`_. We are glad to discuss with you and know what you are interested in here.
+
+Figure out the scope of your change
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+**Small:** Most of the pull requests are small. If your change is small, such as fixing a line of bug, please go ahead to push it.
+
+**Big:** But if you are making a new feature, or planning to push a large change, it is recommended to contact us on `Discord <https://discord.com/invite/ezzszrRZvT>`_ first.
+
+**Unknown:** If you have no idea how big it will be, we are here to help you. Please post your idea on `issues <https://github.com/SylphAI-Inc/LightRAG/issues>`_. We will read it carefully and get back to you.
+
+Add your code
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Please check our `code contribution guidelines <./contribute_to_code.html>`_ to work with code.
+
+Pull requests
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+**WIP PR:** If you are working on an in pull request that is not ready for review, you can create a PR with **"[WIP]"** to inform us that this PR is a draft **“work in progress”**.
+
+**Finished PR:** You can name your finished PR as **"[New Retriever Integration]"** for example.
+We will carry out code review regularly and provide feedbacks as soon as possible. 
+Please iterate your PR with the feedbacks. We will try our best to reduce the revision workload on your side.
+Once your PR is approved, we will merge the PR for you.
+If you have any concerns about our feedbacks, please feel free to contact us on `Discord <https://discord.com/invite/ezzszrRZvT>`_.
+
+Writing Documentation
+----------------------------
+It is a good practice to submit your code with documentations to help the ``LightRAG`` team and other developers better understand your updates.
+Please see our `documentation contribution guidelines <./contribute_to_document.html>`_ for more details on ``LightRAG`` documentation standard.
+
+
+
+
+.. admonition:: Resources
+   :class: highlight
+
+

_sources/contributor/index.rst.txt

@@ -5,5 +5,7 @@ For Contributors
 .. toctree::
    :maxdepth: 2
 
-   contribution_guide
-   contribute_to_document
+   contribution
+   contribute_to_code
+   contribute_to_document
+   .. version_control

_sources/contributor/version_control.rst.txt

@@ -0,0 +1,86 @@
+.. _release-guide:
+
+Release Version Control Guide
+=======================================
+
+Overview
+--------
+**The version will mainly be managed by the LightRAG team. But we are glad to share how we will release the latest version here.**
+
+This guide outlines the process for releasing a new version of ``LightRAG``. 
+The workflow pipeline validates the version tag, builds the package, runs tests, publishes to PyPI, and creates a release on GitHub. The workflow is triggered by tags pushed to the **Release** branch. See `GitHub tags <https://docs.github.com/en/desktop/managing-commits/managing-tags-in-github-desktop>`_ for more details on version release tagging.
+
+Steps to Release a New Version
+------------------------------
+1. Update the **./lightrag/pyproject.toml** version number and the latest dependencies before pushing a new release. Make sure to follow the `PEP 440 rules <https://peps.python.org/pep-0440/>`_ to define the version, otherwise, the workflow will fail. For example:
+
+   .. code-block:: python
+
+      [tool.poetry]
+      name = "lightrag"
+      
+      version = "0.0.0-rc.1"
+      description = "The 'PyTorch' library for LLM applications. RAG=Retriever-Agent-Generator."
+
+2. Ensure your updates are the latest and correct. Update the version number following `Semantic Versioning <https://semver.org/>`_. Here is a list of sample tags:
+
+   .. code-block:: none
+
+      Stable Release Tags:
+      v1.0.0
+      v1.2.3
+
+      Pre-release Tags:
+      v1.0.0-alpha.1
+      v1.0.0-beta.1
+      v1.0.0-rc.1
+
+      Custom Pre-release Tags:
+      v1.0.0-test.1
+      v1.1.0-dev.2
+      v1.2.3-pre.3
+      v2.0.0-exp.4
+      v2.1.0-nightly.5
+
+3. The workflow will be triggered when new releases are pushed to the **release** branch. Push the **./lightrag/pyproject.toml** to the release branch:
+
+   .. code-block:: python
+
+      git add lightrag/pyproject.toml
+      git commit -m "new version release"
+      git push origin release
+   
+   Since the workflow only processes **tags**, your file submission will not go through the version release workflow.
+
+   Only the tags you pushed will get checked.
+
+   To push the new version tag, please run:
+
+   .. code-block:: python
+
+      git tag -a vx.y.z -m "Release version x.y.z"
+      git push origin release
+
+4. To delete redundant local tags:
+
+   .. code-block:: python
+
+      git tags # list the existing tags
+      
+      git tag -d <tag>
+      git push origin --delete <tag>
+
+Important Notes
+---------------
+- **Do Not Reuse Tags:** If you need to fix a problem and update the package, you must create a new version number. Duplicated tag number or version number won't work. Never reuse version numbers as this can lead to confusion and potential deployment issues.
+- **Monitor the Workflow:** After pushing the tag, monitor the GitHub Actions workflow to ensure that it completes successfully. Check the ``"Actions"`` tab in the GitHub repository to see the progress and logs.
+
+Common Problems
+-----------------
+- If the workflow fails, review the logs for errors. Common issues might include:
+
+      **Tag Validation:** If your tag validation failed, you should check if your pushed tag meets the standard.
+
+      **Package Build:** Pay attention to the error during the package building and see if there is any bug.
+
+- If you encounter errors related to tagging (e.g., "tag already exists"), check that you're incrementing the version numbers correctly.

_sources/developer_notes/agent.rst.txt

@@ -353,8 +353,8 @@ Now, let's run the test function to see the agent in action.
 
 .. code-block:: python
 
-   test_react_agent(ModelClientType.GROQ(), llama3_model_kwargs)
-   test_react_agent(ModelClientType.OPENAI(), gpt_model_kwargs)
+   test_react_agent(ModelClientType.GROQ, llama3_model_kwargs)
+   test_react_agent(ModelClientType.OPENAI, gpt_model_kwargs)
 
 Our agent will show the core steps for developers via colored printout, including input_query, steps, and the final answer.
 The printout of the first query with llama3 is shown below (without the color here):