diff --git a/README.md b/README.md index 4780c735..57030906 100644 --- a/README.md +++ b/README.md @@ -2,10 +2,10 @@ + [![License](https://img.shields.io/github/license/SylphAI-Inc/LightRAG)](https://opensource.org/license/MIT) [![PyPI](https://img.shields.io/pypi/v/lightRAG?style=flat-square)](https://pypi.org/project/lightRAG/) [![PyPI - Downloads](https://img.shields.io/pypi/dm/lightRAG?style=flat-square)](https://pypistats.org/packages/lightRAG) -[![GitHub star chart](https://img.shields.io/github/stars/SylphAI-Inc/LightRAG?style=flat-square)](https://star-history.com/#SylphAI-Inc/LightRAG) [![Open Issues](https://img.shields.io/github/issues-raw/SylphAI-Inc/LightRAG?style=flat-square)](https://github.com/SylphAI-Inc/LightRAG/issues) [![](https://dcbadge.vercel.app/api/server/zt2mTPcu?compact=true&style=flat)](https://discord.gg/zt2mTPcu) @@ -22,9 +22,10 @@ It is *light*, *modular*, and *robust*, with a 100% readable codebase. LLMs are like water; they can be shaped into anything, from GenAI applications such as chatbots, translation, summarization, code generation, and autonomous agents to classical NLP tasks like text classification and named entity recognition. They interact with the world beyond the model’s internal knowledge via retrievers, memory, and tools (function calls). Each use case is unique in its data, business logic, and user experience. -Because of this, no library can provide out-of-the-box solutions. Users must build toward their own use case. This requires the library to be modular, robust, and have a clean, readable codebase. The only code you should put into production is code you either 100% trust or are 100% clear about how to customize and iterate. +Because of this, no library can provide out-of-the-box solutions. Users must build towards their own use case. This requires the library to be modular, robust, and have a clean, readable codebase. The only code you should put into production is code you either 100% trust or are 100% clear about how to customize and iterate. + +This is what LightRAG is: light, modular, and robust, with a 100% readable codebase. -LightRAG is born to be light, modular, and robust, with a 100% readable codebase. Further reading: [Introduction](https://lightrag.sylph.ai/), [Design Philosophy](https://lightrag.sylph.ai/tutorials/lightrag_design_philosophy.html) and [Class hierarchy](https://lightrag.sylph.ai/tutorials/class_hierarchy.html). @@ -57,8 +58,11 @@ class Net(nn.Module): ``` --> # LightRAG Task Pipeline +We will ask the model to respond with ``explanation`` and ``example`` of a concept. To achieve this, we will build a simple pipeline to get the structured output as ``QAOutput``. + +## Well-designed Base Classes -We will ask the model to respond with ``explanation`` and ``example`` of a concept. And we will build a pipeline to get the structured output as ``QAOutput``. +This leverages our two and only powerful base classes: `Component` as building blocks for the pipeline and `DataClass` to ease the data interaction with LLMs. ```python @@ -119,9 +123,9 @@ output = qa("What is LLM?") print(output) ``` -**Structure of the pipeline** +## Clear Pipeline Structure -Here is what we get from ``print(qa)``: +Simply by using `print(qa)`, you can see the pipeline structure, which helps users understand any LLM workflow quickly. ``` QA( @@ -161,16 +165,17 @@ QA( ) ``` -**The output** +**The Output** +We structure the output to both track the data and potential errors if any part of the Generator component fails. Here is what we get from ``print(output)``: ``` GeneratorOutput(data=QAOutput(explanation='LLM stands for Large Language Model, which refers to a type of artificial intelligence designed to process and generate human-like language.', example='For instance, LLMs are used in chatbots and virtual assistants, such as Siri and Alexa, to understand and respond to natural language input.'), error=None, usage=None, raw_response='```\n{\n "explanation": "LLM stands for Large Language Model, which refers to a type of artificial intelligence designed to process and generate human-like language.",\n "example": "For instance, LLMs are used in chatbots and virtual assistants, such as Siri and Alexa, to understand and respond to natural language input."\n}', metadata=None) ``` -**See the prompt** +**Focus on the Prompt** -Use the following code: +Use the following code will let us see the prompt after it is formatted: ```python @@ -203,6 +208,28 @@ User: What is LLM? You: ```` +## Model-agnostic + + +You can switch to any model simply by using a different model_client (provider) and model_kwargs. +Let's use OpenAI's gpt-3.5-turbo model on the same pipeline. + + +You can switch to any model simply by using a different `model_client` (provider) and `model_kwargs`. +Let's use OpenAI's `gpt-3.5-turbo` model. + +```python +from lightrag.components.model_client import OpenAIClient + +self.generator = Generator( + model_client=OpenAIClient(), + model_kwargs={"model": "gpt-3.5-turbo"}, + template=qa_template, + prompt_kwargs={"output_format_str": parser.format_instructions()}, + output_processors=parser, +) +``` + # Quick Install diff --git a/docs/source/apis/components/index.rst b/docs/source/apis/components/index.rst index 4b31f774..089a42f3 100644 --- a/docs/source/apis/components/index.rst +++ b/docs/source/apis/components/index.rst @@ -82,6 +82,7 @@ Reasoning .. toctree:: :maxdepth: 1 + :hidden: components.model_client components.retriever diff --git a/docs/source/index.rst b/docs/source/index.rst index 8e113932..864913a7 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -9,14 +9,14 @@ .. raw:: html
- PyPI Version - GitHub Stars - Discord + Discord License
+.. GitHub Stars + .. PyPI Downloads @@ -244,9 +244,10 @@ We are building a library that unites the two worlds, forming a healthy LLM appl .. resources/index .. hide the for contributors now -.. :glob: -.. :maxdepth: 1 -.. :caption: For Contributors -.. :hidden: -.. contributor/index + .. :glob: + .. :maxdepth: 1 + .. :caption: For Contributors + .. :hidden: + + .. contributor/index diff --git a/docs/source/tutorials/agent.rst b/docs/source/tutorials/agent.rst index 26afd674..28f47ab7 100644 --- a/docs/source/tutorials/agent.rst +++ b/docs/source/tutorials/agent.rst @@ -504,6 +504,8 @@ The above example will be formated as: **Subclass ReActAgent** If you want to customize the agent further, you can subclass the :class:`ReActAgent` and override the methods you want to change. + + .. .. figure:: /_static/images/query_1.png .. :align: center .. :alt: DataClass diff --git a/docs/source/tutorials/component.rst b/docs/source/tutorials/component.rst index 02ade72e..8e6a152c 100644 --- a/docs/source/tutorials/component.rst +++ b/docs/source/tutorials/component.rst @@ -253,7 +253,7 @@ Using a decorator is an even more convenient way to create a component from a fu .. code-block:: python - .. @fun_to_component + @fun_to_component def add_one(x): return x + 1 @@ -275,7 +275,7 @@ Let's put the `FunComponent`` and `DocQA`` together in a sequence: .. code-block:: python - from lightrag.core.component import Sequential + from lightrag.core.container import Sequential @fun_to_component def enhance_query(query:str) -> str: @@ -318,7 +318,7 @@ The structure of the sequence using ``print(seq)``: - :class:`core.component.Component` - :class:`core.component.FunComponent` - - :class:`core.component.Sequential` + - :class:`core.container.Sequential` - :func:`core.component.fun_to_component` diff --git a/docs/source/tutorials/generator.rst b/docs/source/tutorials/generator.rst index 60b6bdc9..369807b2 100644 --- a/docs/source/tutorials/generator.rst +++ b/docs/source/tutorials/generator.rst @@ -12,7 +12,7 @@ Generator `Generator` is a user-facing orchestration component with a simple and unified interface for LLM prediction. -It is a pipeline consisting of three subcomponents. +It is a pipeline consisting of three subcomponents. By switching the prompt template, model client, and output parser, users have full control and flexibility. Design --------------------------------------- @@ -26,11 +26,10 @@ Design - The :class:`Generator` is designed to achieve the following goals: 1. Model Agnostic: The Generator should be able to call any LLM model with the same prompt. -2. Unified Interface: It should manage the pipeline from prompt(input)->model call -> output parsing. +2. Unified interface: It manages the pipeline from prompt (input) -> model call -> output parsing, while still giving users full control over each part. 3. Unified Output: This will make it easy to log and save records of all LLM predictions. 4. Work with Optimizer: It should be able to work with Optimizer to optimize the prompt. @@ -443,6 +442,7 @@ Besides these examples, LLM is like water, even in our library, we have componen - :class:`LLMRetriever` is a retriever that uses Generator to call LLM to retrieve the most relevant documents. - :class:`DefaultLLMJudge` is a judge that uses Generator to call LLM to evaluate the quality of the response. - :class:`LLMOptimizer` is an optimizer that uses Generator to call LLM to optimize the prompt. +- :class:`ReAct Agent Planner` is an LLM planner that uses Generator to plan and to call functions in ReAct Agent. Tracing --------------------------------------- @@ -479,6 +479,7 @@ Coming soon! - :class:`tracing.generator_call_logger.GeneratorCallLogger` - :class:`tracing.generator_state_logger.GeneratorStateLogger` - :class:`components.retriever.llm_retriever.LLMRetriever` + - :class:`components.agent.react.ReActAgent` - :class:`eval.llm_as_judge.DefaultLLMJudge` - :class:`optim.llm_optimizer.LLMOptimizer` - :func:`utils.config.new_component` diff --git a/docs/source/tutorials/index.rst b/docs/source/tutorials/index.rst index 03440921..fa1f251d 100644 --- a/docs/source/tutorials/index.rst +++ b/docs/source/tutorials/index.rst @@ -59,7 +59,7 @@ Additionally, what shines in LightRAG is that all orchestrator components, like You can easily make each component work with different models from different providers by switching out the `ModelClient` and its `model_kwargs`. -We will introduce the libraries starting from the core base classes, then move to the RAG essentials, and finally to the agent essentials. +We will introduce the library starting from the core base classes, then move to the RAG essentials, and finally to the agent essentials. With these building blocks, we will further introduce optimizing, where the optimizer uses building blocks such as Generator for auto-prompting and retriever for dynamic few-shot in-context learning (ICL). Building @@ -126,8 +126,7 @@ Code path: :ref:`lightrag.core`. For abstract classes: * - :doc:`embedder` - The component that orchestrates model client (Embedding models in particular) and output processors. * - :doc:`retriever` - - The base class for all retrievers who in particular retrieve relevant documents from a given database to add **context** to the generator. - + - The base class for all retrievers, which in particular retrieve relevant documents from a given database to add *context* to the generator. Data Pipeline and Storage ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/docs/source/tutorials/retriever.rst b/docs/source/tutorials/retriever.rst index 6fe4e99b..4b8d1bae 100644 --- a/docs/source/tutorials/retriever.rst +++ b/docs/source/tutorials/retriever.rst @@ -83,7 +83,7 @@ LightRAG library does not prioritize the coverage of integration for the followi Instead, our design goals are: -1. Representative and valable coverage: +1. Cover representative and valuable retriever methods: a. High-precision retrieval methods and enabling them to work locally and in-memory so that researchers and developers can build and test more efficiently. b. Showcase how to work with cloud databases for large-scale data, utilizing their built-in search and filter methods. @@ -120,9 +120,14 @@ Working with ``DialogTurn`` can help manage ``conversation_history``, especiall Retriever Data Types -^^^^^^^^^^^^^^^^^^^^^^^^ -In most cases, the query is string. But there are cases we might need both text and images as a query, such as "find me a cloth that looks like this". -We defined the query type as: +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**Query** + +In most cases, the query is string. But there are cases where we might need both text and images as a query, such as "find me a cloth that looks like this". +We defined the query type `RetrieverQueriesType` so that all of our retrievers should handle both single query and multiple queries at once. +For text-based retrievers, we defined `RetrieverStrQueriesType` as a string or a sequence of strings. + .. code-block:: python @@ -131,28 +136,29 @@ We defined the query type as: RetrieverQueriesType = Union[RetrieverQueryType, Sequence[RetrieverQueryType]] RetrieverStrQueriesType = Union[str, Sequence[RetrieverStrQueryType]] -As we see, our retriever should be able to handle both single query and multiple queries at once. +**Documents** -The documents are a sequence of document of any type that will be later specified by the subclass: +The documents are a sequence of documents of any type, which will be later specified by the subclass: .. code-block:: python RetrieverDocumentType = TypeVar("RetrieverDocumentType", contravariant=True) # a single document RetrieverDocumentsType = Sequence[RetrieverDocumentType] # The final documents types retriever can use +**Output** -We further define the same output format so that we can easily switch between different retrievers in our task pipeline. -Here is our output format: +We further definied the unified output data structure :class:`RetrieverOutput` so that we can easily switch between different retrievers in our task pipeline. +A retriever should return a list of `RetrieverOutput` to support multiple queries at once. This is helpful for: +(1) Batch-processing: Especially for semantic search, where multiple queries can be represented as numpy array and computed all at once, providing faster speeds than processing each query one by one. +(2) Query expansion: To increase recall, users often generate multiple queries from the original query. -.. code-block:: python - class RetrieverOutput(DataClass): - __doc__ = r"""Save the output of a single query in retrievers. - It is up to the subclass of Retriever to specify the type of query and document. - """ +.. code-block:: python + @dataclass + class RetrieverOutput(DataClass): doc_indices: List[int] = field(metadata={"desc": "List of document indices"}) doc_scores: Optional[List[float]] = field( default=None, metadata={"desc": "List of document scores"} @@ -167,11 +173,24 @@ Here is our output format: RetrieverOutputType = List[RetrieverOutput] # so to support multiple queries at once -You can find the types in :ref:`types`. The list of queries and `RetrieverOutput` can be helpful for: -(1) Batch-processing: especially for semantic search where multiple queries can be represented as numpy array and be computed all at once with faster speed than doing one by one. -(2) For `query expansion` where to increase the recall, users often generate multiple queries from the original query. +**Document and TextSplitter** + +If your documents (in text format) are too large, it is common practise to first use :class:`TextSplitter` to split the text into smaller chunks. +Please refer to the :doc:`text_splitter` tutorial on how to use it. + + + +Retriever Base Class +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Functionally, the base retriever :class:`Retriever` defines another required method ``build_index_from_documents`` where the subclass will prepare the retriever for the actual retrieval calls. +Optionally, the subclass can implement ``save_to_file`` and ``load_from_file`` to save and load the retriever to/from disk. +As the retriever is a subclass of component, you already inherited powerful serialization and deserialization methods such as ``to_dict``, ``from_dict``, and ``from_config`` to help +with the saving and loading process. As for helper attributes, we have ``indexed`` and ``index_keys`` to differentiate if the retriever is ready for retrieval and the attributes that are key to restore the functionality/states of the retriever. +It is up the subclass to decide how to decide the storage of the index, it can be in-memory, local disk, or cloud storage, or save as json or pickle file or even a db table. +As an example, :class:`BM25Retriever` has the following key attributes to index. .. code-block:: python @@ -196,24 +215,9 @@ You can find the types in :ref:`types`. The list of queries and `Ret raise NotImplementedError(f"Async retrieve is not implemented") -**Document and TextSplitter** - -If your documents(text format) are too large and it is a common practise to first use ``TextSplitter`` to split them into smaller chunks. -Please refer to :doc:`text_splitter` and our provided notebook on how to use it. - +.. code:: python -Retriever Base Class -^^^^^^^^^^^^^^^^^^^^^^^^ - -Functionally, the base retriever :class:`Retriever` defines another required method ``build_index_from_documents`` where the subclass will prepare the retriever for the actual retrieval calls. -Optionally, the subclass can implement ``save_to_file`` and ``load_from_file`` to save and load the retriever to/from disk. -As the retriever is a subclass of component, you already inherited powerful serialization and deserialization methods such as ``to_dict``, ``from_dict``, and ``from_config`` to help -with the saving and loading process. As for helper attributes, we have ``indexed`` and ``index_keys`` to differentiate if the retriever is ready for retrieval and the attributes that are key to restore the functionality/states of the retriever. -It is up the subclass to decide how to decide the storage of the index, it can be in-memory, local disk, or cloud storage, or save as json or pickle file or even a db table. -As an example, :class:`BM25Retriever` has the following key attributes to index. - -.. code:: python self.index_keys = ["nd", "t2d", "idf","doc_len","avgdl","total_documents","top_k","k1","b","epsilon","indexed"] @@ -254,7 +258,7 @@ In this note, we will use the following documents and queries for demonstration: The first query should retrieve the first and the last document, and the second query should retrieve the second and the third document. FAISSRetriever -^^^^^^^^^^^^^^^^^^^^^^^^ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ First, let's do semantic search, here we will use in-memory :class:`FAISSRetriever`. FAISS retriever takes embeddings which can be ``List[float]`` or ``np.ndarray`` and build an index using FAISS library. The query can take both embeddings and str formats. @@ -334,7 +338,7 @@ In default, the score is a simulated probabity in range ``[0, 1]`` using consine You can check the retriever for more type of scores. BM25Retriever -^^^^^^^^^^^^^^^^^^^^^^^^ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ So the semantic search works pretty well. We will see how :class:`BM25Retriever` works in comparison. We reimplemented the code in [9]_ with one improvement: instead of using ``text.split(" ")``, we use tokenizer to split the text. Here is a comparison of how they different: @@ -408,7 +412,8 @@ This time the retrieval gives us the right answer. [RetrieverOutput(doc_indices=[2, 1], doc_scores=[0.5343238380789569, 0.4568096570283078], query='solar panels?', documents=None)] Reranker as Retriever -^^^^^^^^^^^^^^^^^^^^^^^^ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + Semantic search works well, and reranker basd on mostly `cross-encoder` model is supposed to work even better. We have integrated two rerankers: ``BAAI/bge-reranker-base`` [10]_ hosted on ``transformers`` and rerankers provided by ``Cohere`` [11]_. These models follow the ``ModelClient`` protocol and are directly accessible as retriever from :class:`RerankerRetriever`. @@ -518,7 +523,8 @@ Also, if we use both the `title` and `content`, it will also got the right respo LLM as Retriever -^^^^^^^^^^^^^^^^^^^^^^^^ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + There are differen ways to use LLM as a retriever: @@ -598,12 +604,16 @@ The response is: [RetrieverOutput(doc_indices=[1, 2], doc_scores=None, query='How do solar panels impact the environment?', documents=None)] + PostgresRetriever -^^^^^^^^^^^^^^^^^^^^^^^^ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + Coming soon. Use Score Threshold instead of top_k -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + In some cases, when the retriever has a computed score and you might prefer to use the score instead of ``top_k`` to filter out the relevant documents. To do so, you can simplify set the ``top_k`` to the full size of the documents and use a post-processing step or a component(to chain with the retriever) to filter out the documents with the score below the threshold. @@ -613,7 +623,8 @@ Use together with Database When the scale of data is large, we will use a database to store the computed embeddings and indexes from the documents. With LocalDB -^^^^^^^^^^^^^^^^^^^^^^^^ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + We have previously computed embeddings, now let us :class:`LocalDB` to help with the persistence. (Although you can totally persist them yourself such as using pickle). Additionally, ``LocalDB`` help us keep track of our initial documents and its transformed documents. diff --git a/lightrag/CHANGELOG.md b/lightrag/CHANGELOG.md index fd876f53..00076909 100644 --- a/lightrag/CHANGELOG.md +++ b/lightrag/CHANGELOG.md @@ -1,3 +1,6 @@ +### Added +- `Sequential` adds `acall` method. + ## [0.0.0-beta.1] - 2024-07-10 ### Added diff --git a/lightrag/README.md b/lightrag/README.md index 71b3294c..57030906 100644 --- a/lightrag/README.md +++ b/lightrag/README.md @@ -1,12 +1,36 @@ ![LightRAG Logo](https://raw.githubusercontent.com/SylphAI-Inc/LightRAG/main/docs/source/_static/images/LightRAG-logo-doc.jpeg) + + + +[![License](https://img.shields.io/github/license/SylphAI-Inc/LightRAG)](https://opensource.org/license/MIT) +[![PyPI](https://img.shields.io/pypi/v/lightRAG?style=flat-square)](https://pypi.org/project/lightRAG/) +[![PyPI - Downloads](https://img.shields.io/pypi/dm/lightRAG?style=flat-square)](https://pypistats.org/packages/lightRAG) +[![Open Issues](https://img.shields.io/github/issues-raw/SylphAI-Inc/LightRAG?style=flat-square)](https://github.com/SylphAI-Inc/LightRAG/issues) +[![](https://dcbadge.vercel.app/api/server/zt2mTPcu?compact=true&style=flat)](https://discord.gg/zt2mTPcu) + ### ⚡ The Lightning Library for Large Language Model Applications ⚡ -*LightRAG* helps developers with both building and optimizing *Retriever-Agent-Generator (RAG)* pipelines. -It is *light*, *modular*, and *robust*. +*LightRAG* helps developers with both building and optimizing *Retriever-Agent-Generator* pipelines. +It is *light*, *modular*, and *robust*, with a 100% readable codebase. + + + + +# Why LightRAG? + +LLMs are like water; they can be shaped into anything, from GenAI applications such as chatbots, translation, summarization, code generation, and autonomous agents to classical NLP tasks like text classification and named entity recognition. They interact with the world beyond the model’s internal knowledge via retrievers, memory, and tools (function calls). Each use case is unique in its data, business logic, and user experience. + +Because of this, no library can provide out-of-the-box solutions. Users must build towards their own use case. This requires the library to be modular, robust, and have a clean, readable codebase. The only code you should put into production is code you either 100% trust or are 100% clear about how to customize and iterate. + +This is what LightRAG is: light, modular, and robust, with a 100% readable codebase. +Further reading: [Introduction](https://lightrag.sylph.ai/), [Design Philosophy](https://lightrag.sylph.ai/tutorials/lightrag_design_philosophy.html) and [Class hierarchy](https://lightrag.sylph.ai/tutorials/class_hierarchy.html). + + + +# LightRAG Task Pipeline + +We will ask the model to respond with ``explanation`` and ``example`` of a concept. To achieve this, we will build a simple pipeline to get the structured output as ``QAOutput``. -**LightRAG** +## Well-designed Base Classes + +This leverages our two and only powerful base classes: `Component` as building blocks for the pipeline and `DataClass` to ease the data interaction with LLMs. ```python -from lightrag.core import Component, Generator +from dataclasses import dataclass, field + +from lightrag.core import Component, Generator, DataClass from lightrag.components.model_client import GroqAPIClient -from lightrag.utils import setup_env #noqa +from lightrag.components.output_parsers import JsonOutputParser -class SimpleQA(Component): - def __init__(self): - super().__init__() - template = r""" +@dataclass +class QAOutput(DataClass): + explanation: str = field( + metadata={"desc": "A brief explanation of the concept in one sentence."} + ) + example: str = field(metadata={"desc": "An example of the concept in a sentence."}) + + + +qa_template = r""" +You are a helpful assistant. + +{{output_format_str}} + + +User: {{input_str}} +You:""" + +class QA(Component): + def __init__(self): + super().__init__() + + parser = JsonOutputParser(data_class=QAOutput, return_data_class=True) + self.generator = Generator( + model_client=GroqAPIClient(), + model_kwargs={"model": "llama3-8b-8192"}, + template=qa_template, + prompt_kwargs={"output_format_str": parser.format_instructions()}, + output_processors=parser, + ) + + def call(self, query: str): + return self.generator.call({"input_str": query}) + + async def acall(self, query: str): + return await self.generator.acall({"input_str": query}) +``` + + +Run the following code for visualization and calling the model. + +```python + +qa = QA() +print(qa) + +# call +output = qa("What is LLM?") +print(output) +``` + +## Clear Pipeline Structure + +Simply by using `print(qa)`, you can see the pipeline structure, which helps users understand any LLM workflow quickly. + +``` +QA( + (generator): Generator( + model_kwargs={'model': 'llama3-8b-8192'}, + (prompt): Prompt( + template: You are a helpful assistant. + + {{output_format_str}} + User: {{input_str}} - You: - """ - self.generator = Generator( - model_client=GroqAPIClient(), - model_kwargs={"model": "llama3-8b-8192"}, - template=template, + You:, prompt_kwargs: {'output_format_str': 'Your output should be formatted as a standard JSON instance with the following schema:\n```\n{\n "explanation": "A brief explanation of the concept in one sentence. (str) (required)",\n "example": "An example of the concept in a sentence. (str) (required)"\n}\n```\n-Make sure to always enclose the JSON output in triple backticks (```). Please do not add anything other than valid JSON output!\n-Use double quotes for the keys and string values.\n-Follow the JSON formatting conventions.'}, prompt_variables: ['output_format_str', 'input_str'] + ) + (model_client): GroqAPIClient() + (output_processors): JsonOutputParser( + data_class=QAOutput, examples=None, exclude_fields=None, return_data_class=True + (json_output_format_prompt): Prompt( + template: Your output should be formatted as a standard JSON instance with the following schema: + ``` + {{schema}} + ``` + {% if example %} + Examples: + ``` + {{example}} + ``` + {% endif %} + -Make sure to always enclose the JSON output in triple backticks (```). Please do not add anything other than valid JSON output! + -Use double quotes for the keys and string values. + -Follow the JSON formatting conventions., prompt_variables: ['schema', 'example'] ) + (output_processors): JsonParser() + ) + ) +) +``` + +**The Output** + +We structure the output to both track the data and potential errors if any part of the Generator component fails. +Here is what we get from ``print(output)``: + +``` +GeneratorOutput(data=QAOutput(explanation='LLM stands for Large Language Model, which refers to a type of artificial intelligence designed to process and generate human-like language.', example='For instance, LLMs are used in chatbots and virtual assistants, such as Siri and Alexa, to understand and respond to natural language input.'), error=None, usage=None, raw_response='```\n{\n "explanation": "LLM stands for Large Language Model, which refers to a type of artificial intelligence designed to process and generate human-like language.",\n "example": "For instance, LLMs are used in chatbots and virtual assistants, such as Siri and Alexa, to understand and respond to natural language input."\n}', metadata=None) +``` +**Focus on the Prompt** + +Use the following code will let us see the prompt after it is formatted: + +```python + +qa2.generator.print_prompt( + output_format_str=qa2.generator.output_processors.format_instructions(), + input_str="What is LLM?", +) +``` + + +The output will be: + +````markdown + +You are a helpful assistant. + +Your output should be formatted as a standard JSON instance with the following schema: +``` +{ + "explanation": "A brief explanation of the concept in one sentence. (str) (required)", + "example": "An example of the concept in a sentence. (str) (required)" +} +``` +-Make sure to always enclose the JSON output in triple backticks (```). Please do not add anything other than valid JSON output! +-Use double quotes for the keys and string values. +-Follow the JSON formatting conventions. + + +User: What is LLM? +You: +```` + +## Model-agnostic - def call(self, query): - return self.generator({"input_str": query}) - async def acall(self, query): - return await self.generator.acall({"input_str": query}) +You can switch to any model simply by using a different model_client (provider) and model_kwargs. +Let's use OpenAI's gpt-3.5-turbo model on the same pipeline. + + +You can switch to any model simply by using a different `model_client` (provider) and `model_kwargs`. +Let's use OpenAI's `gpt-3.5-turbo` model. + +```python +from lightrag.components.model_client import OpenAIClient + +self.generator = Generator( + model_client=OpenAIClient(), + model_kwargs={"model": "gpt-3.5-turbo"}, + template=qa_template, + prompt_kwargs={"output_format_str": parser.format_instructions()}, + output_processors=parser, +) ``` -## Quick Install + +# Quick Install Install LightRAG with pip: @@ -75,20 +243,22 @@ Please refer to the [full installation guide](https://lightrag.sylph.ai/get_star + # Documentation LightRAG full documentation available at [lightrag.sylph.ai](https://lightrag.sylph.ai/): - [Introduction](https://lightrag.sylph.ai/) - [Full installation guide](https://lightrag.sylph.ai/get_started/installation.html) -- [Design philosophy](https://lightrag.sylph.ai/tutorials/lightrag_design_philosophy.html): Design based on three principles: Simplicity over complexity, Quality over quantity, and Optimizing over building. -- [Class hierarchy](https://lightrag.sylph.ai/tutorials/class_hierarchy.html): We have no more than two levels of subclasses. The bare minimum abstraction provides developers with maximum customizability and simplicity. -- [Tutorials](https://lightrag.sylph.ai/tutorials/index.html): Learn the `why` and `how-to` (customize and integrate) behind each core part within the `LightRAG` library. +- [Design philosophy](https://lightrag.sylph.ai/tutorials/lightrag_design_philosophy.html) +- [Class hierarchy](https://lightrag.sylph.ai/tutorials/class_hierarchy.html) +- [Tutorials](https://lightrag.sylph.ai/tutorials/index.html) - [API reference](https://lightrag.sylph.ai/apis/index.html) -## Contributors + +# Contributors [![contributors](https://contrib.rocks/image?repo=SylphAI-Inc/LightRAG&max=2000)](https://github.com/SylphAI-Inc/LightRAG/graphs/contributors) diff --git a/lightrag/lightrag/core/container.py b/lightrag/lightrag/core/container.py index 7170e1b6..6f7c14e5 100644 --- a/lightrag/lightrag/core/container.py +++ b/lightrag/lightrag/core/container.py @@ -13,7 +13,7 @@ class Sequential(Component): __doc__ = r"""A sequential container. - Follows the same design pattern as PyTorch's ``nn.Sequential``. + Adapted from PyTorch's ``nn.Sequential``. Components will be added to it in the order they are passed to the constructor. Alternatively, an ``OrderedDict`` of components can be passed in. @@ -97,7 +97,7 @@ def call(self, input: int) -> int: >>> result = seq.call(2, 3) """ - _components: Dict[str, Component] # = OrderedDict() + _components: Dict[str, Component] = OrderedDict() # type: ignore[assignment] @overload def __init__(self, *args: Component) -> None: ... @@ -114,7 +114,7 @@ def __init__(self, *args): for idx, component in enumerate(args): self.add_component(str(idx), component) - def _get_item_by_idx(self, iterator: Iterator[T], idx: int) -> T: + def _get_item_by_idx(self, iterator: Iterator[Component], idx: int) -> Component: """Get the idx-th item of the iterator.""" size = len(self) idx = operator.index(idx) @@ -132,15 +132,18 @@ def __getitem__( elif isinstance(idx, str): return self._components[idx] else: - return self._get_item_by_idx(self._components.values(), idx) + return self._get_item_by_idx(iter(self._components.values()), idx) def __setitem__(self, idx: Union[int, str], component: Component) -> None: """Set the idx-th component of the Sequential.""" if isinstance(idx, str): self._components[idx] = component else: - key: str = self._get_item_by_idx(self._components.keys(), idx) - return setattr(self, key, component) + # key: str = self._get_item_by_idx(iter(self._components.keys()), idx) + # self._components[key] = component + key_list = list(self._components.keys()) + key = key_list[idx] + self._components[key] = component def __delitem__(self, idx: Union[slice, int, str]) -> None: """Delete the idx-th component of the Sequential.""" @@ -150,15 +153,18 @@ def __delitem__(self, idx: Union[slice, int, str]) -> None: elif isinstance(idx, str): del self._components[idx] else: - key = self._get_item_by_idx(self._components.keys(), idx) + # key = self._get_item_by_idx(iter(self._components.keys()), idx) + key_list = list(self._components.keys()) + key = key_list[idx] + delattr(self, key) - # To preserve numbering - str_indices = [str(i) for i in range(len(self._components))] + + # Reordering is needed if numerical keys are used to keep the sequence self._components = OrderedDict( - list(zip(str_indices, self._components.values())) + (str(i), comp) for i, comp in enumerate(self._components.values()) ) - def __iter__(self) -> Iterable[Component]: + def __iter__(self) -> Iterator[Component]: r"""Iterates over the components of the Sequential. Examples: @@ -250,6 +256,33 @@ def call(self, *args: Any, **kwargs: Any) -> object: kwargs = {} return args[0] if len(args) == 1 else (args, kwargs) + @overload + async def acall(self, input: Any) -> object: ... + + @overload + async def acall(self, *args: Any, **kwargs: Any) -> object: ... + + async def acall(self, *args: Any, **kwargs: Any) -> object: + r"""When you for loop or multiple await calls inside each component, use acall method can potentially speed up the execution.""" + if len(args) == 1 and not kwargs: + input = args[0] + for component in self._components.values(): + input = await component(input) + return input + else: + for component in self._components.values(): + result = await component(*args, **kwargs) + if ( + isinstance(result, tuple) + and len(result) == 2 + and isinstance(result[1], dict) + ): + args, kwargs = result + else: + args = (result,) + kwargs = {} + return args[0] if len(args) == 1 else (args, kwargs) + def append(self, component: Component) -> "Sequential": r"""Appends a component to the end of the Sequential.""" idx = len(self._components) @@ -259,7 +292,7 @@ def append(self, component: Component) -> "Sequential": def insert(self, idx: int, component: Component) -> None: r"""Inserts a component at a given index in the Sequential.""" if not isinstance(component, Component): - raise AssertionError( + raise TypeError( f"component should be an instance of Component, but got {type(component)}" ) n = len(self._components) @@ -272,7 +305,6 @@ def insert(self, idx: int, component: Component) -> None: for i in range(n, idx, -1): self._components[str(i)] = self._components[str(i - 1)] self._components[str(idx)] = component - return self def extend(self, components: Iterable[Component]) -> "Sequential": r"""Extends the Sequential with components from an iterable."""