Run generator for each document in the pipeline

[nqbao]

As title, i want to run a generator for each document in the pipeline (my use case is to extract some data from the retrieved documents)

I can only find examples to run Generator component for all documents but not for each document in pipeline. I can work around using a custom component, but i wonder if there is a better way.

Thanks!

[julian-risch]

Hello nqbao, I would recommend to use a custom component that loops through the documents internally and runs a generator component for each of them.
That way, you have maximum flexibility and can further optimize that component. For example you could build on the example of ConcurrentGenerators in the following example and run generators concurrently within your custom component: https://haystack.deepset.ai/cookbook/concurrent_tasks
Another idea for optimization is to use prompt caching if it's supported by the model you are using, for example https://docs.haystack.deepset.ai/docs/anthropicchatgenerator#prompt-caching

For information extraction , I can also recommend our two examples here as inspiration: https://haystack.deepset.ai/cookbook/information_extraction_raven and https://haystack.deepset.ai/cookbook/information-extraction-gorilla