Rate limit can cause document processing Errors

[lon-tierney]

We are seeing some users having issues with documents not processing correctly. This appears to be an issue with the Rate Limit of the ADA embedding model in Azure OpenAI.

Symptom: Some documents are processing fine, but larger documents are not.

Resolution: In the Azure Portal, navigate to the Azure OpenAI instance where your deployment is pointed, and find your ADA model deployment. Edit it's rate limit (advanced settings) and increase to 240K.

At this time, we don't know how big of a document that will process, but we know it should handle 1000+ page documents. If you see a rate limit error in your log messages, please try this first as a resolution. If your document is still in a Queued state, it should process after adjusting.

If you still have issues, please post to this discussion.

[ArpitaisAn0maly]

In addition Here are some best practices when working Ada:

• For higher consumption and higher document count it is a good practice to increase TPM to higher limit to meet the demand of document processing. Ada's max tokens allowed are 8K. by increasing TPM to higher limits you can increase no of API calls, for example upto 30 API calls per minute.

• For lower consumption and lower document count it is good practice to set lower TPM. beacuse If the actual overall token count is substantially lower what TPM is going to allocate, that means that extra TPM pressure will be applied until the request finishes. It will also translate into more cost.

Based on your use case choose TPM value.

some other things to try in addition to TPM to increase throughput.

• load balancing in different region/subscriptions. some region offers upto 350K TPM for Ada.

• Latency at night is smaller in most region so choose document processing time accordingly.

[dayland]

Also note that Azure AI Document Intelligence is capped at 1000 pages per document, so if you are processing PDF files, the largest supported size is 1000 pages. If you have a larger PDF, we suggest you pre-process the PDF and break it down into 1000 or less individual PDF files.