Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Azure OpenAI content filter response breaks streamText #4220

Open
jb-dev1 opened this issue Dec 29, 2024 · 1 comment
Open

Azure OpenAI content filter response breaks streamText #4220

jb-dev1 opened this issue Dec 29, 2024 · 1 comment
Labels
ai/provider bug Something isn't working

Comments

@jb-dev1
Copy link

jb-dev1 commented Dec 29, 2024

Description

When a user prompt is blocked using the AI SDK's Google Generative AI provider with "HARM_CATEGORY_DANGEROUS_CONTENT" (as an example), the onFinish finishReason = content-filter which allows me to capture the value and move on. However, when a user prompt is filtered using the AI SDK's Azure OpenAI provider where the model endpoint's content filter is triggered, the errors below are encountered.

My expected result is an onFinish finishReason = content-filter.

"@ai-sdk/azure": "^1.0.13",
"@ai-sdk/google": "^1.0.12",
"ai": "4.0.10",
 ⨯ Error: failed to pipe response
    at pipeToNodeResponse
[cause]: [Error [AI_APICallError]: The response was filtered due to the prompt triggering Azure OpenAI's content management policy. Please modify your prompt and retry. To learn more about our content filtering policies please read our documentation: https://go.microsoft.com/fwlink/?linkid=2198766] {
    cause: undefined,
    url: 'https://<hidden>.openai.azure.com/openai/deployments/gpt-4o-mini-safety-test/chat/completions?api-version=2024-10-01-preview',
    requestBodyValues: {
      model: 'gpt-4o-mini',
      temperature: 0.6,
      messages: [Array],
      stream: true,
      stream_options: [Object]
      <some values hidden>
    },
    statusCode: 400,
    responseHeaders: {
      'apim-request-id': 'd21fc18c-8a1f-49a7-927b-3cece6bbf39d',
      'azureml-model-session': 'd090-20241221033500',
      'content-length': '666',
      'content-type': 'application/json',
      date: 'Sun, 29 Dec 2024 15:47:13 GMT',
      'ms-azureml-model-error-reason': 'model_error',
      'ms-azureml-model-error-statuscode': '400',
      'strict-transport-security': 'max-age=31536000; includeSubDomains; preload',
      'x-content-type-options': 'nosniff',
      'x-envoy-upstream-service-time': '169',
      'x-ms-client-request-id': 'd21fc18c-8a1f-49a7-927b-3cece6bbf39d',
      'x-ms-rai-invoked': 'true',
      'x-ms-region': 'East US',
      'x-ratelimit-remaining-requests': '98',
      'x-ratelimit-remaining-tokens': '8674',
      'x-request-id': 'd076474e-fda0-4363-97cc-4d43e1446ff6'
    },
    responseBody: `{"error":{"message":"The response was filtered due to the prompt triggering Azure OpenAI's content management policy. Please modify your prompt and retry. To learn more about our content filtering policies please read our documentation: https://go.microsoft.com/fwlink/?linkid=2198766","type":null,"param":"prompt","code":"content_filter","status":400,"innererror":{"code":"ResponsibleAIPolicyViolation","content_filter_result":{"hate":{"filtered":true,"severity":"low"},"jailbreak":{"filtered":false,"detected":false},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":true,"severity":"medium"}}}}}`,
    isRetryable: false,
    data: { error: [Object] }
  }
}
 POST /api/chat 500 in 1019ms
 ⨯ ../../src/server/pipe-readable.ts (144:11) @ pipeToNodeResponse
 ⨯ Internal error: Error: failed to pipe response

Code example

const result = streamText({
  model: azure("gpt-4o-mini-safety-test"),
  system: systemPrompt,
  messages: coreMessages,
  temperature: 0.7,
  onFinish: async ({ finishReason, usage, warnings, response }) => {
    console.log("onFinish Callback Triggered:");
    console.log("Finish Reason:", finishReason);
    console.log("Token Usage:", usage);
    console.log("Warnings (if any):", warnings);
    console.log("Response:", response);

AI provider

@ai-sdk/azure 1.0.13

Additional context

No response

@jb-dev1 jb-dev1 added the bug Something isn't working label Dec 29, 2024
@jb-dev1
Copy link
Author

jb-dev1 commented Dec 29, 2024

May be related to #3655

Updated to "ai": "4.0.20", with no improvement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ai/provider bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants