[Performance] benchmark result on ZenDNN EP

[lauthu]

Describe the issue

When trying to benchmark bert model with latest ZenDNN release, I'm facing the similar degradation issue with this issue: #5. And I tried the fix in it but the issue still exist.

Do we have any benchmark result for Bert model on CPU, how much throughput improvement should I expect with ZenDNN.

To reproduce

No now.

Urgency

No response

Platform

Linux

OS Version

Ubuntu 22.04

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

v1.17

ONNX Runtime API

Python

Architecture

X64

Execution Provider

Default CPU, Other / Unknown

Execution Provider Library Version

No response

Model File

No response

Is this a quantized model?

No

[ajeet1203singh]

Hello @lauthu may I know the following:

1. Which bert variant are you running?
2. What is the batch size and sequence length you are using?
3. Have you tried the recommended setting in issue [Performance] Performance degradation with ZenDNN #5
   Based on the information you provide we can help you out.

[lauthu]

Hi @ajeet1203singh, yes I tried the recommended settings of that issue, and I also replied in that issue. I'm trying the same command in that issue (the bert varint, the batch size, and the sequence length).

python -m onnxruntime.transformers.benchmark -m bert-large-uncased --model_class AutoModel -p fp32 -i 3 -t 10 -b 24 -s 16 -n 48 -v --provider zendnn

[ajeet1203singh]

Hello @lauthu can we know which processor are you using and how many cores it has ?