[Perf] Linux/x64: System.Text.Tests Regressions on 11/27/2024 1:41:48 AM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	abee17762558f6b4435163fe192c92b9caefde40
Compare	fe3d5b5f7039e9f38f022dac77aeb6f01025b19d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Tests.Perf_Encoding

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
GetBytes - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	65.88 ns	76.70 ns	1.16	0.12	False
GetByteCount - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	17.59 ns	29.50 ns	1.68	0.53	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*'

System.Text.Tests.Perf_Encoding.GetBytes(size: 512, encName: "utf-8")

ETL Files

Histogram

JIT Disasms

System.Text.Tests.Perf_Encoding.GetByteCount(size: 512, encName: "utf-8")

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	abee17762558f6b4435163fe192c92b9caefde40
Compare	fe3d5b5f7039e9f38f022dac77aeb6f01025b19d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Tests.BinaryWriterExtendedTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
WriteAsciiString - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	149.09 μs	177.99 μs	1.19	0.50	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.BinaryWriterExtendedTests*'

System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 2000000)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	abee17762558f6b4435163fe192c92b9caefde40
Compare	fe3d5b5f7039e9f38f022dac77aeb6f01025b19d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Perf_Utf8Encoding

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
GetByteCount - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	5.00 μs	7.17 μs	1.44	0.46	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Utf8Encoding*'

System.Text.Perf_Utf8Encoding.GetByteCount(Input: EnglishAllAscii)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[LoopedBard3]

Due to #104488, FYI @tannergooding. Change seems purposeful, maybe this is acceptable?

Commit range: d58ffd1...cf7a744

Related Regressions:
Windows x64:

• [Perf] Windows/x64: 2 Regressions on 11/27/2024 1:41:48 AM perf-autofiling-issues#45863

[jeffhandley]

Assigned to @PranavSenthilnathan for triage.

[EgorBo]

@EgorBot -intel -commit cf7a744 vs previous --envvars DOTNET_JitDisasm:GetIndexOfFirstNonAsciiChar DOTNET_EnableAVX512F:0 --filter System.Text.Perf_Utf8Encoding*

[EgorBo]

@EgorBot -intel -commit cf7a744 vs previous --envvars DOTNET_JitDisasm:GetIndexOfFirstNonAsciiChar --filter System.Text.Perf_Utf8Encoding.GetByteCount

[EgorBo]

Judging by the ADX, the regression reproduces on multiple Intel CPUs, but not AMDs. Let me kick in the benchmark again on some intels

[PranavSenthilnathan]

@EgorBot -intel -aws_icelake -aws_sapphirelake -gcp_emeraldrapids -commit cf7a744 vs previous --envvars DOTNET_JitDisasm:Vector DOTNET_EnableAVX512F:0 --filter System.Text.Tests.Perf_Encoding.GetByte512utf*

[EgorBo]

@PranavSenthilnathan are you actually sure the regression comes from that commit specifically?

[PranavSenthilnathan]

Looks like it was, only seems to be an issue when AVX-512 is not enabled/available.

[PranavSenthilnathan]

@EgorBot -intel -profiler -commit cf7a744 vs previous --envvars DOTNET_JitDisasm:Vector DOTNET_EnableAVX512F:0 --filter System.Text.Tests.Perf_Encoding.GetByteCount512utf*

[PranavSenthilnathan]

@EgorBot -intel -profiler -commit cf7a744 vs previous --envvars DOTNET_JitDisasm:Vector DOTNET_EnableAVX512F:0

using System.Text;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;

// Actual runner is optional, but if it exists, it has to be like this:
BenchmarkSwitcher.FromAssembly(typeof(Bench).Assembly).Run(args);

public class Bench
{
    public int size = 512;

    public string encName = "utf-8";
    private Encoding _enc;
    private string _toEncode;
    private char[] _chars;
    [GlobalSetup]
    public void SetupGetBytes()
    {
        _enc = Encoding.GetEncoding(encName);
        _toEncode = CreateString(size);
        _chars = _toEncode.ToCharArray();
    }

    [Benchmark]
    [MemoryRandomization]
    public int GetByteCount() => _enc.GetByteCount(_chars);

    public static string CreateString(int length)
    {
        char[] str = new char[length];

        for (int i = 0; i < str.Length; i++)
        {
            // Add path separator so folders aren't too long.
            if (i % 20 == 0)
            {
                str[i] = Path.DirectorySeparatorChar;
            }
            else
            {
                str[i] = 'a';
            }
        }

        return new string(str);
    }
}