Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Partial Evaluation Output Recording support #1387

Merged
merged 12 commits into from
Apr 17, 2024

Conversation

swernli
Copy link
Collaborator

@swernli swernli commented Apr 15, 2024

This adds logic to insert output recording calls for recognized types into RIR programs generated by partial evaluation.

This adds logic to insert output recording calls for recognized types into RIR programs generated by partial evaluation.
Copy link

Benchmark for 4d59a6b

Click to view benchmark
Test Base PR %
Array append evaluation 342.6±6.79µs 337.1±6.31µs -1.61%
Array literal evaluation 190.6±2.72µs 180.8±2.79µs -5.14%
Array update evaluation 421.6±1.72µs 421.1±6.98µs -0.12%
Core + Standard library compilation 17.0±0.52ms 17.4±0.49ms +2.35%
Deutsch-Jozsa evaluation 5.1±0.04ms 5.1±0.04ms 0.00%
Large file parity evaluation 33.8±1.08ms 33.6±0.27ms -0.59%
Large input file compilation 11.5±0.12ms 11.7±0.54ms +1.74%
Large input file compilation (interpreter) 44.6±0.87ms 45.6±1.63ms +2.24%
Large nested iteration 33.7±0.29ms 33.3±2.00ms -1.19%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1488.6±37.23µs 1485.5±43.50µs -0.21%
Perform Runtime Capabilities Analysis (RCA) on large file sample 7.8±0.37ms 7.7±0.08ms -1.28%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1400.5±31.69µs 1404.0±135.53µs +0.25%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 21.3±0.19ms 21.3±0.37ms 0.00%
Teleport evaluation 81.4±3.97µs 81.8±3.80µs +0.49%

Copy link

Benchmark for 99b9752

Click to view benchmark
Test Base PR %
Array append evaluation 331.1±1.57µs 345.6±4.10µs +4.38%
Array literal evaluation 170.3±0.89µs 201.9±16.58µs +18.56%
Array update evaluation 412.1±3.77µs 429.7±7.93µs +4.27%
Core + Standard library compilation 18.2±0.82ms 16.9±0.26ms -7.14%
Deutsch-Jozsa evaluation 5.1±0.04ms 5.2±0.04ms +1.96%
Large file parity evaluation 33.7±0.39ms 33.7±0.32ms 0.00%
Large input file compilation 11.8±0.29ms 11.3±0.40ms -4.24%
Large input file compilation (interpreter) 45.9±1.50ms 47.1±2.62ms +2.61%
Large nested iteration 32.9±1.38ms 33.7±0.54ms +2.43%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1499.9±33.09µs 1495.7±52.95µs -0.28%
Perform Runtime Capabilities Analysis (RCA) on large file sample 7.9±0.12ms 7.8±0.14ms -1.27%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1413.9±77.19µs 1406.7±62.01µs -0.51%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 21.6±0.35ms 21.2±0.41ms -1.85%
Teleport evaluation 80.5±3.64µs 81.9±3.74µs +1.74%

Copy link

Benchmark for 9f14a9e

Click to view benchmark
Test Base PR %
Array append evaluation 371.3±2.99µs 343.4±3.45µs -7.51%
Array literal evaluation 171.0±1.03µs 200.3±7.13µs +17.13%
Array update evaluation 416.3±2.73µs 429.1±2.78µs +3.07%
Core + Standard library compilation 20.0±0.60ms 19.8±0.45ms -1.00%
Deutsch-Jozsa evaluation 5.1±0.04ms 5.3±0.05ms +3.92%
Large file parity evaluation 33.7±0.12ms 33.8±0.59ms +0.30%
Large input file compilation 12.9±0.53ms 11.7±0.30ms -9.30%
Large input file compilation (interpreter) 50.9±1.63ms 51.4±1.24ms +0.98%
Large nested iteration 33.1±0.68ms 33.7±0.28ms +1.81%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1530.6±115.15µs 1534.1±131.34µs +0.23%
Perform Runtime Capabilities Analysis (RCA) on large file sample 8.1±0.13ms 8.1±0.13ms 0.00%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1443.1±125.82µs 1440.7±110.57µs -0.17%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 22.1±0.22ms 21.9±0.20ms -0.90%
Teleport evaluation 82.4±3.97µs 82.4±4.03µs 0.00%

Copy link

Benchmark for 8f40caf

Click to view benchmark
Test Base PR %
Array append evaluation 341.4±2.27µs 338.7±3.19µs -0.79%
Array literal evaluation 190.0±5.79µs 187.8±2.37µs -1.16%
Array update evaluation 422.3±2.62µs 421.7±3.17µs -0.14%
Core + Standard library compilation 18.2±1.05ms 17.7±0.96ms -2.75%
Deutsch-Jozsa evaluation 5.1±0.31ms 5.2±0.04ms +1.96%
Large file parity evaluation 33.8±0.15ms 33.8±0.66ms 0.00%
Large input file compilation 11.7±0.33ms 11.7±0.62ms 0.00%
Large input file compilation (interpreter) 47.5±1.44ms 46.5±1.60ms -2.11%
Large nested iteration 33.6±0.61ms 32.8±0.28ms -2.38%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1509.8±125.94µs 1490.4±41.57µs -1.28%
Perform Runtime Capabilities Analysis (RCA) on large file sample 7.8±0.12ms 7.7±0.11ms -1.28%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1421.5±71.42µs 1410.8±73.15µs -0.75%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 21.5±0.28ms 21.4±0.88ms -0.47%
Teleport evaluation 81.2±3.56µs 81.3±3.94µs +0.12%

@swernli swernli marked this pull request as ready for review April 16, 2024 06:01
@swernli swernli mentioned this pull request Apr 16, 2024
64 tasks
Copy link

Benchmark for aa6f34d

Click to view benchmark
Test Base PR %
Array append evaluation 347.2±2.81µs 337.9±2.54µs -2.68%
Array literal evaluation 195.3±4.87µs 172.3±3.98µs -11.78%
Array update evaluation 434.4±2.47µs 419.8±1.67µs -3.36%
Core + Standard library compilation 17.7±0.86ms 17.4±0.78ms -1.69%
Deutsch-Jozsa evaluation 5.1±0.03ms 5.0±0.05ms -1.96%
Large file parity evaluation 33.7±0.25ms 33.6±0.07ms -0.30%
Large input file compilation 11.6±0.26ms 11.7±0.28ms +0.86%
Large input file compilation (interpreter) 46.1±1.73ms 45.9±1.40ms -0.43%
Large nested iteration 34.2±0.24ms 33.2±0.26ms -2.92%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1516.6±117.54µs 1498.4±41.60µs -1.20%
Perform Runtime Capabilities Analysis (RCA) on large file sample 7.8±0.09ms 7.8±0.10ms 0.00%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1414.5±34.09µs 1409.6±35.77µs -0.35%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 21.2±0.20ms 21.3±0.23ms +0.47%
Teleport evaluation 81.4±3.81µs 81.6±4.21µs +0.25%

Copy link

Benchmark for c0e523c

Click to view benchmark
Test Base PR %
Array append evaluation 348.1±2.63µs 361.8±7.88µs +3.94%
Array literal evaluation 189.9±1.00µs 190.6±1.33µs +0.37%
Array update evaluation 434.7±2.05µs 444.8±3.13µs +2.32%
Core + Standard library compilation 17.1±0.36ms 17.1±0.42ms 0.00%
Deutsch-Jozsa evaluation 5.1±0.04ms 5.2±0.22ms +1.96%
Large file parity evaluation 33.6±0.10ms 33.8±0.07ms +0.60%
Large input file compilation 11.5±0.13ms 11.8±0.14ms +2.61%
Large input file compilation (interpreter) 44.7±0.99ms 45.2±0.96ms +1.12%
Large nested iteration 34.4±0.49ms 35.4±0.20ms +2.91%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1491.6±32.14µs 1501.4±33.49µs +0.66%
Perform Runtime Capabilities Analysis (RCA) on large file sample 7.7±0.07ms 7.7±0.07ms 0.00%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1404.2±27.03µs 1411.6±39.05µs +0.53%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 21.1±0.19ms 21.1±0.19ms 0.00%
Teleport evaluation 81.7±4.05µs 82.3±4.42µs +0.73%

Copy link

Benchmark for 29a4672

Click to view benchmark
Test Base PR %
Array append evaluation 343.6±1.32µs 332.9±11.76µs -3.11%
Array literal evaluation 201.3±0.78µs 179.8±1.08µs -10.68%
Array update evaluation 427.4±1.10µs 417.3±1.22µs -2.36%
Core + Standard library compilation 17.4±0.58ms 17.9±1.07ms +2.87%
Deutsch-Jozsa evaluation 5.1±0.07ms 5.0±0.04ms -1.96%
Large file parity evaluation 33.9±1.02ms 33.7±0.18ms -0.59%
Large input file compilation 11.6±0.29ms 11.7±0.41ms +0.86%
Large input file compilation (interpreter) 46.2±1.40ms 46.7±1.58ms +1.08%
Large nested iteration 33.6±0.22ms 32.6±0.32ms -2.98%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1496.4±65.15µs 1507.0±56.21µs +0.71%
Perform Runtime Capabilities Analysis (RCA) on large file sample 8.0±0.14ms 7.9±0.12ms -1.25%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1409.7±70.14µs 1420.2±47.27µs +0.74%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 21.6±0.32ms 21.6±0.30ms 0.00%
Teleport evaluation 82.3±4.10µs 79.8±3.55µs -3.04%

@swernli swernli enabled auto-merge April 17, 2024 04:24
Copy link

Benchmark for a142099

Click to view benchmark
Test Base PR %
Array append evaluation 348.3±8.26µs 334.1±2.66µs -4.08%
Array literal evaluation 201.0±4.03µs 174.4±4.80µs -13.23%
Array update evaluation 431.5±9.27µs 416.8±2.78µs -3.41%
Core + Standard library compilation 17.9±1.12ms 19.4±1.32ms +8.38%
Deutsch-Jozsa evaluation 5.1±0.05ms 5.2±0.35ms +1.96%
Large file parity evaluation 33.9±0.17ms 33.8±0.15ms -0.29%
Large input file compilation 12.0±0.57ms 12.4±0.85ms +3.33%
Large input file compilation (interpreter) 50.5±3.36ms 50.6±2.51ms +0.20%
Large nested iteration 34.0±0.66ms 32.9±0.30ms -3.24%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1490.6±30.66µs 1488.8±38.12µs -0.12%
Perform Runtime Capabilities Analysis (RCA) on large file sample 7.8±0.08ms 7.9±0.31ms +1.28%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1408.3±36.44µs 1410.2±53.96µs +0.13%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 21.6±0.50ms 21.5±0.34ms -0.46%
Teleport evaluation 81.3±3.85µs 83.8±3.96µs +3.08%

@swernli swernli added this pull request to the merge queue Apr 17, 2024
Merged via the queue into main with commit ad4578e Apr 17, 2024
17 checks passed
@swernli swernli deleted the swernli/partial-eval-output-recording branch April 17, 2024 05:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants