Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] Update the download-artifacts version #2685

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

[CI] Update the download-artifacts version #2685

wants to merge 1 commit into from

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Jan 9, 2025 •
edited
Loading

No description provided.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Jan 9, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2685

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

❌ 7 New Failures, 2 Cancelled Jobs, 1 Unrelated Failure

As of commit 7277ccc with merge base ed656a1 (image):

NEW FAILURES - The following jobs have failed:

CANCELLED JOBS - The following jobs were cancelled. Please retry:

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jan 9, 2025
@vmoens vmoens added the CI Has to do with CI setup (e.g. wheels & builds, tests...) label Jan 9, 2025
@github-actions GitHub Actions
Copy link

github-actions bot commented Jan 9, 2025 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}11$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_simple 0.5348s 0.4426s 2.2595 Ops/s 2.2422 Ops/s $\color{#35bf28}+0.77\%$
test_transformed 0.5987s 0.5964s 1.6767 Ops/s 1.6149 Ops/s $\color{#35bf28}+3.83\%$
test_serial 1.4591s 1.3609s 0.7348 Ops/s 0.7381 Ops/s $\color{#d91a1a}-0.44\%$
test_parallel 1.3020s 1.2047s 0.8301 Ops/s 0.8282 Ops/s $\color{#35bf28}+0.23\%$
test_step_mdp_speed[True-True-True-True-True] 0.2002ms 30.2020μs 33.1104 KOps/s 33.1916 KOps/s $\color{#d91a1a}-0.24\%$
test_step_mdp_speed[True-True-True-True-False] 41.9690μs 17.7011μs 56.4936 KOps/s 55.8222 KOps/s $\color{#35bf28}+1.20\%$
test_step_mdp_speed[True-True-True-False-True] 67.5860μs 17.0675μs 58.5908 KOps/s 58.2891 KOps/s $\color{#35bf28}+0.52\%$
test_step_mdp_speed[True-True-True-False-False] 28.9440μs 10.1180μs 98.8335 KOps/s 99.1268 KOps/s $\color{#d91a1a}-0.30\%$
test_step_mdp_speed[True-True-False-True-True] 87.3630μs 32.0928μs 31.1597 KOps/s 30.8745 KOps/s $\color{#35bf28}+0.92\%$
test_step_mdp_speed[True-True-False-True-False] 72.7560μs 19.5000μs 51.2821 KOps/s 50.4245 KOps/s $\color{#35bf28}+1.70\%$
test_step_mdp_speed[True-True-False-False-True] 57.8780μs 18.9262μs 52.8369 KOps/s 52.2334 KOps/s $\color{#35bf28}+1.16\%$
test_step_mdp_speed[True-True-False-False-False] 52.6590μs 11.8665μs 84.2706 KOps/s 84.7315 KOps/s $\color{#d91a1a}-0.54\%$
test_step_mdp_speed[True-False-True-True-True] 75.2800μs 34.2813μs 29.1704 KOps/s 29.3424 KOps/s $\color{#d91a1a}-0.59\%$
test_step_mdp_speed[True-False-True-True-False] 69.9010μs 21.3094μs 46.9276 KOps/s 46.2944 KOps/s $\color{#35bf28}+1.37\%$
test_step_mdp_speed[True-False-True-False-True] 50.1030μs 18.9691μs 52.7173 KOps/s 52.7066 KOps/s $\color{#35bf28}+0.02\%$
test_step_mdp_speed[True-False-True-False-False] 64.4300μs 11.8102μs 84.6725 KOps/s 85.0956 KOps/s $\color{#d91a1a}-0.50\%$
test_step_mdp_speed[True-False-False-True-True] 74.6800μs 35.3962μs 28.2516 KOps/s 28.0248 KOps/s $\color{#35bf28}+0.81\%$
test_step_mdp_speed[True-False-False-True-False] 60.3610μs 23.2991μs 42.9202 KOps/s 43.1713 KOps/s $\color{#d91a1a}-0.58\%$
test_step_mdp_speed[True-False-False-False-True] 67.3160μs 20.7306μs 48.2378 KOps/s 48.3684 KOps/s $\color{#d91a1a}-0.27\%$
test_step_mdp_speed[True-False-False-False-False] 52.4180μs 13.6404μs 73.3115 KOps/s 73.7818 KOps/s $\color{#d91a1a}-0.64\%$
test_step_mdp_speed[False-True-True-True-True] 89.4270μs 34.0022μs 29.4098 KOps/s 29.4538 KOps/s $\color{#d91a1a}-0.15\%$
test_step_mdp_speed[False-True-True-True-False] 74.3810μs 21.8310μs 45.8065 KOps/s 46.6331 KOps/s $\color{#d91a1a}-1.77\%$
test_step_mdp_speed[False-True-True-False-True] 51.1760μs 21.7557μs 45.9649 KOps/s 45.6741 KOps/s $\color{#35bf28}+0.64\%$
test_step_mdp_speed[False-True-True-False-False] 70.4420μs 13.2677μs 75.3708 KOps/s 75.3970 KOps/s $\color{#d91a1a}-0.03\%$
test_step_mdp_speed[False-True-False-True-True] 73.9980μs 35.2108μs 28.4003 KOps/s 27.8155 KOps/s $\color{#35bf28}+2.10\%$
test_step_mdp_speed[False-True-False-True-False] 58.2980μs 23.1996μs 43.1043 KOps/s 42.8558 KOps/s $\color{#35bf28}+0.58\%$
test_step_mdp_speed[False-True-False-False-True] 2.9632ms 23.4152μs 42.7072 KOps/s 42.3211 KOps/s $\color{#35bf28}+0.91\%$
test_step_mdp_speed[False-True-False-False-False] 45.6760μs 14.8791μs 67.2085 KOps/s 66.5179 KOps/s $\color{#35bf28}+1.04\%$
test_step_mdp_speed[False-False-True-True-True] 87.1030μs 37.4646μs 26.6919 KOps/s 26.6782 KOps/s $\color{#35bf28}+0.05\%$
test_step_mdp_speed[False-False-True-True-False] 54.4520μs 24.9912μs 40.0142 KOps/s 40.1018 KOps/s $\color{#d91a1a}-0.22\%$
test_step_mdp_speed[False-False-True-False-True] 64.2290μs 23.0708μs 43.3448 KOps/s 42.8578 KOps/s $\color{#35bf28}+1.14\%$
test_step_mdp_speed[False-False-True-False-False] 42.3890μs 14.7932μs 67.5986 KOps/s 66.1431 KOps/s $\color{#35bf28}+2.20\%$
test_step_mdp_speed[False-False-False-True-True] 86.7120μs 38.8604μs 25.7332 KOps/s 25.2996 KOps/s $\color{#35bf28}+1.71\%$
test_step_mdp_speed[False-False-False-True-False] 75.5910μs 26.4992μs 37.7370 KOps/s 37.2543 KOps/s $\color{#35bf28}+1.30\%$
test_step_mdp_speed[False-False-False-False-True] 68.7480μs 24.7142μs 40.4626 KOps/s 40.0337 KOps/s $\color{#35bf28}+1.07\%$
test_step_mdp_speed[False-False-False-False-False] 57.2760μs 16.4602μs 60.7527 KOps/s 59.5194 KOps/s $\color{#35bf28}+2.07\%$
test_values[generalized_advantage_estimate-True-True] 9.7915ms 9.5746ms 104.4431 Ops/s 102.1388 Ops/s $\color{#35bf28}+2.26\%$
test_values[vec_generalized_advantage_estimate-True-True] 40.3276ms 33.8880ms 29.5090 Ops/s 29.1904 Ops/s $\color{#35bf28}+1.09\%$
test_values[td0_return_estimate-False-False] 0.2371ms 0.1758ms 5.6888 KOps/s 5.6432 KOps/s $\color{#35bf28}+0.81\%$
test_values[td1_return_estimate-False-False] 27.7650ms 23.5647ms 42.4364 Ops/s 41.2088 Ops/s $\color{#35bf28}+2.98\%$
test_values[vec_td1_return_estimate-False-False] 34.8959ms 33.5745ms 29.7845 Ops/s 29.1820 Ops/s $\color{#35bf28}+2.06\%$
test_values[td_lambda_return_estimate-True-False] 36.4547ms 34.0908ms 29.3334 Ops/s 28.3219 Ops/s $\color{#35bf28}+3.57\%$
test_values[vec_td_lambda_return_estimate-True-False] 40.0992ms 33.8998ms 29.4987 Ops/s 29.0405 Ops/s $\color{#35bf28}+1.58\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 12.6709ms 8.3859ms 119.2473 Ops/s 112.0193 Ops/s $\textbf{\color{#35bf28}+6.45\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.4611ms 1.8390ms 543.7847 Ops/s 559.8558 Ops/s $\color{#d91a1a}-2.87\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4425ms 0.3547ms 2.8192 KOps/s 2.7396 KOps/s $\color{#35bf28}+2.91\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 44.4502ms 41.0203ms 24.3781 Ops/s 22.2592 Ops/s $\textbf{\color{#35bf28}+9.52\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.8430ms 3.0176ms 331.3920 Ops/s 330.5385 Ops/s $\color{#35bf28}+0.26\%$
test_dqn_speed[False-None] 6.0086ms 1.4186ms 704.9256 Ops/s 675.2347 Ops/s $\color{#35bf28}+4.40\%$
test_dqn_speed[False-backward] 1.9738ms 1.8949ms 527.7427 Ops/s 516.6599 Ops/s $\color{#35bf28}+2.15\%$
test_dqn_speed[True-None] 0.6398ms 0.4733ms 2.1127 KOps/s 2.0606 KOps/s $\color{#35bf28}+2.53\%$
test_dqn_speed[True-backward] 1.0659ms 0.9281ms 1.0775 KOps/s 1.0456 KOps/s $\color{#35bf28}+3.05\%$
test_dqn_speed[reduce-overhead-None] 0.7616ms 0.4790ms 2.0877 KOps/s 2.0736 KOps/s $\color{#35bf28}+0.68\%$
test_dqn_speed[reduce-overhead-backward] 0.9559ms 0.9294ms 1.0759 KOps/s 1.0599 KOps/s $\color{#35bf28}+1.51\%$
test_ddpg_speed[False-None] 3.6759ms 2.9009ms 344.7250 Ops/s 343.0950 Ops/s $\color{#35bf28}+0.48\%$
test_ddpg_speed[False-backward] 4.0859ms 3.9835ms 251.0351 Ops/s 246.8918 Ops/s $\color{#35bf28}+1.68\%$
test_ddpg_speed[True-None] 1.4680ms 1.0126ms 987.5831 Ops/s 981.8895 Ops/s $\color{#35bf28}+0.58\%$
test_ddpg_speed[True-backward] 1.9642ms 1.8997ms 526.4023 Ops/s 499.4462 Ops/s $\textbf{\color{#35bf28}+5.40\%}$
test_ddpg_speed[reduce-overhead-None] 1.2984ms 1.0082ms 991.9087 Ops/s 958.1463 Ops/s $\color{#35bf28}+3.52\%$
test_ddpg_speed[reduce-overhead-backward] 2.0081ms 1.9209ms 520.6006 Ops/s 516.9969 Ops/s $\color{#35bf28}+0.70\%$
test_sac_speed[False-None] 10.2156ms 8.0442ms 124.3137 Ops/s 120.5560 Ops/s $\color{#35bf28}+3.12\%$
test_sac_speed[False-backward] 11.9530ms 10.8507ms 92.1599 Ops/s 92.0930 Ops/s $\color{#35bf28}+0.07\%$
test_sac_speed[True-None] 2.3156ms 1.8369ms 544.3829 Ops/s 534.3302 Ops/s $\color{#35bf28}+1.88\%$
test_sac_speed[True-backward] 3.6003ms 3.4972ms 285.9405 Ops/s 285.1755 Ops/s $\color{#35bf28}+0.27\%$
test_sac_speed[reduce-overhead-None] 3.0017ms 1.8608ms 537.4119 Ops/s 534.4287 Ops/s $\color{#35bf28}+0.56\%$
test_sac_speed[reduce-overhead-backward] 4.4452ms 3.5567ms 281.1571 Ops/s 279.1637 Ops/s $\color{#35bf28}+0.71\%$
test_redq_speed[False-None] 14.8449ms 13.0348ms 76.7174 Ops/s 77.3596 Ops/s $\color{#d91a1a}-0.83\%$
test_redq_speed[False-backward] 24.2931ms 22.4854ms 44.4733 Ops/s 44.6285 Ops/s $\color{#d91a1a}-0.35\%$
test_redq_speed[True-None] 5.4952ms 4.5103ms 221.7152 Ops/s 209.1658 Ops/s $\textbf{\color{#35bf28}+6.00\%}$
test_redq_speed[True-backward] 12.4808ms 12.0545ms 82.9568 Ops/s 77.1600 Ops/s $\textbf{\color{#35bf28}+7.51\%}$
test_redq_speed[reduce-overhead-None] 5.2405ms 4.5709ms 218.7729 Ops/s 203.8707 Ops/s $\textbf{\color{#35bf28}+7.31\%}$
test_redq_speed[reduce-overhead-backward] 13.0044ms 12.3871ms 80.7293 Ops/s 77.7828 Ops/s $\color{#35bf28}+3.79\%$
test_redq_deprec_speed[False-None] 15.3942ms 13.0426ms 76.6718 Ops/s 73.3094 Ops/s $\color{#35bf28}+4.59\%$
test_redq_deprec_speed[False-backward] 20.4233ms 19.3205ms 51.7585 Ops/s 50.4721 Ops/s $\color{#35bf28}+2.55\%$
test_redq_deprec_speed[True-None] 4.5483ms 3.7759ms 264.8391 Ops/s 274.5425 Ops/s $\color{#d91a1a}-3.53\%$
test_redq_deprec_speed[True-backward] 8.8655ms 8.1700ms 122.3986 Ops/s 121.6159 Ops/s $\color{#35bf28}+0.64\%$
test_redq_deprec_speed[reduce-overhead-None] 4.2373ms 3.5823ms 279.1469 Ops/s 278.7104 Ops/s $\color{#35bf28}+0.16\%$
test_redq_deprec_speed[reduce-overhead-backward] 8.8304ms 8.0054ms 124.9152 Ops/s 123.7001 Ops/s $\color{#35bf28}+0.98\%$
test_td3_speed[False-None] 8.4017ms 8.0813ms 123.7419 Ops/s 122.2798 Ops/s $\color{#35bf28}+1.20\%$
test_td3_speed[False-backward] 10.7797ms 10.4283ms 95.8930 Ops/s 92.2345 Ops/s $\color{#35bf28}+3.97\%$
test_td3_speed[True-None] 2.0812ms 1.7410ms 574.3921 Ops/s 569.1279 Ops/s $\color{#35bf28}+0.92\%$
test_td3_speed[True-backward] 3.3891ms 3.3148ms 301.6733 Ops/s 301.2518 Ops/s $\color{#35bf28}+0.14\%$
test_td3_speed[reduce-overhead-None] 1.8451ms 1.7301ms 578.0022 Ops/s 572.1403 Ops/s $\color{#35bf28}+1.02\%$
test_td3_speed[reduce-overhead-backward] 4.0199ms 3.6506ms 273.9238 Ops/s 300.6206 Ops/s $\textbf{\color{#d91a1a}-8.88\%}$
test_cql_speed[False-None] 40.4193ms 37.5402ms 26.6381 Ops/s 27.2731 Ops/s $\color{#d91a1a}-2.33\%$
test_cql_speed[False-backward] 49.3190ms 47.0898ms 21.2360 Ops/s 21.2610 Ops/s $\color{#d91a1a}-0.12\%$
test_cql_speed[True-None] 16.7981ms 15.7645ms 63.4338 Ops/s 64.2924 Ops/s $\color{#d91a1a}-1.34\%$
test_cql_speed[True-backward] 30.0750ms 23.1364ms 43.2219 Ops/s 44.8082 Ops/s $\color{#d91a1a}-3.54\%$
test_cql_speed[reduce-overhead-None] 16.9427ms 15.8147ms 63.2323 Ops/s 63.7523 Ops/s $\color{#d91a1a}-0.82\%$
test_cql_speed[reduce-overhead-backward] 24.9621ms 22.9609ms 43.5524 Ops/s 44.5578 Ops/s $\color{#d91a1a}-2.26\%$
test_a2c_speed[False-None] 8.2643ms 7.2402ms 138.1171 Ops/s 138.9469 Ops/s $\color{#d91a1a}-0.60\%$
test_a2c_speed[False-backward] 15.6945ms 14.3586ms 69.6446 Ops/s 70.3220 Ops/s $\color{#d91a1a}-0.96\%$
test_a2c_speed[True-None] 4.9568ms 4.2653ms 234.4500 Ops/s 235.6618 Ops/s $\color{#d91a1a}-0.51\%$
test_a2c_speed[True-backward] 11.5095ms 10.9634ms 91.2123 Ops/s 93.4874 Ops/s $\color{#d91a1a}-2.43\%$
test_a2c_speed[reduce-overhead-None] 5.1633ms 4.2822ms 233.5260 Ops/s 237.4711 Ops/s $\color{#d91a1a}-1.66\%$
test_a2c_speed[reduce-overhead-backward] 12.4279ms 10.7311ms 93.1868 Ops/s 93.5880 Ops/s $\color{#d91a1a}-0.43\%$
test_ppo_speed[False-None] 9.3475ms 7.4577ms 134.0888 Ops/s 133.9752 Ops/s $\color{#35bf28}+0.08\%$
test_ppo_speed[False-backward] 16.9969ms 14.6412ms 68.3004 Ops/s 67.7771 Ops/s $\color{#35bf28}+0.77\%$
test_ppo_speed[True-None] 4.0697ms 3.6733ms 272.2376 Ops/s 268.9985 Ops/s $\color{#35bf28}+1.20\%$
test_ppo_speed[True-backward] 10.0912ms 9.5910ms 104.2644 Ops/s 102.0101 Ops/s $\color{#35bf28}+2.21\%$
test_ppo_speed[reduce-overhead-None] 4.3373ms 3.6936ms 270.7367 Ops/s 265.8147 Ops/s $\color{#35bf28}+1.85\%$
test_ppo_speed[reduce-overhead-backward] 9.8128ms 9.5889ms 104.2873 Ops/s 104.1953 Ops/s $\color{#35bf28}+0.09\%$
test_reinforce_speed[False-None] 7.8610ms 6.5564ms 152.5229 Ops/s 150.7821 Ops/s $\color{#35bf28}+1.15\%$
test_reinforce_speed[False-backward] 10.2190ms 9.8070ms 101.9679 Ops/s 97.3256 Ops/s $\color{#35bf28}+4.77\%$
test_reinforce_speed[True-None] 4.0971ms 2.6906ms 371.6627 Ops/s 370.4161 Ops/s $\color{#35bf28}+0.34\%$
test_reinforce_speed[True-backward] 9.4789ms 8.6092ms 116.1548 Ops/s 116.2847 Ops/s $\color{#d91a1a}-0.11\%$
test_reinforce_speed[reduce-overhead-None] 3.0668ms 2.6524ms 377.0138 Ops/s 372.4595 Ops/s $\color{#35bf28}+1.22\%$
test_reinforce_speed[reduce-overhead-backward] 9.7229ms 8.6434ms 115.6948 Ops/s 111.6715 Ops/s $\color{#35bf28}+3.60\%$
test_iql_speed[False-None] 34.8450ms 32.8093ms 30.4792 Ops/s 30.4199 Ops/s $\color{#35bf28}+0.20\%$
test_iql_speed[False-backward] 47.3900ms 45.9145ms 21.7796 Ops/s 21.7131 Ops/s $\color{#35bf28}+0.31\%$
test_iql_speed[True-None] 12.3061ms 11.3487ms 88.1155 Ops/s 90.6535 Ops/s $\color{#d91a1a}-2.80\%$
test_iql_speed[True-backward] 22.9911ms 22.1008ms 45.2472 Ops/s 45.3544 Ops/s $\color{#d91a1a}-0.24\%$
test_iql_speed[reduce-overhead-None] 13.2714ms 10.9168ms 91.6017 Ops/s 89.6887 Ops/s $\color{#35bf28}+2.13\%$
test_iql_speed[reduce-overhead-backward] 23.4653ms 22.0319ms 45.3887 Ops/s 43.5620 Ops/s $\color{#35bf28}+4.19\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.8496ms 4.9175ms 203.3562 Ops/s 194.6340 Ops/s $\color{#35bf28}+4.48\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8694ms 0.5175ms 1.9322 KOps/s 584.0540 Ops/s $\textbf{\color{#35bf28}+230.83\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.9183ms 0.4874ms 2.0518 KOps/s 1.9841 KOps/s $\color{#35bf28}+3.41\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.9661ms 4.5851ms 218.0995 Ops/s 210.3751 Ops/s $\color{#35bf28}+3.67\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.3994ms 0.5036ms 1.9857 KOps/s 1.9776 KOps/s $\color{#35bf28}+0.41\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.8338ms 0.4772ms 2.0955 KOps/s 2.0464 KOps/s $\color{#35bf28}+2.40\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.1248ms 1.6167ms 618.5393 Ops/s 600.8403 Ops/s $\color{#35bf28}+2.95\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.3333ms 1.5313ms 653.0192 Ops/s 639.6144 Ops/s $\color{#35bf28}+2.10\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.7379ms 4.7877ms 208.8668 Ops/s 208.0396 Ops/s $\color{#35bf28}+0.40\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0794ms 0.6382ms 1.5670 KOps/s 1.5286 KOps/s $\color{#35bf28}+2.51\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9123ms 0.6140ms 1.6286 KOps/s 1.5880 KOps/s $\color{#35bf28}+2.55\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.3345ms 4.6500ms 215.0531 Ops/s 208.8221 Ops/s $\color{#35bf28}+2.98\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.4249ms 0.5149ms 1.9423 KOps/s 1.9273 KOps/s $\color{#35bf28}+0.78\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7896ms 0.4918ms 2.0334 KOps/s 1.9936 KOps/s $\color{#35bf28}+2.00\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.3635ms 4.5426ms 220.1372 Ops/s 213.9739 Ops/s $\color{#35bf28}+2.88\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.8348ms 0.5050ms 1.9801 KOps/s 1.9299 KOps/s $\color{#35bf28}+2.60\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7871ms 0.4850ms 2.0617 KOps/s 2.1047 KOps/s $\color{#d91a1a}-2.05\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.8237ms 4.6812ms 213.6197 Ops/s 204.9598 Ops/s $\color{#35bf28}+4.23\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.3793ms 0.6444ms 1.5517 KOps/s 1.5412 KOps/s $\color{#35bf28}+0.68\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8378ms 0.6122ms 1.6336 KOps/s 1.5749 KOps/s $\color{#35bf28}+3.72\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.4398s 12.8639ms 77.7372 Ops/s 245.5972 Ops/s $\textbf{\color{#d91a1a}-68.35\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 7.6007ms 2.3977ms 417.0746 Ops/s 408.6230 Ops/s $\color{#35bf28}+2.07\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 5.2566ms 1.3809ms 724.1825 Ops/s 785.6457 Ops/s $\textbf{\color{#d91a1a}-7.82\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 5.5696ms 4.1176ms 242.8600 Ops/s 239.9320 Ops/s $\color{#35bf28}+1.22\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 8.9095ms 2.5319ms 394.9577 Ops/s 412.5219 Ops/s $\color{#d91a1a}-4.26\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.7619ms 1.3514ms 739.9566 Ops/s 703.6678 Ops/s $\textbf{\color{#35bf28}+5.16\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.3868s 12.0015ms 83.3230 Ops/s 242.3719 Ops/s $\textbf{\color{#d91a1a}-65.62\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 6.7449ms 2.5023ms 399.6364 Ops/s 398.6025 Ops/s $\color{#35bf28}+0.26\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 6.6375ms 1.5296ms 653.7690 Ops/s 39.0005 Ops/s $\textbf{\color{#35bf28}+1576.31\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 13.5192ms 12.7176ms 78.6309 Ops/s 74.8820 Ops/s $\textbf{\color{#35bf28}+5.01\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 18.4833ms 14.7690ms 67.7092 Ops/s 64.8530 Ops/s $\color{#35bf28}+4.40\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 24.1489ms 21.6885ms 46.1073 Ops/s 45.2747 Ops/s $\color{#35bf28}+1.84\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 16.4009ms 14.8665ms 67.2651 Ops/s 64.3418 Ops/s $\color{#35bf28}+4.54\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 22.9620ms 21.5402ms 46.4248 Ops/s 45.5455 Ops/s $\color{#35bf28}+1.93\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 18.6018ms 16.2328ms 61.6037 Ops/s 58.5469 Ops/s $\textbf{\color{#35bf28}+5.22\%}$

@github-actions GitHub Actions
Copy link

github-actions bot commented Jan 9, 2025 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}15$. Worsened: $\large\color{#d91a1a}11$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_simple 0.7104s 0.7100s 1.4084 Ops/s 1.3656 Ops/s $\color{#35bf28}+3.13\%$
test_transformed 0.9592s 0.9583s 1.0436 Ops/s 1.0192 Ops/s $\color{#35bf28}+2.39\%$
test_serial 2.1977s 2.1143s 0.4730 Ops/s 0.4732 Ops/s $\color{#d91a1a}-0.05\%$
test_parallel 1.8885s 1.8054s 0.5539 Ops/s 0.5481 Ops/s $\color{#35bf28}+1.06\%$
test_step_mdp_speed[True-True-True-True-True] 0.2483ms 39.6519μs 25.2195 KOps/s 24.8153 KOps/s $\color{#35bf28}+1.63\%$
test_step_mdp_speed[True-True-True-True-False] 52.9010μs 22.8713μs 43.7230 KOps/s 43.2518 KOps/s $\color{#35bf28}+1.09\%$
test_step_mdp_speed[True-True-True-False-True] 66.3010μs 21.9173μs 45.6260 KOps/s 45.9854 KOps/s $\color{#d91a1a}-0.78\%$
test_step_mdp_speed[True-True-True-False-False] 50.5010μs 12.7098μs 78.6794 KOps/s 77.4642 KOps/s $\color{#35bf28}+1.57\%$
test_step_mdp_speed[True-True-False-True-True] 80.0820μs 41.9998μs 23.8096 KOps/s 23.5265 KOps/s $\color{#35bf28}+1.20\%$
test_step_mdp_speed[True-True-False-True-False] 59.5510μs 24.9852μs 40.0237 KOps/s 39.5983 KOps/s $\color{#35bf28}+1.07\%$
test_step_mdp_speed[True-True-False-False-True] 93.1220μs 23.5935μs 42.3845 KOps/s 40.8915 KOps/s $\color{#35bf28}+3.65\%$
test_step_mdp_speed[True-True-False-False-False] 46.4310μs 15.1453μs 66.0272 KOps/s 65.4314 KOps/s $\color{#35bf28}+0.91\%$
test_step_mdp_speed[True-False-True-True-True] 93.1220μs 44.8181μs 22.3124 KOps/s 22.0597 KOps/s $\color{#35bf28}+1.15\%$
test_step_mdp_speed[True-False-True-True-False] 72.9810μs 27.6490μs 36.1677 KOps/s 35.5881 KOps/s $\color{#35bf28}+1.63\%$
test_step_mdp_speed[True-False-True-False-True] 56.5810μs 24.2857μs 41.1764 KOps/s 40.9299 KOps/s $\color{#35bf28}+0.60\%$
test_step_mdp_speed[True-False-True-False-False] 41.4000μs 15.1116μs 66.1743 KOps/s 65.1150 KOps/s $\color{#35bf28}+1.63\%$
test_step_mdp_speed[True-False-False-True-True] 88.0420μs 46.4378μs 21.5342 KOps/s 21.1688 KOps/s $\color{#35bf28}+1.73\%$
test_step_mdp_speed[True-False-False-True-False] 57.5310μs 30.0255μs 33.3051 KOps/s 33.4378 KOps/s $\color{#d91a1a}-0.40\%$
test_step_mdp_speed[True-False-False-False-True] 93.5120μs 26.0818μs 38.3410 KOps/s 37.2403 KOps/s $\color{#35bf28}+2.96\%$
test_step_mdp_speed[True-False-False-False-False] 82.6420μs 16.9542μs 58.9825 KOps/s 57.3089 KOps/s $\color{#35bf28}+2.92\%$
test_step_mdp_speed[False-True-True-True-True] 70.8310μs 45.0724μs 22.1865 KOps/s 22.3037 KOps/s $\color{#d91a1a}-0.53\%$
test_step_mdp_speed[False-True-True-True-False] 0.1050ms 27.6563μs 36.1582 KOps/s 35.5941 KOps/s $\color{#35bf28}+1.58\%$
test_step_mdp_speed[False-True-True-False-True] 62.5310μs 27.8980μs 35.8448 KOps/s 35.1743 KOps/s $\color{#35bf28}+1.91\%$
test_step_mdp_speed[False-True-True-False-False] 49.8710μs 17.1459μs 58.3231 KOps/s 58.2820 KOps/s $\color{#35bf28}+0.07\%$
test_step_mdp_speed[False-True-False-True-True] 76.3710μs 47.0633μs 21.2480 KOps/s 21.3206 KOps/s $\color{#d91a1a}-0.34\%$
test_step_mdp_speed[False-True-False-True-False] 56.5810μs 30.3604μs 32.9377 KOps/s 33.8659 KOps/s $\color{#d91a1a}-2.74\%$
test_step_mdp_speed[False-True-False-False-True] 3.2427ms 31.0497μs 32.2065 KOps/s 32.5942 KOps/s $\color{#d91a1a}-1.19\%$
test_step_mdp_speed[False-True-False-False-False] 65.6410μs 19.0162μs 52.5868 KOps/s 51.7104 KOps/s $\color{#35bf28}+1.69\%$
test_step_mdp_speed[False-False-True-True-True] 83.9010μs 49.6143μs 20.1555 KOps/s 20.5266 KOps/s $\color{#d91a1a}-1.81\%$
test_step_mdp_speed[False-False-True-True-False] 80.8510μs 32.7471μs 30.5371 KOps/s 30.7778 KOps/s $\color{#d91a1a}-0.78\%$
test_step_mdp_speed[False-False-True-False-True] 61.7810μs 30.8605μs 32.4039 KOps/s 32.9267 KOps/s $\color{#d91a1a}-1.59\%$
test_step_mdp_speed[False-False-True-False-False] 45.2110μs 19.2587μs 51.9247 KOps/s 52.1231 KOps/s $\color{#d91a1a}-0.38\%$
test_step_mdp_speed[False-False-False-True-True] 99.2020μs 51.0678μs 19.5818 KOps/s 19.4348 KOps/s $\color{#35bf28}+0.76\%$
test_step_mdp_speed[False-False-False-True-False] 59.7810μs 35.1429μs 28.4553 KOps/s 28.6751 KOps/s $\color{#d91a1a}-0.77\%$
test_step_mdp_speed[False-False-False-False-True] 61.6010μs 32.1668μs 31.0879 KOps/s 31.1750 KOps/s $\color{#d91a1a}-0.28\%$
test_step_mdp_speed[False-False-False-False-False] 54.8010μs 21.4847μs 46.5447 KOps/s 46.5181 KOps/s $\color{#35bf28}+0.06\%$
test_values[generalized_advantage_estimate-True-True] 25.0978ms 24.4379ms 40.9201 Ops/s 40.3477 Ops/s $\color{#35bf28}+1.42\%$
test_values[vec_generalized_advantage_estimate-True-True] 0.1022s 2.9378ms 340.3927 Ops/s 344.9144 Ops/s $\color{#d91a1a}-1.31\%$
test_values[td0_return_estimate-False-False] 0.1095ms 79.8289μs 12.5268 KOps/s 12.9010 KOps/s $\color{#d91a1a}-2.90\%$
test_values[td1_return_estimate-False-False] 57.5031ms 54.4991ms 18.3489 Ops/s 18.2698 Ops/s $\color{#35bf28}+0.43\%$
test_values[vec_td1_return_estimate-False-False] 1.3320ms 1.0813ms 924.8440 Ops/s 932.4317 Ops/s $\color{#d91a1a}-0.81\%$
test_values[td_lambda_return_estimate-True-False] 93.1977ms 89.5145ms 11.1714 Ops/s 11.6186 Ops/s $\color{#d91a1a}-3.85\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.3079ms 1.0784ms 927.3383 Ops/s 934.0417 Ops/s $\color{#d91a1a}-0.72\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 26.0266ms 24.1388ms 41.4271 Ops/s 41.3137 Ops/s $\color{#35bf28}+0.27\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.0417ms 0.7540ms 1.3263 KOps/s 1.3470 KOps/s $\color{#d91a1a}-1.54\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7589ms 0.6716ms 1.4890 KOps/s 1.5070 KOps/s $\color{#d91a1a}-1.19\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5526ms 1.4783ms 676.4625 Ops/s 681.3614 Ops/s $\color{#d91a1a}-0.72\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.7552ms 0.6978ms 1.4331 KOps/s 1.4955 KOps/s $\color{#d91a1a}-4.17\%$
test_dqn_speed[False-None] 1.6121ms 1.4970ms 667.9946 Ops/s 673.9351 Ops/s $\color{#d91a1a}-0.88\%$
test_dqn_speed[False-backward] 2.1313ms 2.0984ms 476.5578 Ops/s 474.7857 Ops/s $\color{#35bf28}+0.37\%$
test_dqn_speed[True-None] 0.6678ms 0.5549ms 1.8020 KOps/s 1.8256 KOps/s $\color{#d91a1a}-1.29\%$
test_dqn_speed[True-backward] 1.1662ms 1.0993ms 909.6776 Ops/s 902.9478 Ops/s $\color{#35bf28}+0.75\%$
test_dqn_speed[reduce-overhead-None] 0.6277ms 0.5624ms 1.7780 KOps/s 1.7729 KOps/s $\color{#35bf28}+0.29\%$
test_dqn_speed[reduce-overhead-backward] 1.1052ms 0.9619ms 1.0396 KOps/s 1.0393 KOps/s $\color{#35bf28}+0.03\%$
test_ddpg_speed[False-None] 3.1269ms 2.8130ms 355.4916 Ops/s 352.9791 Ops/s $\color{#35bf28}+0.71\%$
test_ddpg_speed[False-backward] 4.4705ms 4.0585ms 246.3967 Ops/s 246.0320 Ops/s $\color{#35bf28}+0.15\%$
test_ddpg_speed[True-None] 1.4440ms 1.0845ms 922.0514 Ops/s 923.2090 Ops/s $\color{#d91a1a}-0.13\%$
test_ddpg_speed[True-backward] 2.2281ms 2.1473ms 465.7052 Ops/s 463.1703 Ops/s $\color{#35bf28}+0.55\%$
test_ddpg_speed[reduce-overhead-None] 1.2193ms 1.1002ms 908.9062 Ops/s 907.6461 Ops/s $\color{#35bf28}+0.14\%$
test_ddpg_speed[reduce-overhead-backward] 1.7361ms 1.6220ms 616.5220 Ops/s 606.6256 Ops/s $\color{#35bf28}+1.63\%$
test_sac_speed[False-None] 8.4351ms 7.9750ms 125.3920 Ops/s 124.8572 Ops/s $\color{#35bf28}+0.43\%$
test_sac_speed[False-backward] 11.6953ms 10.8478ms 92.1842 Ops/s 91.5191 Ops/s $\color{#35bf28}+0.73\%$
test_sac_speed[True-None] 1.6096ms 1.5390ms 649.7727 Ops/s 649.7484 Ops/s $+0.00\%$
test_sac_speed[True-backward] 3.3495ms 3.1984ms 312.6596 Ops/s 296.0723 Ops/s $\textbf{\color{#35bf28}+5.60\%}$
test_sac_speed[reduce-overhead-None] 23.3976ms 12.7450ms 78.4619 Ops/s 78.2211 Ops/s $\color{#35bf28}+0.31\%$
test_sac_speed[reduce-overhead-backward] 1.3824ms 1.3281ms 752.9354 Ops/s 654.6976 Ops/s $\textbf{\color{#35bf28}+15.01\%}$
test_redq_speed[False-None] 8.2518ms 7.4781ms 133.7239 Ops/s 132.0938 Ops/s $\color{#35bf28}+1.23\%$
test_redq_speed[False-backward] 11.9440ms 11.2185ms 89.1388 Ops/s 85.4041 Ops/s $\color{#35bf28}+4.37\%$
test_redq_speed[True-None] 2.1120ms 1.9626ms 509.5344 Ops/s 495.2697 Ops/s $\color{#35bf28}+2.88\%$
test_redq_speed[True-backward] 3.7308ms 3.6085ms 277.1222 Ops/s 276.3648 Ops/s $\color{#35bf28}+0.27\%$
test_redq_speed[reduce-overhead-None] 2.1254ms 1.9757ms 506.1607 Ops/s 503.3417 Ops/s $\color{#35bf28}+0.56\%$
test_redq_speed[reduce-overhead-backward] 3.7216ms 3.6352ms 275.0863 Ops/s 275.3863 Ops/s $\color{#d91a1a}-0.11\%$
test_redq_deprec_speed[False-None] 9.4477ms 8.9682ms 111.5057 Ops/s 109.9248 Ops/s $\color{#35bf28}+1.44\%$
test_redq_deprec_speed[False-backward] 12.3419ms 11.8900ms 84.1046 Ops/s 82.8878 Ops/s $\color{#35bf28}+1.47\%$
test_redq_deprec_speed[True-None] 2.4184ms 2.3330ms 428.6288 Ops/s 426.0193 Ops/s $\color{#35bf28}+0.61\%$
test_redq_deprec_speed[True-backward] 4.0286ms 3.9332ms 254.2472 Ops/s 251.5444 Ops/s $\color{#35bf28}+1.07\%$
test_redq_deprec_speed[reduce-overhead-None] 2.4068ms 2.3206ms 430.9207 Ops/s 426.4651 Ops/s $\color{#35bf28}+1.04\%$
test_redq_deprec_speed[reduce-overhead-backward] 4.0406ms 3.9608ms 252.4756 Ops/s 242.7458 Ops/s $\color{#35bf28}+4.01\%$
test_td3_speed[False-None] 7.9118ms 7.8521ms 127.3550 Ops/s 126.4617 Ops/s $\color{#35bf28}+0.71\%$
test_td3_speed[False-backward] 10.6159ms 10.1299ms 98.7175 Ops/s 96.1819 Ops/s $\color{#35bf28}+2.64\%$
test_td3_speed[True-None] 1.6312ms 1.5987ms 625.5002 Ops/s 622.4879 Ops/s $\color{#35bf28}+0.48\%$
test_td3_speed[True-backward] 3.3957ms 3.1326ms 319.2240 Ops/s 327.7786 Ops/s $\color{#d91a1a}-2.61\%$
test_td3_speed[reduce-overhead-None] 51.6688ms 26.3961ms 37.8843 Ops/s 38.3761 Ops/s $\color{#d91a1a}-1.28\%$
test_td3_speed[reduce-overhead-backward] 1.3828ms 1.2951ms 772.1366 Ops/s 764.9653 Ops/s $\color{#35bf28}+0.94\%$
test_cql_speed[False-None] 17.2920ms 16.6877ms 59.9243 Ops/s 59.3239 Ops/s $\color{#35bf28}+1.01\%$
test_cql_speed[False-backward] 22.4348ms 21.7605ms 45.9549 Ops/s 45.7703 Ops/s $\color{#35bf28}+0.40\%$
test_cql_speed[True-None] 3.3117ms 3.0298ms 330.0569 Ops/s 343.8701 Ops/s $\color{#d91a1a}-4.02\%$
test_cql_speed[True-backward] 5.4392ms 5.2040ms 192.1588 Ops/s 197.5549 Ops/s $\color{#d91a1a}-2.73\%$
test_cql_speed[reduce-overhead-None] 0.3600s 15.0592ms 66.4047 Ops/s 75.1914 Ops/s $\textbf{\color{#d91a1a}-11.69\%}$
test_cql_speed[reduce-overhead-backward] 1.7773ms 1.7002ms 588.1778 Ops/s 646.4784 Ops/s $\textbf{\color{#d91a1a}-9.02\%}$
test_a2c_speed[False-None] 3.2987ms 3.2000ms 312.4968 Ops/s 308.1720 Ops/s $\color{#35bf28}+1.40\%$
test_a2c_speed[False-backward] 6.9271ms 6.3233ms 158.1457 Ops/s 163.2341 Ops/s $\color{#d91a1a}-3.12\%$
test_a2c_speed[True-None] 1.0607ms 1.0080ms 992.0464 Ops/s 988.5527 Ops/s $\color{#35bf28}+0.35\%$
test_a2c_speed[True-backward] 3.0263ms 2.7344ms 365.7117 Ops/s 386.5295 Ops/s $\textbf{\color{#d91a1a}-5.39\%}$
test_a2c_speed[reduce-overhead-None] 21.1975ms 11.3969ms 87.7435 Ops/s 87.7850 Ops/s $\color{#d91a1a}-0.05\%$
test_a2c_speed[reduce-overhead-backward] 1.1527ms 1.1147ms 897.1157 Ops/s 998.8853 Ops/s $\textbf{\color{#d91a1a}-10.19\%}$
test_ppo_speed[False-None] 3.7894ms 3.6806ms 271.6935 Ops/s 270.9339 Ops/s $\color{#35bf28}+0.28\%$
test_ppo_speed[False-backward] 7.3964ms 6.9944ms 142.9714 Ops/s 147.0212 Ops/s $\color{#d91a1a}-2.75\%$
test_ppo_speed[True-None] 1.0119ms 0.9524ms 1.0500 KOps/s 1.0361 KOps/s $\color{#35bf28}+1.34\%$
test_ppo_speed[True-backward] 2.6575ms 2.5454ms 392.8687 Ops/s 371.3459 Ops/s $\textbf{\color{#35bf28}+5.80\%}$
test_ppo_speed[reduce-overhead-None] 0.6016ms 0.5296ms 1.8884 KOps/s 68.5517 Ops/s $\textbf{\color{#35bf28}+2654.70\%}$
test_ppo_speed[reduce-overhead-backward] 1.0157ms 0.9637ms 1.0377 KOps/s 846.7976 Ops/s $\textbf{\color{#35bf28}+22.54\%}$
test_reinforce_speed[False-None] 2.4250ms 2.2471ms 445.0153 Ops/s 439.5744 Ops/s $\color{#35bf28}+1.24\%$
test_reinforce_speed[False-backward] 3.8209ms 3.2543ms 307.2871 Ops/s 297.5467 Ops/s $\color{#35bf28}+3.27\%$
test_reinforce_speed[True-None] 0.9680ms 0.8324ms 1.2013 KOps/s 1.1375 KOps/s $\textbf{\color{#35bf28}+5.61\%}$
test_reinforce_speed[True-backward] 2.5058ms 2.3903ms 418.3502 Ops/s 387.0923 Ops/s $\textbf{\color{#35bf28}+8.08\%}$
test_reinforce_speed[reduce-overhead-None] 0.2929s 12.2248ms 81.8013 Ops/s 88.6747 Ops/s $\textbf{\color{#d91a1a}-7.75\%}$
test_reinforce_speed[reduce-overhead-backward] 1.1058ms 1.0296ms 971.2302 Ops/s 828.0031 Ops/s $\textbf{\color{#35bf28}+17.30\%}$
test_iql_speed[False-None] 9.8626ms 9.3145ms 107.3596 Ops/s 108.4916 Ops/s $\color{#d91a1a}-1.04\%$
test_iql_speed[False-backward] 13.7260ms 12.9367ms 77.2994 Ops/s 75.8300 Ops/s $\color{#35bf28}+1.94\%$
test_iql_speed[True-None] 1.9028ms 1.7703ms 564.8889 Ops/s 568.6219 Ops/s $\color{#d91a1a}-0.66\%$
test_iql_speed[True-backward] 4.5289ms 4.1795ms 239.2647 Ops/s 229.6830 Ops/s $\color{#35bf28}+4.17\%$
test_iql_speed[reduce-overhead-None] 20.2119ms 11.4640ms 87.2297 Ops/s 68.0421 Ops/s $\textbf{\color{#35bf28}+28.20\%}$
test_iql_speed[reduce-overhead-backward] 1.4759ms 1.4174ms 705.5411 Ops/s 686.4069 Ops/s $\color{#35bf28}+2.79\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 8.0029ms 6.4767ms 154.4004 Ops/s 152.0102 Ops/s $\color{#35bf28}+1.57\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.4855ms 0.3095ms 3.2310 KOps/s 2.7862 KOps/s $\textbf{\color{#35bf28}+15.96\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5666ms 0.2780ms 3.5968 KOps/s 3.1280 KOps/s $\textbf{\color{#35bf28}+14.99\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.5667ms 6.1879ms 161.6061 Ops/s 160.2895 Ops/s $\color{#35bf28}+0.82\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.4026ms 0.3121ms 3.2043 KOps/s 3.7746 KOps/s $\textbf{\color{#d91a1a}-15.11\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.4833ms 0.2791ms 3.5827 KOps/s 4.0632 KOps/s $\textbf{\color{#d91a1a}-11.82\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.5607ms 1.3135ms 761.3054 Ops/s 783.3429 Ops/s $\color{#d91a1a}-2.81\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.4827ms 1.2953ms 772.0357 Ops/s 756.4541 Ops/s $\color{#35bf28}+2.06\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.4483ms 6.3471ms 157.5518 Ops/s 155.0121 Ops/s $\color{#35bf28}+1.64\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9108ms 0.4297ms 2.3271 KOps/s 2.2755 KOps/s $\color{#35bf28}+2.26\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7193ms 0.4299ms 2.3261 KOps/s 2.2699 KOps/s $\color{#35bf28}+2.48\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.2941ms 6.1978ms 161.3480 Ops/s 158.8842 Ops/s $\color{#35bf28}+1.55\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.5879ms 0.3215ms 3.1105 KOps/s 2.9432 KOps/s $\textbf{\color{#35bf28}+5.69\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5400ms 0.3423ms 2.9214 KOps/s 2.9705 KOps/s $\color{#d91a1a}-1.66\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.4407ms 6.1116ms 163.6245 Ops/s 159.7942 Ops/s $\color{#35bf28}+2.40\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9110ms 0.3329ms 3.0042 KOps/s 3.5703 KOps/s $\textbf{\color{#d91a1a}-15.86\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6879ms 0.3285ms 3.0437 KOps/s 3.2196 KOps/s $\textbf{\color{#d91a1a}-5.46\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.4845ms 6.2753ms 159.3537 Ops/s 155.1680 Ops/s $\color{#35bf28}+2.70\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.8038ms 0.4285ms 2.3338 KOps/s 2.2914 KOps/s $\color{#35bf28}+1.85\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.5810ms 0.3930ms 2.5442 KOps/s 2.3931 KOps/s $\textbf{\color{#35bf28}+6.31\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 7.3727ms 5.7216ms 174.7778 Ops/s 183.6871 Ops/s $\color{#d91a1a}-4.85\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 10.7509ms 2.2443ms 445.5821 Ops/s 435.3090 Ops/s $\color{#35bf28}+2.36\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.3834ms 1.1571ms 864.2573 Ops/s 860.4167 Ops/s $\color{#35bf28}+0.45\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 8.6643ms 5.4553ms 183.3093 Ops/s 185.6805 Ops/s $\color{#d91a1a}-1.28\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 7.2825ms 2.0676ms 483.6442 Ops/s 459.8308 Ops/s $\textbf{\color{#35bf28}+5.18\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 8.5973ms 1.3051ms 766.2177 Ops/s 789.9760 Ops/s $\color{#d91a1a}-3.01\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.5022s 15.7236ms 63.5987 Ops/s 32.8815 Ops/s $\textbf{\color{#35bf28}+93.42\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 10.1663ms 2.3063ms 433.5973 Ops/s 463.0983 Ops/s $\textbf{\color{#d91a1a}-6.37\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 8.0632ms 1.4358ms 696.4747 Ops/s 872.5844 Ops/s $\textbf{\color{#d91a1a}-20.18\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 15.6485ms 15.4254ms 64.8283 Ops/s 63.9884 Ops/s $\color{#35bf28}+1.31\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 19.4760ms 17.5244ms 57.0634 Ops/s 57.0304 Ops/s $\color{#35bf28}+0.06\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 20.9298ms 19.7846ms 50.5444 Ops/s 49.3282 Ops/s $\color{#35bf28}+2.47\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 19.9579ms 17.8314ms 56.0809 Ops/s 56.4301 Ops/s $\color{#d91a1a}-0.62\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 19.8832ms 19.6273ms 50.9494 Ops/s 50.1642 Ops/s $\color{#35bf28}+1.57\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 20.6926ms 19.1850ms 52.1241 Ops/s 50.6715 Ops/s $\color{#35bf28}+2.87\%$

@vmoens vmoens force-pushed the fix-wehels-download-artifact branch from 3fb4d9b to 7277ccc Compare January 9, 2025 18:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CI Has to do with CI setup (e.g. wheels & builds, tests...) CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot