Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BugFix,CI] Fix storage filename tests #850

Merged
merged 1 commit into from
Jul 3, 2024
Merged

[BugFix,CI] Fix storage filename tests #850

merged 1 commit into from
Jul 3, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Jul 3, 2024

Interestingly apply silicon runners are the only machines that don't have root permission in the CI, and hence are the only one that run the TestReadWrite test.

All file-stored storages used to have a filename associated but we deprecated this since there was not guarantee that a change in a non-writable file would be reflected on the corresponding opened tensor (making the filename useless and misleading).
This PR fixes that test.

cc @mikaylagawarecki

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 3, 2024
@vmoens vmoens merged commit 03b86d3 into main Jul 3, 2024
13 of 21 checks passed
@vmoens vmoens deleted the fix-ci-metal branch July 3, 2024 15:35
Copy link

github-actions bot commented Jul 3, 2024

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 144. Improved: $\large\color{#35bf28}12$. Worsened: $\large\color{#d91a1a}14$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_plain_set_nested 38.6920μs 17.0797μs 58.5490 KOps/s 59.6863 KOps/s $\color{#d91a1a}-1.91\%$
test_plain_set_stack_nested 40.2640μs 17.3575μs 57.6121 KOps/s 58.5141 KOps/s $\color{#d91a1a}-1.54\%$
test_plain_set_nested_inplace 61.6550μs 19.3138μs 51.7765 KOps/s 51.8764 KOps/s $\color{#d91a1a}-0.19\%$
test_plain_set_stack_nested_inplace 51.5250μs 19.1915μs 52.1065 KOps/s 51.8073 KOps/s $\color{#35bf28}+0.58\%$
test_items 26.7490μs 2.6807μs 373.0431 KOps/s 380.9033 KOps/s $\color{#d91a1a}-2.06\%$
test_items_nested 1.1261ms 0.2743ms 3.6453 KOps/s 3.6689 KOps/s $\color{#d91a1a}-0.64\%$
test_items_nested_locked 0.5806ms 0.2743ms 3.6458 KOps/s 3.6080 KOps/s $\color{#35bf28}+1.05\%$
test_items_nested_leaf 0.1550ms 78.1454μs 12.7967 KOps/s 12.7547 KOps/s $\color{#35bf28}+0.33\%$
test_items_stack_nested 0.4604ms 0.2757ms 3.6267 KOps/s 3.6672 KOps/s $\color{#d91a1a}-1.10\%$
test_items_stack_nested_leaf 0.1345ms 80.8053μs 12.3754 KOps/s 12.6864 KOps/s $\color{#d91a1a}-2.45\%$
test_items_stack_nested_locked 0.4803ms 0.2734ms 3.6574 KOps/s 3.6013 KOps/s $\color{#35bf28}+1.56\%$
test_keys 29.3340μs 3.9221μs 254.9645 KOps/s 258.7433 KOps/s $\color{#d91a1a}-1.46\%$
test_keys_nested 0.2951ms 0.1367ms 7.3156 KOps/s 7.2222 KOps/s $\color{#35bf28}+1.29\%$
test_keys_nested_locked 0.7455ms 0.1422ms 7.0320 KOps/s 6.9674 KOps/s $\color{#35bf28}+0.93\%$
test_keys_nested_leaf 0.2250ms 0.1170ms 8.5467 KOps/s 8.4225 KOps/s $\color{#35bf28}+1.48\%$
test_keys_stack_nested 0.2408ms 0.1368ms 7.3117 KOps/s 7.1728 KOps/s $\color{#35bf28}+1.94\%$
test_keys_stack_nested_leaf 0.2453ms 0.1158ms 8.6375 KOps/s 8.5467 KOps/s $\color{#35bf28}+1.06\%$
test_keys_stack_nested_locked 0.2441ms 0.1417ms 7.0587 KOps/s 6.9023 KOps/s $\color{#35bf28}+2.27\%$
test_values 11.4838μs 1.1753μs 850.8113 KOps/s 820.8534 KOps/s $\color{#35bf28}+3.65\%$
test_values_nested 0.1336ms 49.8258μs 20.0699 KOps/s 19.7996 KOps/s $\color{#35bf28}+1.37\%$
test_values_nested_locked 0.1046ms 50.2644μs 19.8948 KOps/s 19.6637 KOps/s $\color{#35bf28}+1.18\%$
test_values_nested_leaf 96.6000μs 45.1708μs 22.1382 KOps/s 21.6163 KOps/s $\color{#35bf28}+2.41\%$
test_values_stack_nested 98.5240μs 50.3923μs 19.8443 KOps/s 19.6069 KOps/s $\color{#35bf28}+1.21\%$
test_values_stack_nested_leaf 0.1084ms 45.1960μs 22.1258 KOps/s 21.5116 KOps/s $\color{#35bf28}+2.86\%$
test_values_stack_nested_locked 0.1085ms 50.5471μs 19.7835 KOps/s 19.6651 KOps/s $\color{#35bf28}+0.60\%$
test_membership 43.1610μs 1.3484μs 741.6225 KOps/s 735.8870 KOps/s $\color{#35bf28}+0.78\%$
test_membership_nested 31.2980μs 3.4316μs 291.4071 KOps/s 289.9090 KOps/s $\color{#35bf28}+0.52\%$
test_membership_nested_leaf 44.0510μs 3.4505μs 289.8122 KOps/s 288.8017 KOps/s $\color{#35bf28}+0.35\%$
test_membership_stacked_nested 33.5720μs 3.4522μs 289.6691 KOps/s 286.3188 KOps/s $\color{#35bf28}+1.17\%$
test_membership_stacked_nested_leaf 22.5110μs 3.4635μs 288.7293 KOps/s 285.6006 KOps/s $\color{#35bf28}+1.10\%$
test_membership_nested_last 33.8730μs 4.1697μs 239.8262 KOps/s 227.8350 KOps/s $\textbf{\color{#35bf28}+5.26\%}$
test_membership_nested_leaf_last 30.6070μs 4.2101μs 237.5264 KOps/s 232.2311 KOps/s $\color{#35bf28}+2.28\%$
test_membership_stacked_nested_last 50.8240μs 4.1524μs 240.8220 KOps/s 233.1916 KOps/s $\color{#35bf28}+3.27\%$
test_membership_stacked_nested_leaf_last 28.7730μs 4.2707μs 234.1527 KOps/s 233.2182 KOps/s $\color{#35bf28}+0.40\%$
test_nested_getleaf 54.9420μs 10.6533μs 93.8673 KOps/s 93.6396 KOps/s $\color{#35bf28}+0.24\%$
test_nested_get 51.1050μs 10.1244μs 98.7709 KOps/s 98.6291 KOps/s $\color{#35bf28}+0.14\%$
test_stacked_getleaf 31.1290μs 10.4333μs 95.8473 KOps/s 94.2112 KOps/s $\color{#35bf28}+1.74\%$
test_stacked_get 52.4870μs 9.9930μs 100.0699 KOps/s 99.8855 KOps/s $\color{#35bf28}+0.18\%$
test_nested_getitemleaf 53.5290μs 11.0613μs 90.4050 KOps/s 89.7904 KOps/s $\color{#35bf28}+0.68\%$
test_nested_getitem 30.9970μs 10.1841μs 98.1926 KOps/s 97.0539 KOps/s $\color{#35bf28}+1.17\%$
test_stacked_getitemleaf 77.1780μs 11.9756μs 83.5031 KOps/s 90.2807 KOps/s $\textbf{\color{#d91a1a}-7.51\%}$
test_stacked_getitem 72.6010μs 9.6245μs 103.9015 KOps/s 97.9087 KOps/s $\textbf{\color{#35bf28}+6.12\%}$
test_lock_nested 59.6835ms 0.3943ms 2.5361 KOps/s 2.9508 KOps/s $\textbf{\color{#d91a1a}-14.05\%}$
test_lock_stack_nested 0.6236ms 0.3010ms 3.3220 KOps/s 3.2235 KOps/s $\color{#35bf28}+3.05\%$
test_unlock_nested 0.7456ms 0.3381ms 2.9579 KOps/s 2.8715 KOps/s $\color{#35bf28}+3.01\%$
test_unlock_stack_nested 0.5328ms 0.3083ms 3.2436 KOps/s 3.1655 KOps/s $\color{#35bf28}+2.47\%$
test_flatten_speed 0.2315ms 96.8789μs 10.3222 KOps/s 10.2190 KOps/s $\color{#35bf28}+1.01\%$
test_unflatten_speed 0.5135ms 0.4070ms 2.4568 KOps/s 2.4010 KOps/s $\color{#35bf28}+2.32\%$
test_common_ops 5.3584ms 0.7328ms 1.3646 KOps/s 1.3763 KOps/s $\color{#d91a1a}-0.85\%$
test_creation 0.1028ms 1.9034μs 525.3756 KOps/s 525.6087 KOps/s $\color{#d91a1a}-0.04\%$
test_creation_empty 37.0390μs 10.9394μs 91.4129 KOps/s 96.6610 KOps/s $\textbf{\color{#d91a1a}-5.43\%}$
test_creation_nested_1 43.6710μs 13.5982μs 73.5391 KOps/s 77.2074 KOps/s $\color{#d91a1a}-4.75\%$
test_creation_nested_2 94.1150μs 17.1882μs 58.1794 KOps/s 60.2006 KOps/s $\color{#d91a1a}-3.36\%$
test_clone 0.1330ms 13.2108μs 75.6955 KOps/s 75.8123 KOps/s $\color{#d91a1a}-0.15\%$
test_getitem[int] 36.2070μs 11.0074μs 90.8478 KOps/s 88.3882 KOps/s $\color{#35bf28}+2.78\%$
test_getitem[slice_int] 84.4670μs 22.0917μs 45.2658 KOps/s 44.7272 KOps/s $\color{#35bf28}+1.20\%$
test_getitem[range] 80.6900μs 64.0490μs 15.6130 KOps/s 16.4977 KOps/s $\textbf{\color{#d91a1a}-5.36\%}$
test_getitem[tuple] 57.4470μs 18.4277μs 54.2662 KOps/s 53.7298 KOps/s $\color{#35bf28}+1.00\%$
test_getitem[list] 0.1353ms 41.1997μs 24.2720 KOps/s 23.3570 KOps/s $\color{#35bf28}+3.92\%$
test_setitem_dim[int] 78.3760μs 34.1722μs 29.2635 KOps/s 31.5031 KOps/s $\textbf{\color{#d91a1a}-7.11\%}$
test_setitem_dim[slice_int] 0.1220ms 61.8874μs 16.1584 KOps/s 17.1766 KOps/s $\textbf{\color{#d91a1a}-5.93\%}$
test_setitem_dim[range] 0.1302ms 83.9495μs 11.9119 KOps/s 12.1705 KOps/s $\color{#d91a1a}-2.12\%$
test_setitem_dim[tuple] 89.7970μs 50.9680μs 19.6202 KOps/s 20.7306 KOps/s $\textbf{\color{#d91a1a}-5.36\%}$
test_setitem 67.3950μs 20.5219μs 48.7284 KOps/s 49.1331 KOps/s $\color{#d91a1a}-0.82\%$
test_set 68.6080μs 20.1629μs 49.5960 KOps/s 51.7608 KOps/s $\color{#d91a1a}-4.18\%$
test_set_shared 2.0541ms 0.1426ms 7.0148 KOps/s 6.9759 KOps/s $\color{#35bf28}+0.56\%$
test_update 0.1303ms 22.8883μs 43.6904 KOps/s 46.0823 KOps/s $\textbf{\color{#d91a1a}-5.19\%}$
test_update_nested 0.1341ms 32.0798μs 31.1723 KOps/s 33.7084 KOps/s $\textbf{\color{#d91a1a}-7.52\%}$
test_update__nested 87.6830μs 24.9504μs 40.0796 KOps/s 40.4968 KOps/s $\color{#d91a1a}-1.03\%$
test_set_nested 0.1005ms 21.9694μs 45.5178 KOps/s 47.1346 KOps/s $\color{#d91a1a}-3.43\%$
test_set_nested_new 98.1730μs 25.9841μs 38.4850 KOps/s 39.6612 KOps/s $\color{#d91a1a}-2.97\%$
test_select 0.1523ms 42.2378μs 23.6755 KOps/s 25.1339 KOps/s $\textbf{\color{#d91a1a}-5.80\%}$
test_select_nested 0.1161ms 57.3170μs 17.4468 KOps/s 17.2783 KOps/s $\color{#35bf28}+0.98\%$
test_exclude_nested 0.6238ms 0.1219ms 8.2034 KOps/s 8.4054 KOps/s $\color{#d91a1a}-2.40\%$
test_empty[True] 0.7206ms 0.3973ms 2.5169 KOps/s 2.5128 KOps/s $\color{#35bf28}+0.16\%$
test_empty[False] 8.3456μs 1.0126μs 987.5897 KOps/s 964.9591 KOps/s $\color{#35bf28}+2.35\%$
test_unbind_speed 1.7698ms 0.2467ms 4.0540 KOps/s 3.9676 KOps/s $\color{#35bf28}+2.18\%$
test_unbind_speed_stack0 0.4602ms 0.2422ms 4.1284 KOps/s 4.0170 KOps/s $\color{#35bf28}+2.77\%$
test_unbind_speed_stack1 73.4897ms 0.7104ms 1.4076 KOps/s 1.3988 KOps/s $\color{#35bf28}+0.63\%$
test_split 76.5445ms 1.6047ms 623.1670 Ops/s 632.4150 Ops/s $\color{#d91a1a}-1.46\%$
test_chunk 76.6694ms 1.5811ms 632.4635 Ops/s 632.8106 Ops/s $\color{#d91a1a}-0.05\%$
test_creation[device0] 0.2148ms 86.0039μs 11.6274 KOps/s 11.6676 KOps/s $\color{#d91a1a}-0.34\%$
test_creation_from_tensor 3.7257ms 87.0229μs 11.4912 KOps/s 11.4869 KOps/s $\color{#35bf28}+0.04\%$
test_add_one[memmap_tensor0] 0.1366ms 5.4932μs 182.0440 KOps/s 177.7100 KOps/s $\color{#35bf28}+2.44\%$
test_contiguous[memmap_tensor0] 13.0340μs 0.6451μs 1.5503 MOps/s 1.5705 MOps/s $\color{#d91a1a}-1.29\%$
test_stack[memmap_tensor0] 27.1910μs 3.5741μs 279.7907 KOps/s 277.0188 KOps/s $\color{#35bf28}+1.00\%$
test_memmaptd_index 0.8660ms 0.2524ms 3.9627 KOps/s 3.8847 KOps/s $\color{#35bf28}+2.01\%$
test_memmaptd_index_astensor 0.6364ms 0.3240ms 3.0866 KOps/s 3.0332 KOps/s $\color{#35bf28}+1.76\%$
test_memmaptd_index_op 1.2218ms 0.6305ms 1.5861 KOps/s 1.5910 KOps/s $\color{#d91a1a}-0.31\%$
test_serialize_model 0.1758s 0.1061s 9.4226 Ops/s 8.9513 Ops/s $\textbf{\color{#35bf28}+5.27\%}$
test_serialize_model_pickle 0.5871s 0.3790s 2.6384 Ops/s 2.6121 Ops/s $\color{#35bf28}+1.01\%$
test_serialize_weights 0.1041s 96.8164ms 10.3288 Ops/s 9.3864 Ops/s $\textbf{\color{#35bf28}+10.04\%}$
test_serialize_weights_returnearly 0.1891s 0.1279s 7.8210 Ops/s 8.5612 Ops/s $\textbf{\color{#d91a1a}-8.65\%}$
test_serialize_weights_pickle 0.7091s 0.4860s 2.0575 Ops/s 1.3502 Ops/s $\textbf{\color{#35bf28}+52.38\%}$
test_serialize_weights_filesystem 0.1042s 94.7502ms 10.5541 Ops/s 9.7507 Ops/s $\textbf{\color{#35bf28}+8.24\%}$
test_serialize_model_filesystem 0.1658s 0.1024s 9.7689 Ops/s 10.4183 Ops/s $\textbf{\color{#d91a1a}-6.23\%}$
test_reshape_pytree 54.0410μs 25.3167μs 39.4997 KOps/s 38.4835 KOps/s $\color{#35bf28}+2.64\%$
test_reshape_td 72.7050μs 33.4060μs 29.9348 KOps/s 29.7030 KOps/s $\color{#35bf28}+0.78\%$
test_view_pytree 63.8590μs 25.0604μs 39.9036 KOps/s 38.8212 KOps/s $\color{#35bf28}+2.79\%$
test_view_td 88.0940μs 38.9506μs 25.6736 KOps/s 26.4006 KOps/s $\color{#d91a1a}-2.75\%$
test_unbind_pytree 66.6840μs 29.1546μs 34.2999 KOps/s 33.2132 KOps/s $\color{#35bf28}+3.27\%$
test_unbind_td 0.3823ms 36.2514μs 27.5851 KOps/s 27.4274 KOps/s $\color{#35bf28}+0.57\%$
test_split_pytree 77.3840μs 28.8241μs 34.6932 KOps/s 33.6386 KOps/s $\color{#35bf28}+3.14\%$
test_split_td 0.1238ms 39.2552μs 25.4743 KOps/s 25.4267 KOps/s $\color{#35bf28}+0.19\%$
test_add_pytree 98.9360μs 34.3683μs 29.0966 KOps/s 27.8919 KOps/s $\color{#35bf28}+4.32\%$
test_add_td 0.1184ms 55.8644μs 17.9005 KOps/s 18.6375 KOps/s $\color{#d91a1a}-3.95\%$
test_distributed 0.1834ms 0.1040ms 9.6127 KOps/s 9.6296 KOps/s $\color{#d91a1a}-0.18\%$
test_tdmodule 0.1122ms 18.1714μs 55.0315 KOps/s 55.7946 KOps/s $\color{#d91a1a}-1.37\%$
test_tdmodule_dispatch 56.2850μs 36.1486μs 27.6636 KOps/s 28.9234 KOps/s $\color{#d91a1a}-4.36\%$
test_tdseq 37.2100μs 21.1729μs 47.2301 KOps/s 48.8895 KOps/s $\color{#d91a1a}-3.39\%$
test_tdseq_dispatch 63.8090μs 41.8941μs 23.8697 KOps/s 25.3122 KOps/s $\textbf{\color{#d91a1a}-5.70\%}$
test_instantiation_functorch 1.6262ms 1.3194ms 757.9184 Ops/s 754.5557 Ops/s $\color{#35bf28}+0.45\%$
test_instantiation_td 1.5896ms 1.0153ms 984.9516 Ops/s 976.1106 Ops/s $\color{#35bf28}+0.91\%$
test_exec_functorch 0.2257ms 0.1592ms 6.2803 KOps/s 6.0265 KOps/s $\color{#35bf28}+4.21\%$
test_exec_functional_call 0.2273ms 0.1473ms 6.7871 KOps/s 6.5331 KOps/s $\color{#35bf28}+3.89\%$
test_exec_td 0.2330ms 0.1436ms 6.9657 KOps/s 6.8293 KOps/s $\color{#35bf28}+2.00\%$
test_exec_td_decorator 0.7917ms 0.2203ms 4.5387 KOps/s 4.5063 KOps/s $\color{#35bf28}+0.72\%$
test_vmap_mlp_speed[True-True] 0.7120ms 0.4865ms 2.0556 KOps/s 2.0395 KOps/s $\color{#35bf28}+0.79\%$
test_vmap_mlp_speed[True-False] 0.6841ms 0.4790ms 2.0877 KOps/s 2.0552 KOps/s $\color{#35bf28}+1.58\%$
test_vmap_mlp_speed[False-True] 0.6220ms 0.3955ms 2.5282 KOps/s 2.5282 KOps/s $+0.00\%$
test_vmap_mlp_speed[False-False] 0.7692ms 0.3922ms 2.5497 KOps/s 2.5278 KOps/s $\color{#35bf28}+0.87\%$
test_vmap_mlp_speed_decorator[True-True] 1.1901ms 0.5556ms 1.7997 KOps/s 1.7808 KOps/s $\color{#35bf28}+1.06\%$
test_vmap_mlp_speed_decorator[True-False] 78.2773ms 0.6057ms 1.6510 KOps/s 1.7872 KOps/s $\textbf{\color{#d91a1a}-7.62\%}$
test_vmap_mlp_speed_decorator[False-True] 0.7735ms 0.4568ms 2.1889 KOps/s 2.1808 KOps/s $\color{#35bf28}+0.37\%$
test_vmap_mlp_speed_decorator[False-False] 0.7082ms 0.4597ms 2.1755 KOps/s 2.1444 KOps/s $\color{#35bf28}+1.45\%$
test_to_module_speed[True] 2.5598ms 1.6834ms 594.0284 Ops/s 523.4568 Ops/s $\textbf{\color{#35bf28}+13.48\%}$
test_to_module_speed[False] 1.7990ms 1.6544ms 604.4556 Ops/s 592.2413 Ops/s $\color{#35bf28}+2.06\%$
test_tc_init 0.1318ms 56.0144μs 17.8525 KOps/s 18.3638 KOps/s $\color{#d91a1a}-2.78\%$
test_tc_init_nested 0.1779ms 0.1126ms 8.8835 KOps/s 8.9024 KOps/s $\color{#d91a1a}-0.21\%$
test_tc_first_layer_tensor 29.4050μs 8.1223μs 123.1174 KOps/s 120.4065 KOps/s $\color{#35bf28}+2.25\%$
test_tc_first_layer_nontensor 43.5310μs 8.1273μs 123.0414 KOps/s 122.4795 KOps/s $\color{#35bf28}+0.46\%$
test_tc_second_layer_tensor 22.2710μs 2.4764μs 403.8109 KOps/s 392.2821 KOps/s $\color{#35bf28}+2.94\%$
test_tc_second_layer_nontensor 32.8110μs 9.1528μs 109.2562 KOps/s 106.0265 KOps/s $\color{#35bf28}+3.05\%$
test_unbind 83.2521ms 13.3778ms 74.7506 Ops/s 68.3634 Ops/s $\textbf{\color{#35bf28}+9.34\%}$
test_full_like 18.3867ms 11.4634ms 87.2342 Ops/s 81.5380 Ops/s $\textbf{\color{#35bf28}+6.99\%}$
test_zeros_like 12.0146ms 6.4390ms 155.3048 Ops/s 160.3573 Ops/s $\color{#d91a1a}-3.15\%$
test_ones_like 14.0383ms 6.7301ms 148.5861 Ops/s 146.1567 Ops/s $\color{#35bf28}+1.66\%$
test_clone 13.5750ms 8.3760ms 119.3894 Ops/s 113.1136 Ops/s $\textbf{\color{#35bf28}+5.55\%}$
test_squeeze 62.8270μs 12.4449μs 80.3541 KOps/s 79.2646 KOps/s $\color{#35bf28}+1.37\%$
test_unsqueeze 0.1560ms 96.0766μs 10.4084 KOps/s 10.5206 KOps/s $\color{#d91a1a}-1.07\%$
test_split 0.6279ms 0.2686ms 3.7236 KOps/s 3.5932 KOps/s $\color{#35bf28}+3.63\%$
test_permute 0.3966ms 0.2237ms 4.4695 KOps/s 4.5457 KOps/s $\color{#d91a1a}-1.68\%$
test_stack 30.8267ms 23.6380ms 42.3047 Ops/s 40.0443 Ops/s $\textbf{\color{#35bf28}+5.64\%}$
test_cat 28.5930ms 23.3694ms 42.7911 Ops/s 40.2818 Ops/s $\textbf{\color{#35bf28}+6.23\%}$

Copy link

github-actions bot commented Jul 3, 2024

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 152. Improved: $\large\color{#35bf28}28$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_plain_set_nested 25.7010μs 11.7502μs 85.1052 KOps/s 74.9584 KOps/s $\textbf{\color{#35bf28}+13.54\%}$
test_plain_set_stack_nested 0.1889ms 11.8937μs 84.0781 KOps/s 74.0539 KOps/s $\textbf{\color{#35bf28}+13.54\%}$
test_plain_set_nested_inplace 0.1830ms 13.3947μs 74.6563 KOps/s 67.1715 KOps/s $\textbf{\color{#35bf28}+11.14\%}$
test_plain_set_stack_nested_inplace 0.2066ms 13.0789μs 76.4588 KOps/s 67.4551 KOps/s $\textbf{\color{#35bf28}+13.35\%}$
test_items 0.1853ms 4.7069μs 212.4551 KOps/s 210.3413 KOps/s $\color{#35bf28}+1.00\%$
test_items_nested 0.3724ms 0.3419ms 2.9252 KOps/s 2.9342 KOps/s $\color{#d91a1a}-0.31\%$
test_items_nested_locked 0.5377ms 0.3441ms 2.9061 KOps/s 2.9206 KOps/s $\color{#d91a1a}-0.50\%$
test_items_nested_leaf 0.1043ms 82.8022μs 12.0770 KOps/s 12.0684 KOps/s $\color{#35bf28}+0.07\%$
test_items_stack_nested 0.3710ms 0.3484ms 2.8699 KOps/s 2.9041 KOps/s $\color{#d91a1a}-1.18\%$
test_items_stack_nested_leaf 0.1029ms 83.7495μs 11.9404 KOps/s 11.9862 KOps/s $\color{#d91a1a}-0.38\%$
test_items_stack_nested_locked 0.3707ms 0.3445ms 2.9027 KOps/s 2.8843 KOps/s $\color{#35bf28}+0.64\%$
test_keys 20.9900μs 4.3550μs 229.6215 KOps/s 226.2890 KOps/s $\color{#35bf28}+1.47\%$
test_keys_nested 0.1012ms 68.8602μs 14.5222 KOps/s 14.4087 KOps/s $\color{#35bf28}+0.79\%$
test_keys_nested_locked 2.3094ms 75.6855μs 13.2126 KOps/s 13.0479 KOps/s $\color{#35bf28}+1.26\%$
test_keys_nested_leaf 87.0010μs 60.0736μs 16.6463 KOps/s 16.3611 KOps/s $\color{#35bf28}+1.74\%$
test_keys_stack_nested 0.1420ms 69.6207μs 14.3635 KOps/s 14.4846 KOps/s $\color{#d91a1a}-0.84\%$
test_keys_stack_nested_leaf 84.2310μs 59.7430μs 16.7384 KOps/s 16.5159 KOps/s $\color{#35bf28}+1.35\%$
test_keys_stack_nested_locked 96.4010μs 74.8519μs 13.3597 KOps/s 13.5698 KOps/s $\color{#d91a1a}-1.55\%$
test_values 7.7167μs 1.8308μs 546.1971 KOps/s 555.3613 KOps/s $\color{#d91a1a}-1.65\%$
test_values_nested 60.0010μs 35.5628μs 28.1193 KOps/s 28.2355 KOps/s $\color{#d91a1a}-0.41\%$
test_values_nested_locked 0.1499ms 37.4006μs 26.7375 KOps/s 26.6918 KOps/s $\color{#35bf28}+0.17\%$
test_values_nested_leaf 0.1292ms 31.8294μs 31.4175 KOps/s 31.7649 KOps/s $\color{#d91a1a}-1.09\%$
test_values_stack_nested 56.6110μs 36.0256μs 27.7580 KOps/s 27.7431 KOps/s $\color{#35bf28}+0.05\%$
test_values_stack_nested_leaf 63.4910μs 31.9247μs 31.3237 KOps/s 31.4255 KOps/s $\color{#d91a1a}-0.32\%$
test_values_stack_nested_locked 62.5110μs 37.9728μs 26.3347 KOps/s 26.3096 KOps/s $\color{#35bf28}+0.10\%$
test_membership 3.5514μs 0.7147μs 1.3992 MOps/s 1.4355 MOps/s $\color{#d91a1a}-2.52\%$
test_membership_nested 30.9310μs 2.5581μs 390.9203 KOps/s 389.8344 KOps/s $\color{#35bf28}+0.28\%$
test_membership_nested_leaf 19.4400μs 2.5338μs 394.6683 KOps/s 391.5232 KOps/s $\color{#35bf28}+0.80\%$
test_membership_stacked_nested 22.3810μs 2.5428μs 393.2628 KOps/s 391.6101 KOps/s $\color{#35bf28}+0.42\%$
test_membership_stacked_nested_leaf 24.6200μs 2.5109μs 398.2676 KOps/s 386.8976 KOps/s $\color{#35bf28}+2.94\%$
test_membership_nested_last 35.1410μs 3.0991μs 322.6711 KOps/s 323.5642 KOps/s $\color{#d91a1a}-0.28\%$
test_membership_nested_leaf_last 22.0900μs 3.1070μs 321.8533 KOps/s 320.5407 KOps/s $\color{#35bf28}+0.41\%$
test_membership_stacked_nested_last 28.4110μs 3.0745μs 325.2532 KOps/s 102.9329 KOps/s $\textbf{\color{#35bf28}+215.99\%}$
test_membership_stacked_nested_leaf_last 22.4110μs 3.0480μs 328.0825 KOps/s 103.1358 KOps/s $\textbf{\color{#35bf28}+218.11\%}$
test_nested_getleaf 0.1547ms 8.4076μs 118.9402 KOps/s 118.5798 KOps/s $\color{#35bf28}+0.30\%$
test_nested_get 30.3700μs 7.8636μs 127.1687 KOps/s 126.9339 KOps/s $\color{#35bf28}+0.18\%$
test_stacked_getleaf 27.4010μs 8.2739μs 120.8619 KOps/s 118.5892 KOps/s $\color{#35bf28}+1.92\%$
test_stacked_get 37.1010μs 7.7808μs 128.5219 KOps/s 126.9079 KOps/s $\color{#35bf28}+1.27\%$
test_nested_getitemleaf 31.1200μs 8.5587μs 116.8405 KOps/s 116.6342 KOps/s $\color{#35bf28}+0.18\%$
test_nested_getitem 30.3810μs 8.0147μs 124.7703 KOps/s 124.5781 KOps/s $\color{#35bf28}+0.15\%$
test_stacked_getitemleaf 35.8610μs 8.4750μs 117.9937 KOps/s 116.3384 KOps/s $\color{#35bf28}+1.42\%$
test_stacked_getitem 25.4300μs 7.9887μs 125.1762 KOps/s 123.8179 KOps/s $\color{#35bf28}+1.10\%$
test_lock_nested 59.0845ms 0.4052ms 2.4681 KOps/s 2.5093 KOps/s $\color{#d91a1a}-1.64\%$
test_lock_stack_nested 0.3646ms 0.2998ms 3.3353 KOps/s 3.4555 KOps/s $\color{#d91a1a}-3.48\%$
test_unlock_nested 61.8660ms 0.4065ms 2.4600 KOps/s 2.5007 KOps/s $\color{#d91a1a}-1.63\%$
test_unlock_stack_nested 0.3323ms 0.3073ms 3.2537 KOps/s 3.3454 KOps/s $\color{#d91a1a}-2.74\%$
test_flatten_speed 0.3643ms 99.8948μs 10.0105 KOps/s 9.6851 KOps/s $\color{#35bf28}+3.36\%$
test_unflatten_speed 0.4814ms 0.2871ms 3.4827 KOps/s 3.4280 KOps/s $\color{#35bf28}+1.60\%$
test_common_ops 1.0325ms 0.5720ms 1.7483 KOps/s 1.6289 KOps/s $\textbf{\color{#35bf28}+7.33\%}$
test_creation 33.2610μs 1.5926μs 627.8960 KOps/s 618.0526 KOps/s $\color{#35bf28}+1.59\%$
test_creation_empty 22.2310μs 6.7335μs 148.5103 KOps/s 101.8708 KOps/s $\textbf{\color{#35bf28}+45.78\%}$
test_creation_nested_1 68.1410μs 8.3719μs 119.4465 KOps/s 86.5297 KOps/s $\textbf{\color{#35bf28}+38.04\%}$
test_creation_nested_2 0.2051ms 10.6682μs 93.7365 KOps/s 72.2893 KOps/s $\textbf{\color{#35bf28}+29.67\%}$
test_clone 70.2710μs 12.3221μs 81.1548 KOps/s 84.9992 KOps/s $\color{#d91a1a}-4.52\%$
test_getitem[int] 76.0610μs 10.8765μs 91.9417 KOps/s 92.1955 KOps/s $\color{#d91a1a}-0.28\%$
test_getitem[slice_int] 44.8910μs 21.2520μs 47.0544 KOps/s 48.9284 KOps/s $\color{#d91a1a}-3.83\%$
test_getitem[range] 68.2210μs 48.9588μs 20.4253 KOps/s 21.0090 KOps/s $\color{#d91a1a}-2.78\%$
test_getitem[tuple] 0.2013ms 18.9040μs 52.8989 KOps/s 53.1218 KOps/s $\color{#d91a1a}-0.42\%$
test_getitem[list] 0.2295ms 35.9661μs 27.8040 KOps/s 29.2926 KOps/s $\textbf{\color{#d91a1a}-5.08\%}$
test_setitem_dim[int] 42.3200μs 25.8123μs 38.7412 KOps/s 35.1225 KOps/s $\textbf{\color{#35bf28}+10.30\%}$
test_setitem_dim[slice_int] 0.1424ms 47.1479μs 21.2099 KOps/s 19.9490 KOps/s $\textbf{\color{#35bf28}+6.32\%}$
test_setitem_dim[range] 0.1879ms 66.8187μs 14.9659 KOps/s 14.9615 KOps/s $\color{#35bf28}+0.03\%$
test_setitem_dim[tuple] 60.8110μs 40.4667μs 24.7117 KOps/s 23.3686 KOps/s $\textbf{\color{#35bf28}+5.75\%}$
test_setitem 0.1552ms 15.9345μs 62.7571 KOps/s 58.3776 KOps/s $\textbf{\color{#35bf28}+7.50\%}$
test_set 0.1295ms 15.3763μs 65.0351 KOps/s 60.7154 KOps/s $\textbf{\color{#35bf28}+7.11\%}$
test_set_shared 1.6308ms 0.1047ms 9.5504 KOps/s 9.9221 KOps/s $\color{#d91a1a}-3.75\%$
test_update 63.9310μs 17.3779μs 57.5444 KOps/s 49.9241 KOps/s $\textbf{\color{#35bf28}+15.26\%}$
test_update_nested 49.7310μs 22.7739μs 43.9099 KOps/s 39.2939 KOps/s $\textbf{\color{#35bf28}+11.75\%}$
test_update__nested 50.5510μs 23.2646μs 42.9837 KOps/s 44.0035 KOps/s $\color{#d91a1a}-2.32\%$
test_set_nested 57.5900μs 16.8927μs 59.1972 KOps/s 55.9658 KOps/s $\textbf{\color{#35bf28}+5.77\%}$
test_set_nested_new 50.1310μs 19.0604μs 52.4647 KOps/s 48.8113 KOps/s $\textbf{\color{#35bf28}+7.48\%}$
test_select 86.9810μs 31.8152μs 31.4315 KOps/s 30.9660 KOps/s $\color{#35bf28}+1.50\%$
test_select_nested 1.0367ms 52.4503μs 19.0657 KOps/s 19.0609 KOps/s $\color{#35bf28}+0.02\%$
test_exclude_nested 0.1359ms 0.1090ms 9.1776 KOps/s 9.3859 KOps/s $\color{#d91a1a}-2.22\%$
test_empty[True] 0.4189ms 0.3457ms 2.8927 KOps/s 2.9305 KOps/s $\color{#d91a1a}-1.29\%$
test_empty[False] 2.8961μs 0.7880μs 1.2691 MOps/s 1.2368 MOps/s $\color{#35bf28}+2.61\%$
test_to 89.8010μs 60.9259μs 16.4134 KOps/s 16.5711 KOps/s $\color{#d91a1a}-0.95\%$
test_to_nonblocking 0.1798ms 37.1036μs 26.9516 KOps/s 26.5466 KOps/s $\color{#35bf28}+1.53\%$
test_unbind_speed 0.3204ms 0.2625ms 3.8102 KOps/s 3.8637 KOps/s $\color{#d91a1a}-1.39\%$
test_unbind_speed_stack0 0.2946ms 0.2619ms 3.8187 KOps/s 3.9217 KOps/s $\color{#d91a1a}-2.63\%$
test_unbind_speed_stack1 78.1013ms 0.7944ms 1.2587 KOps/s 1.2867 KOps/s $\color{#d91a1a}-2.17\%$
test_split 76.8720ms 1.7600ms 568.1749 Ops/s 587.4522 Ops/s $\color{#d91a1a}-3.28\%$
test_chunk 1.7154ms 1.6366ms 611.0379 Ops/s 589.1946 Ops/s $\color{#35bf28}+3.71\%$
test_creation[device0] 0.2064ms 59.8687μs 16.7032 KOps/s 17.4182 KOps/s $\color{#d91a1a}-4.11\%$
test_creation_from_tensor 0.1867ms 55.2633μs 18.0952 KOps/s 18.6368 KOps/s $\color{#d91a1a}-2.91\%$
test_add_one[memmap_tensor0] 0.1072ms 7.8029μs 128.1583 KOps/s 137.6867 KOps/s $\textbf{\color{#d91a1a}-6.92\%}$
test_contiguous[memmap_tensor0] 14.0700μs 0.6721μs 1.4878 MOps/s 1.4719 MOps/s $\color{#35bf28}+1.08\%$
test_stack[memmap_tensor0] 33.6500μs 5.6410μs 177.2745 KOps/s 193.6111 KOps/s $\textbf{\color{#d91a1a}-8.44\%}$
test_memmaptd_index 0.5078ms 0.2963ms 3.3745 KOps/s 3.5508 KOps/s $\color{#d91a1a}-4.96\%$
test_memmaptd_index_astensor 0.6178ms 0.3562ms 2.8072 KOps/s 2.9218 KOps/s $\color{#d91a1a}-3.92\%$
test_memmaptd_index_op 0.9253ms 0.6467ms 1.5464 KOps/s 1.4984 KOps/s $\color{#35bf28}+3.20\%$
test_serialize_model 94.1926ms 90.5917ms 11.0385 Ops/s 10.4837 Ops/s $\textbf{\color{#35bf28}+5.29\%}$
test_serialize_model_pickle 1.3533s 1.2361s 0.8090 Ops/s 0.8062 Ops/s $\color{#35bf28}+0.34\%$
test_serialize_weights 0.1717s 98.1417ms 10.1893 Ops/s 9.5528 Ops/s $\textbf{\color{#35bf28}+6.66\%}$
test_serialize_weights_returnearly 0.2633s 76.9686ms 12.9923 Ops/s 13.1268 Ops/s $\color{#d91a1a}-1.02\%$
test_serialize_weights_pickle 1.3493s 1.2366s 0.8087 Ops/s 0.8083 Ops/s $\color{#35bf28}+0.05\%$
test_reshape_pytree 0.1460ms 27.2805μs 36.6562 KOps/s 38.2958 KOps/s $\color{#d91a1a}-4.28\%$
test_reshape_td 0.2232ms 32.2596μs 30.9985 KOps/s 31.9868 KOps/s $\color{#d91a1a}-3.09\%$
test_view_pytree 0.1359ms 26.8584μs 37.2324 KOps/s 38.6598 KOps/s $\color{#d91a1a}-3.69\%$
test_view_td 0.2303ms 37.5398μs 26.6384 KOps/s 28.1875 KOps/s $\textbf{\color{#d91a1a}-5.50\%}$
test_unbind_pytree 0.1566ms 33.1735μs 30.1445 KOps/s 30.9746 KOps/s $\color{#d91a1a}-2.68\%$
test_unbind_td 0.4918ms 41.8388μs 23.9013 KOps/s 24.7329 KOps/s $\color{#d91a1a}-3.36\%$
test_split_pytree 0.1723ms 36.3142μs 27.5374 KOps/s 28.2701 KOps/s $\color{#d91a1a}-2.59\%$
test_split_td 0.1496ms 40.8772μs 24.4635 KOps/s 25.5374 KOps/s $\color{#d91a1a}-4.21\%$
test_add_pytree 0.2642ms 40.0501μs 24.9687 KOps/s 25.5888 KOps/s $\color{#d91a1a}-2.42\%$
test_add_td 0.1853ms 51.2792μs 19.5011 KOps/s 18.6952 KOps/s $\color{#35bf28}+4.31\%$
test_distributed 5.0399ms 0.1053ms 9.4942 KOps/s 9.6622 KOps/s $\color{#d91a1a}-1.74\%$
test_tdmodule 0.1665ms 14.7777μs 67.6696 KOps/s 60.9513 KOps/s $\textbf{\color{#35bf28}+11.02\%}$
test_tdmodule_dispatch 45.0510μs 28.0107μs 35.7006 KOps/s 31.5596 KOps/s $\textbf{\color{#35bf28}+13.12\%}$
test_tdseq 31.9410μs 15.8715μs 63.0061 KOps/s 56.9760 KOps/s $\textbf{\color{#35bf28}+10.58\%}$
test_tdseq_dispatch 47.3400μs 30.7849μs 32.4835 KOps/s 29.6733 KOps/s $\textbf{\color{#35bf28}+9.47\%}$
test_instantiation_functorch 1.5524ms 1.4349ms 696.9234 Ops/s 722.5857 Ops/s $\color{#d91a1a}-3.55\%$
test_instantiation_td 82.5710ms 1.0893ms 918.0379 Ops/s 930.2008 Ops/s $\color{#d91a1a}-1.31\%$
test_exec_functorch 0.2700ms 0.1526ms 6.5515 KOps/s 6.6402 KOps/s $\color{#d91a1a}-1.34\%$
test_exec_functional_call 0.2052ms 0.1449ms 6.9002 KOps/s 7.1190 KOps/s $\color{#d91a1a}-3.07\%$
test_exec_td 0.2571ms 0.1451ms 6.8909 KOps/s 7.2465 KOps/s $\color{#d91a1a}-4.91\%$
test_exec_td_decorator 0.9639ms 0.2147ms 4.6586 KOps/s 4.6889 KOps/s $\color{#d91a1a}-0.65\%$
test_vmap_mlp_speed[True-True] 1.2863ms 0.5905ms 1.6936 KOps/s 1.6847 KOps/s $\color{#35bf28}+0.53\%$
test_vmap_mlp_speed[True-False] 0.7476ms 0.5882ms 1.7002 KOps/s 1.6846 KOps/s $\color{#35bf28}+0.93\%$
test_vmap_mlp_speed[False-True] 0.6767ms 0.5213ms 1.9184 KOps/s 1.9165 KOps/s $\color{#35bf28}+0.10\%$
test_vmap_mlp_speed[False-False] 0.6642ms 0.5194ms 1.9251 KOps/s 1.9164 KOps/s $\color{#35bf28}+0.45\%$
test_vmap_mlp_speed_decorator[True-True] 0.9982ms 0.6500ms 1.5386 KOps/s 1.5330 KOps/s $\color{#35bf28}+0.36\%$
test_vmap_mlp_speed_decorator[True-False] 0.8411ms 0.6516ms 1.5347 KOps/s 1.5300 KOps/s $\color{#35bf28}+0.31\%$
test_vmap_mlp_speed_decorator[False-True] 0.7541ms 0.5799ms 1.7244 KOps/s 1.7301 KOps/s $\color{#d91a1a}-0.33\%$
test_vmap_mlp_speed_decorator[False-False] 0.7319ms 0.5786ms 1.7282 KOps/s 1.7339 KOps/s $\color{#d91a1a}-0.33\%$
test_vmap_transformer_speed[True-True] 8.0810ms 7.8375ms 127.5920 Ops/s 128.6688 Ops/s $\color{#d91a1a}-0.84\%$
test_vmap_transformer_speed[True-False] 8.0924ms 7.8362ms 127.6128 Ops/s 129.2141 Ops/s $\color{#d91a1a}-1.24\%$
test_vmap_transformer_speed[False-True] 8.1286ms 7.8093ms 128.0519 Ops/s 130.0169 Ops/s $\color{#d91a1a}-1.51\%$
test_vmap_transformer_speed[False-False] 8.4069ms 7.8208ms 127.8636 Ops/s 129.9877 Ops/s $\color{#d91a1a}-1.63\%$
test_vmap_transformer_speed_decorator[True-True] 19.8368ms 19.1352ms 52.2597 Ops/s 53.0487 Ops/s $\color{#d91a1a}-1.49\%$
test_vmap_transformer_speed_decorator[True-False] 19.4331ms 19.0083ms 52.6086 Ops/s 53.0511 Ops/s $\color{#d91a1a}-0.83\%$
test_vmap_transformer_speed_decorator[False-True] 19.4891ms 18.9901ms 52.6589 Ops/s 53.2877 Ops/s $\color{#d91a1a}-1.18\%$
test_vmap_transformer_speed_decorator[False-False] 19.8298ms 18.9951ms 52.6451 Ops/s 53.4078 Ops/s $\color{#d91a1a}-1.43\%$
test_to_module_speed[True] 1.6160ms 1.5030ms 665.3141 Ops/s 665.5345 Ops/s $\color{#d91a1a}-0.03\%$
test_to_module_speed[False] 1.6569ms 1.4903ms 671.0243 Ops/s 670.9475 Ops/s $\color{#35bf28}+0.01\%$
test_tc_init 0.2149ms 46.2273μs 21.6322 KOps/s 18.1214 KOps/s $\textbf{\color{#35bf28}+19.37\%}$
test_tc_init_nested 0.1307ms 92.4960μs 10.8113 KOps/s 8.8880 KOps/s $\textbf{\color{#35bf28}+21.64\%}$
test_tc_first_layer_tensor 0.1889ms 3.7164μs 269.0774 KOps/s 266.4142 KOps/s $\color{#35bf28}+1.00\%$
test_tc_first_layer_nontensor 0.1810ms 3.7321μs 267.9471 KOps/s 268.3066 KOps/s $\color{#d91a1a}-0.13\%$
test_tc_second_layer_tensor 34.3566μs 1.1781μs 848.8483 KOps/s 841.4560 KOps/s $\color{#35bf28}+0.88\%$
test_tc_second_layer_nontensor 18.1210μs 4.2703μs 234.1744 KOps/s 234.5629 KOps/s $\color{#d91a1a}-0.17\%$
test_unbind 0.1135s 13.9259ms 71.8086 Ops/s 67.3155 Ops/s $\textbf{\color{#35bf28}+6.67\%}$
test_full_like 14.4843ms 14.0906ms 70.9693 Ops/s 97.6036 Ops/s $\textbf{\color{#d91a1a}-27.29\%}$
test_zeros_like 8.7411ms 8.0466ms 124.2761 Ops/s 124.3620 Ops/s $\color{#d91a1a}-0.07\%$
test_ones_like 8.5543ms 8.0215ms 124.6649 Ops/s 125.5973 Ops/s $\color{#d91a1a}-0.74\%$
test_clone 10.4796ms 9.8154ms 101.8811 Ops/s 101.6722 Ops/s $\color{#35bf28}+0.21\%$
test_squeeze 0.1550ms 10.6603μs 93.8062 KOps/s 94.1241 KOps/s $\color{#d91a1a}-0.34\%$
test_unsqueeze 0.2318ms 88.0061μs 11.3629 KOps/s 11.6099 KOps/s $\color{#d91a1a}-2.13\%$
test_split 3.5018ms 3.1027ms 322.3016 Ops/s 324.7011 Ops/s $\color{#d91a1a}-0.74\%$
test_permute 0.3811ms 0.2121ms 4.7141 KOps/s 4.9883 KOps/s $\textbf{\color{#d91a1a}-5.50\%}$
test_stack 29.0132ms 28.3501ms 35.2733 Ops/s 35.2254 Ops/s $\color{#35bf28}+0.14\%$
test_cat 28.7781ms 28.0900ms 35.5999 Ops/s 35.4245 Ops/s $\color{#35bf28}+0.50\%$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants