feat: revert values canonicalization in DictArray::into_canonical #1136
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
lwwmanning
force-pushed
the
wm/revert-1032
branch
from
1e9ad30 to
dc5ee60
Compare
robert3005
approved these changes
Oct 24, 2024
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Vortex bytes_at
| Benchmark suite | Current: dc5ee60 | Previous: cdca5a1 | Ratio |
|---|---|---|---|
bytes_at/array_data |
696.8691096128895 ns (1.207205202780699) |
684.8279012637903 ns (1.0315541139266884) |
1.02 |
bytes_at/array_view |
485.4250572513177 ns (0.7008422291318368) |
484.5102035206452 ns (0.9280072041338769) |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
DataFusion
| Benchmark suite | Current: dc5ee60 | Previous: cdca5a1 | Ratio |
|---|---|---|---|
arrow/planning |
769969.3986044915 ns (1599.48065906571) |
768289.6938433044 ns (1475.5945770438993) |
1.00 |
arrow/exec |
1367595.1574665648 ns (4141.610691752983) |
1378742.1047856482 ns (4270.17305936513) |
0.99 |
vortex-pushdown-compressed/planning |
480058.21031606494 ns (1172.0975926326646) |
482360.2294460883 ns (1083.6370342389855) |
1.00 |
vortex-pushdown-compressed/exec |
2531786.355500001 ns (14744.088624999626) |
2533093.113 ns (10152.724156250479) |
1.00 |
vortex-pushdown-uncompressed/planning |
479370.78210816317 ns (2091.878095304448) |
481006.8760063284 ns (775.5766875155387) |
1.00 |
vortex-pushdown-uncompressed/exec |
2536794.4835 ns (5424.692149999784) |
2521936.201999999 ns (3624.2447812505998) |
1.01 |
vortex-nopushdown-compressed/planning |
787777.0896307443 ns (1239.5633070385666) |
788290.7238285992 ns (1488.6235915662837) |
1.00 |
vortex-nopushdown-compressed/exec |
21189305.40666666 ns (85691.97116667032) |
2941077.257058823 ns (15470.509448528988) |
7.20 |
vortex-nopushdown-uncompressed/planning |
783418.903985849 ns (1691.5004662951105) |
781975.1051584266 ns (2559.6528806487913) |
1.00 |
vortex-nopushdown-uncompressed/exec |
4659409.897272727 ns (16944.014795454685) |
4525033.832727274 ns (11895.460397727322) |
1.03 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Random Access
| Benchmark suite | Current: dc5ee60 | Previous: cdca5a1 | Ratio |
|---|---|---|---|
random-access/vortex-tokio-local-disk |
982033.5000857784 ns (5103.199916605081) |
1022944.670390242 ns (10480.614214453264) |
0.96 |
random-access/vortex-local-fs |
1106716.4449551846 ns (6155.407298519858) |
1134650.3734481344 ns (8034.168956903508) |
0.98 |
random-access/parquet-tokio-local-disk |
223698888.4333333 ns (3572325.095416665) |
213621462.8 ns (3699856.649999991) |
1.05 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
TPC-H
| Benchmark suite | Current: dc5ee60 | Previous: cdca5a1 | Ratio |
|---|---|---|---|
tpch_q1/vortex-in-memory-no-pushdown |
577088160.1 ns (5651115.600000024) |
558547761.1 ns (1916794.5) |
1.03 |
tpch_q1/vortex-in-memory-pushdown |
452162066.5 ns (1468160.5750000179) |
448583382.65 ns (1644336.150000006) |
1.01 |
tpch_q1/arrow |
549388873.3 ns (2677158.9249999523) |
542809996.1 ns (2025133.4787499905) |
1.01 |
tpch_q1/parquet |
689720334.9 ns (3058823.8174999356) |
684277317.8 ns (1597712.375) |
1.01 |
tpch_q1/vortex-file-compressed |
518999473.4 ns (2129128.379999995) |
498396195.2 ns (2095120.324999988) |
1.04 |
tpch_q1/vortex-file-uncompressed |
527486618.8 ns (1794529.050000012) |
524469169.1 ns (2398443.107499987) |
1.01 |
tpch_q2/vortex-in-memory-no-pushdown |
121582667.62908728 ns (538178.7529563457) |
119729604.97305557 ns (610706.5944895819) |
1.02 |
tpch_q2/vortex-in-memory-pushdown |
119931613.9651984 ns (793033.6975838393) |
117538386.16309524 ns (550675.2704761848) |
1.02 |
tpch_q2/arrow |
119274571.46793649 ns (459227.48207936436) |
116971473.86940476 ns (623538.829583332) |
1.02 |
tpch_q2/parquet |
146290474.57309526 ns (597121.6890624762) |
147874845.77071428 ns (369296.0402381122) |
0.99 |
tpch_q2/vortex-file-compressed |
162773428.36718255 ns (1318676.2337832302) |
164063951.77285713 ns (1295497.094520837) |
0.99 |
tpch_q2/vortex-file-uncompressed |
167109079.27305558 ns (2076781.79028818) |
164729834.58313492 ns (2132678.0514007956) |
1.01 |
tpch_q3/vortex-in-memory-no-pushdown |
161105548.9284127 ns (1456906.8338988125) |
159634078.66599208 ns (1208235.158128962) |
1.01 |
tpch_q3/vortex-in-memory-pushdown |
176153444.58896825 ns (893582.2674196362) |
176598296.61511904 ns (1102440.9832857251) |
1.00 |
tpch_q3/arrow |
148726723.25611112 ns (1340330.118874982) |
147478399.3994841 ns (746695.2887499928) |
1.01 |
tpch_q3/parquet |
335941437.15 ns (2443357.3474999964) |
335213450.65 ns (1572518.162499994) |
1.00 |
tpch_q3/vortex-file-compressed |
339162788.8 ns (1784196.6499999762) |
300458814.45 ns (1426631.0387500226) |
1.13 |
tpch_q3/vortex-file-uncompressed |
242879028.5333333 ns (1436643.3549999893) |
245443829.26666665 ns (1497953.1412499845) |
0.99 |
tpch_q4/vortex-in-memory-no-pushdown |
112394441.48626983 ns (1210961.6978571415) |
111719693.56428573 ns (583636.2959553525) |
1.01 |
tpch_q4/vortex-in-memory-pushdown |
129129934.23301585 ns (537797.327920638) |
128281000.91285715 ns (574747.9188898876) |
1.01 |
tpch_q4/arrow |
187013108.6 ns (1986552.4662500173) |
181970067.7930159 ns (758061.3807142824) |
1.03 |
tpch_q4/parquet |
203953684.8 ns (662354.0741667002) |
203643677.39999998 ns (777953.5066666752) |
1.00 |
tpch_q4/vortex-file-compressed |
301622993.75 ns (2739725.811250001) |
250315001.36666664 ns (1379349.0166666508) |
1.20 |
tpch_q4/vortex-file-uncompressed |
206253964.19999996 ns (1203483.649999991) |
200927358.1666667 ns (1148126.4537500292) |
1.03 |
tpch_q5/vortex-in-memory-no-pushdown |
292715402.75 ns (3874335.5081249774) |
282912943.35 ns (2193523.763749987) |
1.03 |
tpch_q5/vortex-in-memory-pushdown |
288191889.15 ns (1839362.4806249738) |
287773815.4 ns (947238.599999994) |
1.00 |
tpch_q5/arrow |
272616172.6 ns (2108229.2743749917) |
271625319.25 ns (2046447.7924999893) |
1.00 |
tpch_q5/parquet |
447401275.2 ns (2393957.5568750203) |
442952755.95 ns (1505444.9300000072) |
1.01 |
tpch_q5/vortex-file-compressed |
365960294.15 ns (3941862.3349999785) |
359033565.35 ns (3281901.8125) |
1.02 |
tpch_q5/vortex-file-uncompressed |
349121136.3 ns (3371316.215624988) |
336796368.55 ns (3109387) |
1.04 |
tpch_q6/vortex-in-memory-no-pushdown |
36598940.24366402 ns (255925.6400529109) |
35403231.30096561 ns (187212.82640872896) |
1.03 |
tpch_q6/vortex-in-memory-pushdown |
72914140.5783135 ns (105136.13438491523) |
72448514.75271825 ns (171196.97742187977) |
1.01 |
tpch_q6/arrow |
27408159.152003966 ns (250505.24419023097) |
25517109.84373016 ns (101257.55906646885) |
1.07 |
tpch_q6/parquet |
138165468.95464283 ns (324047.9062872231) |
136585986.39531747 ns (675368.9949176759) |
1.01 |
tpch_q6/vortex-file-compressed |
71068067.28839286 ns (470828.1082693413) |
70498369.54123016 ns (266274.89431995153) |
1.01 |
tpch_q6/vortex-file-uncompressed |
173575524.8572619 ns (2017180.0835074335) |
169436729.41968256 ns (780857.4813293666) |
1.02 |
tpch_q7/vortex-in-memory-no-pushdown |
553117504 ns (5402986) |
545699843.6 ns (3647324.649999976) |
1.01 |
tpch_q7/vortex-in-memory-pushdown |
577492220.4 ns (3767926.0999999642) |
578860543.7 ns (2964261.1737499833) |
1.00 |
tpch_q7/arrow |
531600146.8 ns (4197936.492500007) |
538839417.6 ns (2581787.477499962) |
0.99 |
tpch_q7/parquet |
684473867.7 ns (10883883.900000036) |
663515880.2 ns (5792517.173749983) |
1.03 |
tpch_q7/vortex-file-compressed |
720733750.3 ns (6230488.436250031) |
683445589.5 ns (4350964.107500017) |
1.05 |
tpch_q7/vortex-file-uncompressed |
656590785.5 ns (3322715.7475000024) |
659949209.3 ns (3904099.4000000358) |
0.99 |
tpch_q8/vortex-in-memory-no-pushdown |
226801376.5333333 ns (1454577.7379166484) |
221053629.33333334 ns (1536243.6162500083) |
1.03 |
tpch_q8/vortex-in-memory-pushdown |
227043148.0666667 ns (1473866.8333333284) |
223721986.06666666 ns (1589018.686250016) |
1.01 |
tpch_q8/arrow |
209870269.90000004 ns (2164886.619999975) |
206879769.06666666 ns (1292087.90291664) |
1.01 |
tpch_q8/parquet |
482858690.9 ns (3200387.5556249917) |
480949265.85 ns (1849662.7000000179) |
1.00 |
tpch_q8/vortex-file-compressed |
310181712 ns (2729310.4412499964) |
311870932.45 ns (2594202.805000037) |
0.99 |
tpch_q8/vortex-file-uncompressed |
300671639 ns (3323704.7650000155) |
293710877.8 ns (2654047.025000006) |
1.02 |
tpch_q9/vortex-in-memory-no-pushdown |
426449136.6 ns (4878934.824374944) |
413108295.75 ns (2173145.846875012) |
1.03 |
tpch_q9/vortex-in-memory-pushdown |
435051097.95 ns (4588295.949999988) |
413658075.15 ns (3141268.2931250036) |
1.05 |
tpch_q9/arrow |
410059555.4 ns (5412919.739375025) |
401731686.85 ns (4034189.411250025) |
1.02 |
tpch_q9/parquet |
716531287.7 ns (6505567.807500005) |
693276951.9 ns (2648482.982499957) |
1.03 |
tpch_q9/vortex-file-compressed |
523472696.2 ns (9037692.66625002) |
489337983.05 ns (4189544.424999982) |
1.07 |
tpch_q9/vortex-file-uncompressed |
491687661.7 ns (4734792.933750033) |
478147006.95 ns (2773769.75) |
1.03 |
tpch_q10/vortex-in-memory-no-pushdown |
287837423.15 ns (3675090.974999994) |
275151668.8 ns (1793868.349999994) |
1.05 |
tpch_q10/vortex-in-memory-pushdown |
314457987.9 ns (3491659.587500006) |
300103923.45 ns (1802587.7618749738) |
1.05 |
tpch_q10/arrow |
274039020.55 ns (2612320.1193750203) |
262516722.4 ns (1888054.75) |
1.04 |
tpch_q10/parquet |
511014318.4 ns (2550231.6912499964) |
495892789.6 ns (2465632.2912499905) |
1.03 |
tpch_q10/vortex-file-compressed |
450322434.05 ns (3635480.125) |
407701846.65 ns (1909768.8174999952) |
1.10 |
tpch_q10/vortex-file-uncompressed |
413260318.4 ns (5108314.821875006) |
387182283 ns (2610672.8462499976) |
1.07 |
tpch_q11/vortex-in-memory-no-pushdown |
188902777.5 ns (2442789.8458333164) |
174944113.72142857 ns (763780.3610714376) |
1.08 |
tpch_q11/vortex-in-memory-pushdown |
187310160.2 ns (2839840.898750022) |
176463818.6425 ns (641817.5145416558) |
1.06 |
tpch_q11/arrow |
191543708.96666664 ns (4204456.023749977) |
173408682.81821427 ns (752932.1819211394) |
1.10 |
tpch_q11/parquet |
203709203.93333334 ns (1555235.3729166389) |
183367624.89999998 ns (1322897.7575000077) |
1.11 |
tpch_q11/vortex-file-compressed |
281365329.5 ns (4646715.720625013) |
262780958.8 ns (3081221.0618750006) |
1.07 |
tpch_q11/vortex-file-uncompressed |
285087572.9 ns (5333994.518749952) |
264239076.8 ns (1935711.625) |
1.08 |
tpch_q12/vortex-in-memory-no-pushdown |
235069096.56666666 ns (923610.9479166567) |
231758758.83333334 ns (631941.3604166657) |
1.01 |
tpch_q12/vortex-in-memory-pushdown |
252480133.7 ns (2640308.25) |
249911239.7333333 ns (869934.2575000226) |
1.01 |
tpch_q12/arrow |
189016431.8333333 ns (629536.362499997) |
187440785.86666667 ns (658331.0529166609) |
1.01 |
tpch_q12/parquet |
337002699 ns (1142444.4993749857) |
334298888.1 ns (1134279.8668749928) |
1.01 |
tpch_q12/vortex-file-compressed |
425097978.65 ns (3680728.1443749666) |
397729139.55 ns (1371837.609375) |
1.07 |
tpch_q12/vortex-file-uncompressed |
405561806.8 ns (3078030.815624982) |
384608200.85 ns (1294052.5381249785) |
1.05 |
tpch_q13/vortex-in-memory-no-pushdown |
213616857.3 ns (3489581.940416634) |
169439831.31079367 ns (2112545.1600575596) |
1.26 |
tpch_q13/vortex-in-memory-pushdown |
207007625.40000004 ns (1745285.4512500018) |
170112342.59464285 ns (2168703.270276785) |
1.22 |
tpch_q13/arrow |
208301125.5 ns (2807218.732916653) |
166416451.14285713 ns (1925728.7775000036) |
1.25 |
tpch_q13/parquet |
366744035.65 ns (3558764.476875007) |
325256711.05 ns (2198405.653124988) |
1.13 |
tpch_q13/vortex-file-compressed |
222615268.13333336 ns (5423533.283333331) |
201675687.2 ns (2176601.361250028) |
1.10 |
tpch_q13/vortex-file-uncompressed |
225366582.56666666 ns (3031179.326249987) |
201835942 ns (1660681.950000003) |
1.12 |
tpch_q14/vortex-in-memory-no-pushdown |
49812931.900912695 ns (484706.05460020155) |
43887173.4387963 ns (324864.52463541925) |
1.14 |
tpch_q14/vortex-in-memory-pushdown |
82701789.61686508 ns (608346.18841023) |
73276597.92144841 ns (388765.34738987684) |
1.13 |
tpch_q14/arrow |
41266553.912645504 ns (489020.67802381143) |
35113040.68404762 ns (653133.3116140924) |
1.18 |
tpch_q14/parquet |
232689200.0666667 ns (1352150.1737499833) |
225551514.4333333 ns (603837.0625000447) |
1.03 |
tpch_q14/vortex-file-compressed |
123663166.35011904 ns (1462415.4838095307) |
116617977.02630952 ns (210601.0765119046) |
1.06 |
tpch_q14/vortex-file-uncompressed |
144428590.50015873 ns (1951127.2338224202) |
133993996.15476191 ns (872365.1547618955) |
1.08 |
tpch_q15/vortex-in-memory-no-pushdown |
78751641.5647619 ns (650051.3301919699) |
71570396.02634922 ns (461179.1139563471) |
1.10 |
tpch_q15/vortex-in-memory-pushdown |
107291935.96412697 ns (694968.1754375026) |
102473544.53380951 ns (556684.5332857072) |
1.05 |
tpch_q15/arrow |
63981410.108551584 ns (527750.2205133885) |
56905462.830436505 ns (457401.3526160717) |
1.12 |
tpch_q15/parquet |
308166260.8 ns (1952299.3787500262) |
297928781.1 ns (1034731.2437499762) |
1.03 |
tpch_q15/vortex-file-compressed |
231221457.9333333 ns (1755689.1258333176) |
217451808.0333333 ns (1062503.4066666812) |
1.06 |
tpch_q15/vortex-file-uncompressed |
258916210.85 ns (2012441.349999994) |
250391761.9 ns (1651111.8918749988) |
1.03 |
tpch_q16/vortex-in-memory-no-pushdown |
107754205.15527777 ns (420986.58478818834) |
105933959.80571428 ns (380248.6054761857) |
1.02 |
tpch_q16/vortex-in-memory-pushdown |
124086881.29214284 ns (1206499.3948303685) |
116592053.82916668 ns (418749.83865626156) |
1.06 |
tpch_q16/arrow |
112561123.2829762 ns (1178456.2639181614) |
104697016.39218254 ns (292616.4130446315) |
1.08 |
tpch_q16/parquet |
120321886.67805555 ns (1146890.2965277806) |
115995840.50412698 ns (377442.5152004063) |
1.04 |
tpch_q16/vortex-file-compressed |
136847304.54599208 ns (353169.60202381015) |
131886101.11730158 ns (500552.78572619706) |
1.04 |
tpch_q16/vortex-file-uncompressed |
138570215.20190474 ns (1740415.6099999994) |
132413149.96273811 ns (774328.8651190624) |
1.05 |
tpch_q17/vortex-in-memory-no-pushdown |
645006588.9 ns (20838804.25) |
523846626.4 ns (7336525.967499971) |
1.23 |
tpch_q17/vortex-in-memory-pushdown |
694261191.3 ns (21905057.722500026) |
605784856.1 ns (9834978.829999983) |
1.15 |
tpch_q17/arrow |
573716543.5 ns (14829598.449999988) |
507884479.9 ns (4613469.899999976) |
1.13 |
tpch_q17/parquet |
657645777.3 ns (5622019.350000024) |
638768677.9 ns (2973126.980000019) |
1.03 |
tpch_q17/vortex-file-compressed |
644951399.3 ns (4896925.550000012) |
599432759.6 ns (6390107.51000005) |
1.08 |
tpch_q17/vortex-file-uncompressed |
622038346.2 ns (3653429.192499995) |
609740539.7 ns (4534640.149999976) |
1.02 |
tpch_q18/vortex-in-memory-no-pushdown |
1161418297.2 ns (29108584.981250048) |
1057064908.8 ns (7833619.824999988) |
1.10 |
tpch_q18/vortex-in-memory-pushdown |
1145297675.1 ns (18394459.100000024) |
1050923226.1 ns (9397957.368749976) |
1.09 |
tpch_q18/arrow |
1140032508.3 ns (23738213.33375001) |
1036912626.2 ns (7798027.358749986) |
1.10 |
tpch_q18/parquet |
1307631043.7 ns (17732328.283749938) |
1204252553.1 ns (8849486.814999938) |
1.09 |
tpch_q18/vortex-file-compressed |
1341710226.3 ns (31826083.793750048) |
1107416239.8 ns (4918652.836249948) |
1.21 |
tpch_q18/vortex-file-uncompressed |
1179599022.8 ns (20261980.18750012) |
1092995455.9 ns (7738086.695000052) |
1.08 |
tpch_q19/vortex-in-memory-no-pushdown |
186630739.6666667 ns (1159015.8416666538) |
180965161.49511904 ns (250989.20902974904) |
1.03 |
tpch_q19/vortex-in-memory-pushdown |
270574276.05 ns (1105237.8012500107) |
258851029.6 ns (889785.174999997) |
1.05 |
tpch_q19/arrow |
174031478.5275 ns (1020727.5240520835) |
164538343.141746 ns (1610803.0919097364) |
1.06 |
tpch_q19/parquet |
467279064.95 ns (1544647.875) |
455033206.75 ns (1352987.8256250024) |
1.03 |
tpch_q19/vortex-file-compressed |
413039560.85 ns (4752651.47437501) |
335688795.2 ns (877241.979374975) |
1.23 |
tpch_q19/vortex-file-uncompressed |
379067168.35 ns (5351605.8125) |
358281261.5 ns (1408339.4693749845) |
1.06 |
tpch_q20/vortex-in-memory-no-pushdown |
283125026.6 ns (4648454.003749996) |
244613368.73333335 ns (1336895.248333335) |
1.16 |
tpch_q20/vortex-in-memory-pushdown |
301373953.15 ns (4528679.120000005) |
255249691.35 ns (2254936.764375001) |
1.18 |
tpch_q20/arrow |
273162664.65 ns (2992891.417500019) |
230937239.4 ns (1987286.4999999702) |
1.18 |
tpch_q20/parquet |
382997786.65 ns (2589725.448124975) |
354432004.85 ns (2491831.5150000155) |
1.08 |
tpch_q20/vortex-file-compressed |
388343522.25 ns (3297998.209375024) |
342628538.8 ns (3058932.725000024) |
1.13 |
tpch_q20/vortex-file-uncompressed |
395210983.2 ns (4809098.96312499) |
361555100.7 ns (2949385) |
1.09 |
tpch_q21/vortex-in-memory-no-pushdown |
1030676799.3 ns (15991373.648750007) |
889146259.6 ns (7515201.071250021) |
1.16 |
tpch_q21/vortex-in-memory-pushdown |
1050565770 ns (11664013.446249962) |
903720415.2 ns (6307749.413749993) |
1.16 |
tpch_q21/arrow |
971861497.5 ns (9057668.722500026) |
846101240.1 ns (6141305.949999988) |
1.15 |
tpch_q21/parquet |
1051643620.8 ns (11187885.88625002) |
964861632 ns (5409379.099999964) |
1.09 |
tpch_q21/vortex-file-compressed |
1422193539.1 ns (18510383.07249999) |
1187158177.9 ns (4996717.450000048) |
1.20 |
tpch_q21/vortex-file-uncompressed |
1242719111.1 ns (22897155) |
1056720453.7 ns (6565211.199999988) |
1.18 |
tpch_q22/vortex-in-memory-no-pushdown |
77995306.71269841 ns (296250.88865078986) |
76263518.63194445 ns (236338.32222221792) |
1.02 |
tpch_q22/vortex-in-memory-pushdown |
78407994.42204365 ns (539283.6289412156) |
75663690.35666667 ns (323499.0405520871) |
1.04 |
tpch_q22/arrow |
78605435.62553571 ns (777494.5012499988) |
74770444.11039683 ns (343721.62354265153) |
1.05 |
tpch_q22/parquet |
103572549.67420635 ns (2894119.540287696) |
92834738.61734128 ns (471204.50950197875) |
1.12 |
tpch_q22/vortex-file-compressed |
134589800.77301586 ns (1879864.5841726288) |
118732654.5898016 ns (326224.0749007985) |
1.13 |
tpch_q22/vortex-file-uncompressed |
133249625.22452381 ns (1954930.583095245) |
117167085.57805558 ns (649966.9448888972) |
1.14 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Vortex Compression
| Benchmark suite | Current: dc5ee60 | Previous: cdca5a1 | Ratio |
|---|---|---|---|
compress time/taxi |
1167323560.8 ns (4275283.600000024) |
1215213778 ns (3925130.1999999285) |
0.96 |
compress time/taxi throughput |
470808924 bytes |
470808924 bytes |
1 |
parquet_rs-zstd compress time/taxi |
1789670570 ns (4289446.3000000715) |
1752838757.7 ns (3415356.850000024) |
1.02 |
parquet_rs-zstd compress time/taxi throughput |
470808924 bytes |
470808924 bytes |
1 |
decompress time/taxi |
432042493.7 ns (2478475.599999994) |
407723166.55 ns (3490634.525000006) |
1.06 |
decompress time/taxi throughput |
470808924 bytes |
470808924 bytes |
1 |
parquet_rs-zstd decompress time/taxi |
310566247.75 ns (1020689.9706249833) |
310705647.9 ns (1171310.599999994) |
1.00 |
parquet_rs-zstd decompress time/taxi throughput |
470808924 bytes |
470808924 bytes |
1 |
vortex:parquet-zstd size/taxi |
0.9405492164963302 ratio |
0.9417958031701114 ratio |
1.00 |
vortex:raw size/taxi |
0.11179405766743708 ratio |
0.11194277192587794 ratio |
1.00 |
vortex size/taxi |
52633640 bytes |
52703656 bytes |
1.00 |
compress time/AirlineSentiment |
691268.5395424481 ns (1521.5725193151738) |
696637.2053117048 ns (1413.661745626654) |
0.99 |
compress time/AirlineSentiment throughput |
2020 bytes |
2020 bytes |
1 |
parquet_rs-zstd compress time/AirlineSentiment |
56273.09439752683 ns (113.14979986805338) |
57258.15805588041 ns (188.33440215680093) |
0.98 |
parquet_rs-zstd compress time/AirlineSentiment throughput |
2020 bytes |
2020 bytes |
1 |
decompress time/AirlineSentiment |
35984.26673837371 ns (71.73415965734966) |
40227.1035708342 ns (134.47932581637724) |
0.89 |
decompress time/AirlineSentiment throughput |
2020 bytes |
2020 bytes |
1 |
parquet_rs-zstd decompress time/AirlineSentiment |
32338.49117689386 ns (91.80322678230914) |
32686.437516453734 ns (82.6045451452901) |
0.99 |
parquet_rs-zstd decompress time/AirlineSentiment throughput |
2020 bytes |
2020 bytes |
1 |
vortex:parquet-zstd size/AirlineSentiment |
6.196483971044468 ratio |
6.196483971044468 ratio |
1 |
vortex:raw size/AirlineSentiment |
2.9663366336633663 ratio |
2.9663366336633663 ratio |
1 |
vortex size/AirlineSentiment |
5992 bytes |
5992 bytes |
1 |
compress time/Arade |
2221559888.3 ns (16208657.463750124) |
2208857219.4 ns (5563886.877500057) |
1.01 |
compress time/Arade throughput |
787023760 bytes |
787023760 bytes |
1 |
parquet_rs-zstd compress time/Arade |
3056093026.4 ns (10972765.16624999) |
3077526247.6 ns (9658345.599999905) |
0.99 |
parquet_rs-zstd compress time/Arade throughput |
787023760 bytes |
787023760 bytes |
1 |
decompress time/Arade |
842731807.1 ns (3253674.741250038) |
747154525.3 ns (12687611.902499974) |
1.13 |
decompress time/Arade throughput |
787023760 bytes |
787023760 bytes |
1 |
parquet_rs-zstd decompress time/Arade |
651290467.4 ns (1976260.423750043) |
649646371.5 ns (2141534.626249969) |
1.00 |
parquet_rs-zstd decompress time/Arade throughput |
787023760 bytes |
787023760 bytes |
1 |
vortex:parquet-zstd size/Arade |
0.47890649124129325 ratio |
0.4789069103731621 ratio |
1.00 |
vortex:raw size/Arade |
0.18583271234403392 ratio |
0.1858328749820717 ratio |
1.00 |
vortex size/Arade |
146254760 bytes |
146254888 bytes |
1.00 |
compress time/Bimbo |
11408924083.8 ns (14911071.042499542) |
9593647255.9 ns (24472544.388750076) |
1.19 |
compress time/Bimbo throughput |
7121333608 bytes |
7121333608 bytes |
1 |
parquet_rs-zstd compress time/Bimbo |
21473225405 ns (63198112.3912487) |
21395040747.3 ns (50044105.56249809) |
1.00 |
parquet_rs-zstd compress time/Bimbo throughput |
7121333608 bytes |
7121333608 bytes |
1 |
decompress time/Bimbo |
5153334347.3 ns (76139653.2562499) |
4555849021.7 ns (21751756.538749695) |
1.13 |
decompress time/Bimbo throughput |
7121333608 bytes |
7121333608 bytes |
1 |
parquet_rs-zstd decompress time/Bimbo |
2649471912.8 ns (6065922.700000048) |
2626746098 ns (7096103.981249809) |
1.01 |
parquet_rs-zstd decompress time/Bimbo throughput |
7121333608 bytes |
7121333608 bytes |
1 |
vortex:parquet-zstd size/Bimbo |
1.2342947693780078 ratio |
1.1814937059155102 ratio |
1.04 |
vortex:raw size/Bimbo |
0.06727570120599242 ratio |
0.06439775935855861 ratio |
1.04 |
vortex size/Bimbo |
479092712 bytes |
458597928 bytes |
1.04 |
compress time/CMSprovider |
11826025915.6 ns (39411307.35000038) |
11959834712.7 ns (44918549.44999981) |
0.99 |
compress time/CMSprovider throughput |
5149123964 bytes |
5149123964 bytes |
1 |
parquet_rs-zstd compress time/CMSprovider |
19970645313.3 ns (38409418.93500137) |
20067275921.1 ns (27604922.590000153) |
1.00 |
parquet_rs-zstd compress time/CMSprovider throughput |
5149123964 bytes |
5149123964 bytes |
1 |
decompress time/CMSprovider |
11771320291.5 ns (339445771.5550003) |
7428652913.3 ns (48182453.69750023) |
1.58 |
decompress time/CMSprovider throughput |
5149123964 bytes |
5149123964 bytes |
1 |
parquet_rs-zstd decompress time/CMSprovider |
4940665115.6 ns (23378203.506249905) |
4820717011.9 ns (17805964.353749752) |
1.02 |
parquet_rs-zstd decompress time/CMSprovider throughput |
5149123964 bytes |
5149123964 bytes |
1 |
vortex:parquet-zstd size/CMSprovider |
1.2143093843145671 ratio |
1.201636956454599 ratio |
1.01 |
vortex:raw size/CMSprovider |
0.18147260437562074 ratio |
0.17958044251117197 ratio |
1.01 |
vortex size/CMSprovider |
934424936 bytes |
924681960 bytes |
1.01 |
compress time/Euro2016 |
2652805877.1 ns (3288181.4437499046) |
2652941717.2 ns (6150990.676249981) |
1.00 |
compress time/Euro2016 throughput |
393253221 bytes |
393253221 bytes |
1 |
parquet_rs-zstd compress time/Euro2016 |
1594651123.6 ns (3670244.7537499666) |
1579784943.3 ns (2314624.6175000668) |
1.01 |
parquet_rs-zstd compress time/Euro2016 throughput |
393253221 bytes |
393253221 bytes |
1 |
decompress time/Euro2016 |
297196488.8 ns (1054664.363749981) |
295196974.9 ns (1644819.7631249726) |
1.01 |
decompress time/Euro2016 throughput |
393253221 bytes |
393253221 bytes |
1 |
parquet_rs-zstd decompress time/Euro2016 |
494811887.45 ns (3278997.451875001) |
485170065.85 ns (1190065.5343750417) |
1.02 |
parquet_rs-zstd decompress time/Euro2016 throughput |
393253221 bytes |
393253221 bytes |
1 |
vortex:parquet-zstd size/Euro2016 |
1.4383489987803337 ratio |
1.439598423731457 ratio |
1.00 |
vortex:raw size/Euro2016 |
0.4348474490943839 ratio |
0.43522518026622853 ratio |
1.00 |
vortex size/Euro2016 |
171005160 bytes |
171153704 bytes |
1.00 |
compress time/Food |
1093999892.8 ns (4734065.761250019) |
1123908697.5 ns (2196755.3800001144) |
0.97 |
compress time/Food throughput |
332718229 bytes |
332718229 bytes |
1 |
parquet_rs-zstd compress time/Food |
1125378440.2 ns (4115810.6999999285) |
1131679797 ns (2780140.980000019) |
0.99 |
parquet_rs-zstd compress time/Food throughput |
332718229 bytes |
332718229 bytes |
1 |
decompress time/Food |
336288505.15 ns (933418.2531250119) |
205681624.76666665 ns (911765.3441666812) |
1.63 |
decompress time/Food throughput |
332718229 bytes |
332718229 bytes |
1 |
parquet_rs-zstd decompress time/Food |
216802696.7 ns (606929.2083333433) |
218110648.6 ns (477165.75500001013) |
0.99 |
parquet_rs-zstd decompress time/Food throughput |
332718229 bytes |
332718229 bytes |
1 |
vortex:parquet-zstd size/Food |
1.2389367897825485 ratio |
1.2386506182043049 ratio |
1.00 |
vortex:raw size/Food |
0.1349086526906225 ratio |
0.13487787589780662 ratio |
1.00 |
vortex size/Food |
44886568 bytes |
44876328 bytes |
1.00 |
compress time/HashTags |
2567108739.3 ns (7785207.458750248) |
2612505337.1 ns (6272545.934999943) |
0.98 |
compress time/HashTags throughput |
804495592 bytes |
804495592 bytes |
1 |
parquet_rs-zstd compress time/HashTags |
2525119092.4 ns (8139534.782500267) |
2535188202.6 ns (7130916.028749943) |
1.00 |
parquet_rs-zstd compress time/HashTags throughput |
804495592 bytes |
804495592 bytes |
1 |
decompress time/HashTags |
931779171.5 ns (5933400.97874999) |
628915128.2 ns (2587671.1087499857) |
1.48 |
decompress time/HashTags throughput |
804495592 bytes |
804495592 bytes |
1 |
parquet_rs-zstd decompress time/HashTags |
780703235.2 ns (7363373.528750002) |
784181210.6 ns (7824539.75) |
1.00 |
parquet_rs-zstd decompress time/HashTags throughput |
804495592 bytes |
804495592 bytes |
1 |
vortex:parquet-zstd size/HashTags |
1.656176458210158 ratio |
1.7048629043277994 ratio |
0.97 |
vortex:raw size/HashTags |
0.27578834018024057 ratio |
0.2838956611710061 ratio |
0.97 |
vortex size/HashTags |
221870504 bytes |
228392808 bytes |
0.97 |
compress time/TPC-H l_comment chunked without fsst |
3882220951.4 ns (44104707.26250005) |
3804162595.8 ns (21048403.422499657) |
1.02 |
compress time/TPC-H l_comment chunked without fsst throughput |
249197090 bytes |
249197090 bytes |
1 |
parquet_rs-zstd compress time/TPC-H l_comment chunked without fsst |
910462934.4 ns (2141361.571249962) |
905006471.2 ns (2450995.6624999642) |
1.01 |
parquet_rs-zstd compress time/TPC-H l_comment chunked without fsst throughput |
249197090 bytes |
249197090 bytes |
1 |
decompress time/TPC-H l_comment chunked without fsst |
111706414.33857143 ns (1105735.941961296) |
114162007.33226189 ns (1378784.6311934516) |
0.98 |
decompress time/TPC-H l_comment chunked without fsst throughput |
249197090 bytes |
249197090 bytes |
1 |
parquet_rs-zstd decompress time/TPC-H l_comment chunked without fsst |
251527173.1 ns (1587370.174999997) |
251591800.3 ns (889287.650000006) |
1.00 |
parquet_rs-zstd decompress time/TPC-H l_comment chunked without fsst throughput |
249197090 bytes |
249197090 bytes |
1 |
vortex:parquet-zstd size/TPC-H l_comment chunked without fsst |
4.607569789656717 ratio |
4.607573080060469 ratio |
1.00 |
vortex:raw size/TPC-H l_comment chunked without fsst |
1.052708440535963 ratio |
1.0527068995869895 ratio |
1.00 |
vortex size/TPC-H l_comment chunked without fsst |
262331880 bytes |
262331496 bytes |
1.00 |
compress time/TPC-H l_comment chunked |
918502138.3 ns (4664399.023750007) |
916574878.2 ns (1929249.1700000167) |
1.00 |
compress time/TPC-H l_comment chunked throughput |
249197090 bytes |
249197090 bytes |
1 |
parquet_rs-zstd compress time/TPC-H l_comment chunked |
906435609 ns (1584133.3400000334) |
907805341.9 ns (1899144.5299999714) |
1.00 |
parquet_rs-zstd compress time/TPC-H l_comment chunked throughput |
249197090 bytes |
249197090 bytes |
1 |
decompress time/TPC-H l_comment chunked |
133573197.61757934 ns (676492.1699603051) |
133724709.12 ns (455925.0801458284) |
1.00 |
decompress time/TPC-H l_comment chunked throughput |
249197090 bytes |
249197090 bytes |
1 |
parquet_rs-zstd decompress time/TPC-H l_comment chunked |
252937286.55 ns (975610.1262499988) |
252884998.5 ns (934611.7518749982) |
1.00 |
parquet_rs-zstd decompress time/TPC-H l_comment chunked throughput |
249197090 bytes |
249197090 bytes |
1 |
vortex:parquet-zstd size/TPC-H l_comment chunked |
1.3479625362806418 ratio |
1.3479643479539676 ratio |
1.00 |
vortex:raw size/TPC-H l_comment chunked |
0.3079739655065796 ratio |
0.30797370868175067 ratio |
1.00 |
vortex size/TPC-H l_comment chunked |
76746216 bytes |
76746152 bytes |
1.00 |
compress time/TPC-H l_comment canonical |
919224862.4 ns (2515024.9281249642) |
918632983.3 ns (1363036.2750000358) |
1.00 |
compress time/TPC-H l_comment canonical throughput |
249197106 bytes |
249197106 bytes |
1 |
parquet_rs-zstd compress time/TPC-H l_comment canonical |
917733085.75 ns (1451177.949999988) |
915623009.15 ns (1267345.4912499785) |
1.00 |
parquet_rs-zstd compress time/TPC-H l_comment canonical throughput |
249197106 bytes |
249197106 bytes |
1 |
decompress time/TPC-H l_comment canonical |
132977269.01908731 ns (213387.41652778536) |
132370741.09452382 ns (352370.19202777743) |
1.00 |
decompress time/TPC-H l_comment canonical throughput |
249197106 bytes |
249197106 bytes |
1 |
parquet_rs-zstd decompress time/TPC-H l_comment canonical |
250562786.17444444 ns (709343.2291944623) |
251560932.80255952 ns (519873.2387202382) |
1.00 |
parquet_rs-zstd decompress time/TPC-H l_comment canonical throughput |
249197106 bytes |
249197106 bytes |
1 |
vortex:parquet-zstd size/TPC-H l_comment canonical |
1.3479505092474404 ratio |
1.3479750020387498 ratio |
1.00 |
vortex:raw size/TPC-H l_comment canonical |
0.30797394573274056 ratio |
0.30797368890792814 ratio |
1.00 |
vortex size/TPC-H l_comment canonical |
76746216 bytes |
76746152 bytes |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
|
Oh wait this needs #1121 |
|
closing in favor of #1146 |
lwwmanning
pushed a commit
that referenced
this pull request
Oct 28, 2024
Fixes #1041 Change the IntoCanonical implementation for DictArray to do canonicalize-then-take for stringy things, take-then-canonicalize for everything else. It is always the case for strings that canonicalize-then-take will be faster, regardless of the compression of the values array. Overrides #1136
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
fixes #1041