Benchmarks

JMH is used to build and run benchmarks. Benchmark results are obtained on a PC running under Windows 7 with Intel(R) Core(TM) i7-3770 3.4 GHz CPU and 64-bit JRE build 1.8.0_77-b03 with the following parameters: -Xms1g -Xmx1g. Nothing else is configured and tests run with default settings. To get results in your environment, run in the project root directory:

./gradlew testBenchmarksJMH

Tokyo Cabinet Benchmark

Tokyo Cabinet Benchmark is useful for comparing the performance of key/value storages. For one million 8-char string keys ('00000000', '00000001', etc.) and values equal to the keys, four operations are measured:

1. writing all key/value pairs in ascending order;
2. writing all key/value pairs in random order;
3. reading all key/value pairs in ascending order;
4. reading all key/value pairs in random order.

Currently, benchmark results are available for Xodus stores with key prefixing (Patricia tree) and without (BTree), Berkeley DB JE databases with and without key prefixing, MapDb tree map, and Chronicle Map. All the scores are in seconds per single benchmark run for one million keys. Excerpt of the output of the build running benchmarks is as follows:

# Run complete. Total time: 02:13:06

Benchmark                                                             Mode  Cnt   Score   Error   Units
chronicle.JMHChronicleMapTokyoCabinetReadBenchmark.randomRead           ss   16   0.829 - 0.075    s/op
chronicle.JMHChronicleMapTokyoCabinetReadBenchmark.successiveRead       ss   16   0.098 - 0.005    s/op
chronicle.JMHChronicleMapTokyoCabinetWriteBenchmark.randomWrite         ss   16   1.238 - 0.133    s/op
chronicle.JMHChronicleMapTokyoCabinetWriteBenchmark.successiveWrite     ss   16   1.184 - 0.099    s/op
env.JMHEnvTokyoCabinetReadBenchmark.randomRead                          ss   60   2.301 - 0.044    s/op
env.JMHEnvTokyoCabinetReadBenchmark.successiveRead                      ss   60   0.087 - 0.005    s/op
env.JMHEnvTokyoCabinetWriteBenchmark.randomWrite                        ss   60   2.208 - 0.054    s/op
env.JMHEnvTokyoCabinetWriteBenchmark.successiveWrite                    ss   60   0.691 - 0.014    s/op
env.JMHEnvWithPrefixingTokyoCabinetReadBenchmark.randomRead             ss   60   1.189 - 0.022    s/op
env.JMHEnvWithPrefixingTokyoCabinetReadBenchmark.successiveRead         ss   60   0.271 - 0.047    s/op
env.JMHEnvWithPrefixingTokyoCabinetWriteBenchmark.randomWrite           ss   60   1.667 - 0.070    s/op
env.JMHEnvWithPrefixingTokyoCabinetWriteBenchmark.successiveWrite       ss   60   0.769 - 0.067    s/op
je.JMH_JETokyoCabinetReadBenchmark.randomRead                           ss   60   3.208 - 0.027    s/op
je.JMH_JETokyoCabinetReadBenchmark.successiveRead                       ss   60   1.252 - 0.189    s/op
je.JMH_JETokyoCabinetWriteBenchmark.randomWrite                         ss   60   5.844 - 0.145    s/op
je.JMH_JETokyoCabinetWriteBenchmark.successiveWrite                     ss   60   3.441 - 0.187    s/op
je.JMH_JEWithPrefixingTokyoCabinetReadBenchmark.randomRead              ss   60   3.504 - 0.070    s/op
je.JMH_JEWithPrefixingTokyoCabinetReadBenchmark.successiveRead          ss   60   1.302 - 0.200    s/op
je.JMH_JEWithPrefixingTokyoCabinetWriteBenchmark.randomWrite            ss   60   5.905 - 0.205    s/op
je.JMH_JEWithPrefixingTokyoCabinetWriteBenchmark.successiveWrite        ss   60   3.411 - 0.209    s/op
mapdb.JMHMapDbTokyoCabinetReadBenchmark.randomRead                      ss   16   6.705 - 0.196    s/op
mapdb.JMHMapDbTokyoCabinetReadBenchmark.successiveRead                  ss   16   0.274 - 0.091    s/op
mapdb.JMHMapDbTokyoCabinetWriteBenchmark.randomWrite                    ss   16  21.822 - 0.380    s/op
mapdb.JMHMapDbTokyoCabinetWriteBenchmark.successiveWrite                ss   16  26.588 - 0.517    s/op

The same results in table form:

	Random Read	Successive Read	Random Write	Successive Write
Xodus store with key prefixing (Patricia)	1.189	0.271	1.667	0.769
Xodus store without key prefixing (BTree)	2.301	0.087	2.208	0.691
BDB JE database with key prefixing	3.504	1.302	5.905	3.411
BDB JE database without key prefixing	3.208	1.252	5.844	3.441
MapDB tree map	6.705	0.274	21.822	26.588
Chronicle Map	0.829	0.098	1.238	1.184