[Rpm-maint] Rpm database backend benchmarks
Michael Schroeder
mls at suse.de
Wed Oct 16 13:28:42 UTC 2019
Hi,
I wrote a little benchmarking tool to find out how the different
database backends compare. The backends tested are bdb, ndb,
lmdb, and the new sqlite backend.
I used the 2506 packages I have on my system as data set. The
benchmark tool does the following:
Add all packages to an empty database, in forward/reverse/random
order.
Remove all packages from a database that contains all packages,
in forward/reverse/random order.
Update all packages in forward/reverse/random order. An update
consistes of an reinstall of the package plus a remove of the
old header
Simulate a dependency check of all packages (rpm -Va --nofiles).
The install/erase/update lines look like this:
Operation
forward / reverse / random in seconds with fsync disabled
forward / reverse / random in seconds with fsync enabled
forward / reverse / random disk space used in MByte
Here's the result:
BDB
---
Adding all headers...
3.19s / 3.21s / 3.17s
85.69s / 88.58s / 84.99s
164.41M / 164.48M / 164.52M
Erasing all headers...
3.74s / 3.70s / 3.70s
70.00s / 70.06s / 74.60s
164.29M / 164.30M / 164.29M
Updating all headers...
7.23s / 7.31s / 7.26s
147.16s / 150.98s / 147.26s
174.17M / 174.07M / 174.17M
Dep check
4.82s
NDB
---
Adding all headers...
1.05s / 0.99s / 1.04s
46.91s / 41.94s / 42.09s
177.70M / 175.22M / 173.37M
Erasing all headers...
1.10s / 0.95s / 1.09s
42.47s / 30.93s / 36.13s
0.67M / 0.71M / 0.82M
Updating all headers...
2.34s / 2.42s / 2.61s
78.17s / 70.24s / 72.18s
164.97M / 170.09M / 168.97M
Dep check
2.81s
LMDB
----
Adding all headers...
1.06s / 1.03s / 1.03s
(not implemented)
268.44M / 268.44M / 268.44M
Erasing all headers...
1.13s / 1.14s / 1.13s
(not implemented)
268.44M / 268.44M / 268.44M
Updating all headers...
3.33s / 3.37s / 3.42s
(not implemented)
268.44M / 268.44M / 268.44M
Dep check
2.36s
SQLITE
------
Adding all headers...
4.24s / 4.28s / 4.24s
34.50s / 38.03s / 34.48s
158.58M / 158.54M / 158.58M
Erasing all headers...
19.58s / 19.59s / 20.52s
51.12s / 55.65s / 51.83s
158.58M / 158.58M / 158.58M
Updating all headers...
45.52s / 45.84s / 46.39s
108.55s / 114.18s / 113.97s
172.46M / 171.74M / 173.19M
Dep check
12.50s
Things to note:
- Berkeley db is actually not that fast, both lmdb and ndb are much
faster
- rpm's lmdb code does not implement fsync
- there's something weird with the sqlite package erase, it takes
way too much time with fsync disabled
- sqlite is quite slow
- sqlite's "Adding all header" benchmark is quite fast with fsync
enabled. I wonder what sqlite guarantees if there is a crash
- ndb is the only database that can shrink if packages get
removed
- lmdb won the dep check benchmark. I think this is because it
mmaps the complete database and thus has not to copy the header
data.
If fsync is enabled (aka normal rpm operation), all implementations
take very long. The question is how much this is drowned out by the
time spent in unpacking/erasing all the files on disk.
Cheers,
Michael.
--
Michael Schroeder mls at suse.de
SUSE LINUX GmbH, GF Jeff Hawn, HRB 16746 AG Nuernberg
main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);}
More information about the Rpm-maint
mailing list