Readme.md 4.58 KB
Newer Older
Florian Fischer's avatar
Florian Fischer committed
1
# allocbench - benchmark tool for POSIX memory allocators
Florian Fischer's avatar
Florian Fischer committed
2

Florian Fischer's avatar
Florian Fischer committed
3
4
5
allocbench is a POSIX memory allocator benchmarking framework and tooling.

To obtain allocbench run
6

Florian Fischer's avatar
Florian Fischer committed
7
8
9
10
```shell
git clone https://muhq.space/software/allocbench.git
```

Florian Fischer's avatar
Florian Fischer committed
11
12
## Requirements

Florian Fischer's avatar
Florian Fischer committed
13
* python >= 3.6
14
* make, find, gcc (build dependencies)
Florian Fischer's avatar
Florian Fischer committed
15
16
* perf (`perf stat -d` is the default command to measure benchmark results)
* util-linux (`whereis` is used to find system installed allocators)
Florian Fischer's avatar
Florian Fischer committed
17
18
* git, tar to handle external artifacts
* numpy and matplotlib to summarize results and generate plots
Florian Fischer's avatar
Florian Fischer committed
19
20


Florian Fischer's avatar
Florian Fischer committed
21
## Usage
Florian Fischer's avatar
Florian Fischer committed
22
23
allocbench consists of three small utilities: `bench.py`, `summarize.py` and `merge.py`.
`bench.py` is used to prepare, analyze and run benchmarks.
Florian Fischer's avatar
Florian Fischer committed
24

Florian Fischer's avatar
Florian Fischer committed
25
26
	usage: bench.py [-h] [--analyze] [-r RUNS] [-v]
	                [-b BENCHMARKS [BENCHMARKS ...]]
Florian Fischer's avatar
Florian Fischer committed
27
	                [-xb EXCLUDE_BENCHMARKS [EXCLUDE_BENCHMARKS ...]]
Florian Fischer's avatar
Florian Fischer committed
28
29
	                [-a ALLOCATORS [ALLOCATORS ...]] [-rd RESULTDIR] [--license]
	                [--version]
Florian Fischer's avatar
Florian Fischer committed
30

Florian Fischer's avatar
Florian Fischer committed
31
32
33
34
	benchmark memory allocators

	optional arguments:
	  -h, --help            show this help message and exit
Florian Fischer's avatar
Florian Fischer committed
35
	  --analyze             analyze benchmark behavior using malt
Florian Fischer's avatar
Florian Fischer committed
36
37
38
39
40
41
42
43
44
45
46
	  -r RUNS, --runs RUNS  how often the benchmarks run
	  -v, --verbose         more output
	  -b BENCHMARKS [BENCHMARKS ...], --benchmarks BENCHMARKS [BENCHMARKS ...]
	                        benchmarks to run
	  -xb EXCLUDE_BENCHMARKS [EXCLUDE_BENCHMARKS ...], --exclude-benchmarks EXCLUDE_BENCHMARKS [EXCLUDE_BENCHMARKS ...]
	                        explicitly excluded benchmarks
	  -a ALLOCATORS [ALLOCATORS ...], --allocators ALLOCATORS [ALLOCATORS ...]
	                        allocators to test
	  -rd RESULTDIR, --resultdir RESULTDIR
	                        directory where all results go
	  --license             print license info and exit
Florian Fischer's avatar
Florian Fischer committed
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
	  --version             print version info and exit

`./summarize.py` is used to summarize results created with bench.py.
It groups the included allocators into categories to produce readable and not extremely noisy plots.

	usage: summarize.py [-h] [--license] [--version]
	                    [-b BENCHMARKS [BENCHMARKS ...]]
	                    [-x EXCLUDE_BENCHMARKS [EXCLUDE_BENCHMARKS ...]]
	                    results

	Summarize allocbench results in allocator sets

	positional arguments:
	  results               path to results

	optional arguments:
	  -h, --help            show this help message and exit
	  --license             print license info and exit
	  --version             print version info and exit
	  -b BENCHMARKS [BENCHMARKS ...], --benchmarks BENCHMARKS [BENCHMARKS ...]
	                        benchmarks to summarize
	  -x EXCLUDE_BENCHMARKS [EXCLUDE_BENCHMARKS ...], --exclude-benchmarks EXCLUDE_BENCHMARKS [EXCLUDE_BENCHMARKS ...]
	                        benchmarks to exclude

`./merge.py` can combine the results of different benchmark runs.

	usage: merge.py [-h] [--license] [--version] [-b BENCHMARKS [BENCHMARKS ...]]
	                [-x EXCLUDE_BENCHMARKS [EXCLUDE_BENCHMARKS ...]]
	                src dest

	Merge to allocbench results

	positional arguments:
	  src                   results which should be merged into dest
	  dest                  results in which src should be merged

	optional arguments:
	  -h, --help            show this help message and exit
	  --license             print license info and exit
	  --version             print version info and exit
	  -b BENCHMARKS [BENCHMARKS ...], --benchmarks BENCHMARKS [BENCHMARKS ...]
	                        benchmarks to summarize
	  -x EXCLUDE_BENCHMARKS [EXCLUDE_BENCHMARKS ...], --exclude-benchmarks EXCLUDE_BENCHMARKS [EXCLUDE_BENCHMARKS ...]
	                        benchmarks to exclude
Florian Fischer's avatar
Florian Fischer committed
91

Florian Fischer's avatar
Florian Fischer committed
92
93
94
95
### Examples

	./bench.py -b loop

Florian Fischer's avatar
Florian Fischer committed
96
97
runs only the loop benchmark for all included allocators and will put its
results in `$PWD/results/$HOSTNAME/<time>/loop`.
Florian Fischer's avatar
Florian Fischer committed
98

Florian Fischer's avatar
Florian Fischer committed
99
	./bench.py -a BA_allocators
Florian Fischer's avatar
Florian Fischer committed
100

Florian Fischer's avatar
Florian Fischer committed
101
builds all allocators used in Florian Fischer's [BA thesis](https://muhq.space/ba.html)
Florian Fischer's avatar
Florian Fischer committed
102
and runs all benchmarks.
Florian Fischer's avatar
Florian Fischer committed
103

Florian Fischer's avatar
Florian Fischer committed
104
	./summarize.py <path/to/saved/results>
Florian Fischer's avatar
Florian Fischer committed
105

Florian Fischer's avatar
Florian Fischer committed
106
summarizes the previously created results.
Florian Fischer's avatar
Florian Fischer committed
107
108
109
110

## Benchmarks

You want to compare allocators with your own software or add a new benchmark,
Florian Fischer's avatar
Florian Fischer committed
111
have a look at [doc/Benchmarks.md](doc/Benchmarks.md).
Florian Fischer's avatar
Florian Fischer committed
112
113
114

## Allocators

Florian Fischer's avatar
Florian Fischer committed
115
116
By default all included allocators will be build and measured. For more precise control
about used allocators use the `-a` option.
Florian Fischer's avatar
Florian Fischer committed
117

Florian Fischer's avatar
Florian Fischer committed
118
119
For more details about what allocators are available, how they are used and how to
include new once have a look at [doc/Allocators.md](doc/Allocators.md).
Florian Fischer's avatar
Florian Fischer committed
120

Florian Fischer's avatar
add GPL    
Florian Fischer committed
121
122
123
124
## License

This program is released under GPLv3. You can find a copy of the license
in the LICENSE file in the programs root directory.