Readme.md 4.69 KB
Newer Older
Florian Fischer's avatar
Florian Fischer committed
1
# allocbench - benchmark tool for POSIX memory allocators
Florian Fischer's avatar
Florian Fischer committed
2

Florian Fischer's avatar
Florian Fischer committed
3
4
5
allocbench is a POSIX memory allocator benchmarking framework and tooling.

To obtain allocbench run
6

Florian Fischer's avatar
Florian Fischer committed
7
8
9
10
```shell
git clone https://muhq.space/software/allocbench.git
```

Florian Fischer's avatar
Florian Fischer committed
11
12
## Requirements

Florian Fischer's avatar
Florian Fischer committed
13
* python >= 3.6
14
* make, find, gcc (build dependencies)
Florian Fischer's avatar
Florian Fischer committed
15
16
* perf (`perf stat -d` is the default command to measure benchmark results)
* util-linux (`whereis` is used to find system installed allocators)
Florian Fischer's avatar
Florian Fischer committed
17
18
* git, tar to handle external artifacts
* numpy and matplotlib to summarize results and generate plots
Florian Fischer's avatar
Florian Fischer committed
19
20


Florian Fischer's avatar
Florian Fischer committed
21
## Usage
Florian Fischer's avatar
Florian Fischer committed
22
23
allocbench consists of three small utilities: `bench.py`, `summarize.py` and `merge.py`.
`bench.py` is used to prepare, analyze and run benchmarks.
Florian Fischer's avatar
Florian Fischer committed
24

Florian Fischer's avatar
Florian Fischer committed
25
26
	usage: bench.py [-h] [--analyze] [-r RUNS] [-v]
	                [-b BENCHMARKS [BENCHMARKS ...]]
Florian Fischer's avatar
Florian Fischer committed
27
	                [-xb EXCLUDE_BENCHMARKS [EXCLUDE_BENCHMARKS ...]]
Florian Fischer's avatar
Florian Fischer committed
28
29
	                [-a ALLOCATORS [ALLOCATORS ...]] [-rd RESULTDIR] [--license]
	                [--version]
Florian Fischer's avatar
Florian Fischer committed
30

Florian Fischer's avatar
Florian Fischer committed
31
32
33
34
	benchmark memory allocators

	optional arguments:
	  -h, --help            show this help message and exit
Florian Fischer's avatar
Florian Fischer committed
35
	  --analyze             analyze benchmark behavior using malt
Florian Fischer's avatar
Florian Fischer committed
36
37
38
39
40
41
42
43
44
45
46
	  -r RUNS, --runs RUNS  how often the benchmarks run
	  -v, --verbose         more output
	  -b BENCHMARKS [BENCHMARKS ...], --benchmarks BENCHMARKS [BENCHMARKS ...]
	                        benchmarks to run
	  -xb EXCLUDE_BENCHMARKS [EXCLUDE_BENCHMARKS ...], --exclude-benchmarks EXCLUDE_BENCHMARKS [EXCLUDE_BENCHMARKS ...]
	                        explicitly excluded benchmarks
	  -a ALLOCATORS [ALLOCATORS ...], --allocators ALLOCATORS [ALLOCATORS ...]
	                        allocators to test
	  -rd RESULTDIR, --resultdir RESULTDIR
	                        directory where all results go
	  --license             print license info and exit
Florian Fischer's avatar
Florian Fischer committed
47
48
49
50
51
	  --version             print version info and exit

`./summarize.py` is used to summarize results created with bench.py.
It groups the included allocators into categories to produce readable and not extremely noisy plots.

52
	usage: summarize.py [-h] [-t FILE_EXT] [--license] [--version]
Florian Fischer's avatar
Florian Fischer committed
53
54
55
56
57
58
59
60
61
62
63
	                    [-b BENCHMARKS [BENCHMARKS ...]]
	                    [-x EXCLUDE_BENCHMARKS [EXCLUDE_BENCHMARKS ...]]
	                    results

	Summarize allocbench results in allocator sets

	positional arguments:
	  results               path to results

	optional arguments:
	  -h, --help            show this help message and exit
64
65
	  -t FILE_EXT, --file-ext FILE_EXT
	                        file extension used for plots
Florian Fischer's avatar
Florian Fischer committed
66
67
68
69
70
71
72
	  --license             print license info and exit
	  --version             print version info and exit
	  -b BENCHMARKS [BENCHMARKS ...], --benchmarks BENCHMARKS [BENCHMARKS ...]
	                        benchmarks to summarize
	  -x EXCLUDE_BENCHMARKS [EXCLUDE_BENCHMARKS ...], --exclude-benchmarks EXCLUDE_BENCHMARKS [EXCLUDE_BENCHMARKS ...]
	                        benchmarks to exclude

73

Florian Fischer's avatar
Florian Fischer committed
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
`./merge.py` can combine the results of different benchmark runs.

	usage: merge.py [-h] [--license] [--version] [-b BENCHMARKS [BENCHMARKS ...]]
	                [-x EXCLUDE_BENCHMARKS [EXCLUDE_BENCHMARKS ...]]
	                src dest

	Merge to allocbench results

	positional arguments:
	  src                   results which should be merged into dest
	  dest                  results in which src should be merged

	optional arguments:
	  -h, --help            show this help message and exit
	  --license             print license info and exit
	  --version             print version info and exit
	  -b BENCHMARKS [BENCHMARKS ...], --benchmarks BENCHMARKS [BENCHMARKS ...]
	                        benchmarks to summarize
	  -x EXCLUDE_BENCHMARKS [EXCLUDE_BENCHMARKS ...], --exclude-benchmarks EXCLUDE_BENCHMARKS [EXCLUDE_BENCHMARKS ...]
	                        benchmarks to exclude
Florian Fischer's avatar
Florian Fischer committed
94

Florian Fischer's avatar
Florian Fischer committed
95
96
97
98
### Examples

	./bench.py -b loop

Florian Fischer's avatar
Florian Fischer committed
99
100
runs only the loop benchmark for all included allocators and will put its
results in `$PWD/results/$HOSTNAME/<time>/loop`.
Florian Fischer's avatar
Florian Fischer committed
101

Florian Fischer's avatar
Florian Fischer committed
102
	./bench.py -a BA_allocators
Florian Fischer's avatar
Florian Fischer committed
103

Florian Fischer's avatar
Florian Fischer committed
104
builds all allocators used in Florian Fischer's [BA thesis](https://muhq.space/ba.html)
Florian Fischer's avatar
Florian Fischer committed
105
and runs all benchmarks.
Florian Fischer's avatar
Florian Fischer committed
106

Florian Fischer's avatar
Florian Fischer committed
107
	./summarize.py <path/to/saved/results>
Florian Fischer's avatar
Florian Fischer committed
108

Florian Fischer's avatar
Florian Fischer committed
109
summarizes the previously created results.
Florian Fischer's avatar
Florian Fischer committed
110
111
112
113

## Benchmarks

You want to compare allocators with your own software or add a new benchmark,
Florian Fischer's avatar
Florian Fischer committed
114
have a look at [doc/Benchmarks.md](doc/Benchmarks.md).
Florian Fischer's avatar
Florian Fischer committed
115
116
117

## Allocators

Florian Fischer's avatar
Florian Fischer committed
118
119
By default all included allocators will be build and measured. For more precise control
about used allocators use the `-a` option.
Florian Fischer's avatar
Florian Fischer committed
120

Florian Fischer's avatar
Florian Fischer committed
121
122
For more details about what allocators are available, how they are used and how to
include new once have a look at [doc/Allocators.md](doc/Allocators.md).
Florian Fischer's avatar
Florian Fischer committed
123

Florian Fischer's avatar
add GPL    
Florian Fischer committed
124
125
126
127
## License

This program is released under GPLv3. You can find a copy of the license
in the LICENSE file in the programs root directory.