# Evaluation artifact for the emper IO runtime

We use a lot of connections in our evaluation if you encounter issues during
the connect phase of the client make sure your SOMAXCONN value is big enough.

## prepare

Three git submodules are included for emper, the legacy emper IO system, a rust based
load generator and for an echoserver using the IO design of memcached.

Run `git submodule init --update` to clone and checkout all submodules.

## build

Requirements:

	* pthread
	* libevent
	* rust / cargo
	* golang

Simply run `make` to build all non-emper echo server implementations

All used emper variants will be build when running `eval.py`.

## evaluate

Run `./eval.py -l` to benchmark all included implementations on the localhost.

See `./eval.py -h` for available options.

Evaluation results will be generated in `results/<git-desc>-<clients>+-<server-host>/<timestamp>`

A simple matplotlib plot can be shown by running `plot_results.py results/<experiment-description>/<timestamp>`

## Evaluation across hosts

Per default `eval.py` will start the servers per ssh on faui49big02 so
if you run `eval.py` on any other machine you have a two host setup.
The server host is changeable using the `--host` cli argument.
Additionally clients can be specified with the `--clients` argument.
The client commands will be started via SSH on the remote hosts and
synchronize the start of their echophase using a coordinator process running
on the host where `eval.py` is running.

## Adding server implementations

All server implementations are defined in `bench/server_cmds.py` and the standard
Makefile will recursively build all Makefiles found in `servers/`.
Adding a new server is done by including the needed source and a Makefile to build
it in the `server/` subdirectory and add the command used to run the server in `bench/server_cmds.py`.

EMPER variants are defined in `bench/emper.py`.
See the documentation there for information about how EMPER variants are build and
which options are available.