emper merge requestshttps://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests2021-04-18T11:58:34Zhttps://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/172introduce new CRTP based worker sleep algorithm abstraction2021-04-18T11:58:34ZFlorian Fischerintroduce new CRTP based worker sleep algorithm abstractionIntroduce AbstractWorkerSleepStrategy a CRTP interface class for a worker
sleep algorithm.
A concrete WorkerSleepStrategy must implement the following functions:
template <CallerEnvironment callerEnvironment>
void notifyMany(unsigned ...Introduce AbstractWorkerSleepStrategy a CRTP interface class for a worker
sleep algorithm.
A concrete WorkerSleepStrategy must implement the following functions:
template <CallerEnvironment callerEnvironment>
void notifyMany(unsigned count);
template <CallerEnvironment callerEnvironment>
void notifyOne();
template <CallerEnvironment callerEnvironment>
void notifyAll();
void notifySpecific(workerid_t workerId);
void sleep();
The runtime uses this interface to notify the workers about new work
as well as ensuring that all workers get notified on termination.
All sempahore based worker sleep algorithm code was moved from the Runtime
into SemaphoreWorkerSleepStrategy which takes the used Semaphore as a
template parameter.
This introduces no functional changes and should be an zero-cost abstraction.
Encapsulating the worker sleep algorithm behind an interface allows us
to easier experiment with different approaches not based on semaphores
("Wait in the io_uring", "Empty flag per WSQ").https://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/171[EchClient] calculate and output experiment totals2021-04-16T07:21:01ZFlorian Fischer[EchClient] calculate and output experiment totalsThis makes looking at individual experiment results and identifying
outliers easier.This makes looking at individual experiment results and identifying
outliers easier.https://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/170[Future] improve cancellation2021-04-14T13:42:33ZFlorian Fischer[Future] improve cancellation* Improve warn message in Future::cancel()
* Don't cancel a Future in the destructor if it was already cancelled or received
* Restructure CancelFutureTest into single separate test cases
With this I can not reproduce the `CancelFutureT...* Improve warn message in Future::cancel()
* Don't cancel a Future in the destructor if it was already cancelled or received
* Restructure CancelFutureTest into single separate test cases
With this I can not reproduce the `CancelFutureTest` failures anymore
neither `SIGABRT` nor the `timeout`.
I tested the `CancelFutureTest` (10 times) on various build configurations with out any failure:
```sh
meson build-release --buildtype=release
meson build-debugoptimized
meson build-stats -Dstats=true
meson build-stats-laws -Dstats=true -Ddefault_scheduling_strategy=locality_aware_work_stealing
meson build-ci-failure -Dstats=true -Dio=false
for d in build-*
do
meson test -C ${d} CancelFutureTest --repeat=10
done
```
Not sure if this resolves #16. I will keep it open for now.https://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/169Minor fixes2021-04-13T08:42:46ZFlorian SchmausMinor fixeshttps://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/168[LawsScheduler] Delete stray #define EMPER_OVERFLOW_QUEUE2021-04-13T08:31:37ZFlorian Schmaus[LawsScheduler] Delete stray #define EMPER_OVERFLOW_QUEUEhttps://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/167[check-iwyu] Do not fail right away if IWYU_TOOL fails2021-04-12T16:55:53ZFlorian Schmaus[check-iwyu] Do not fail right away if IWYU_TOOL failsSince only newer versions of IWYU_TOOL fail with an exit code, we have
to disable -e around the IWYU_TOOL invocation. Otherwise we would not
print IWYU_LOG.Since only newer versions of IWYU_TOOL fail with an exit code, we have
to disable -e around the IWYU_TOOL invocation. Otherwise we would not
print IWYU_LOG.https://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/165[EchoClient] overhaul the EchoClient.cpp2021-04-14T17:38:30ZFlorian Fischer[EchoClient] overhaul the EchoClient.cpp* Reconnect (close, new socket, connect) when the TCP connection is unexpectedly closed and count the reconnects
* Send echo messages carrying information: "client-id:iteration"
* Use a Client class holding a clients state instead of a p...* Reconnect (close, new socket, connect) when the TCP connection is unexpectedly closed and count the reconnects
* Send echo messages carrying information: "client-id:iteration"
* Use a Client class holding a clients state instead of a pthread-style clientFunc
* Create the Client object in the client Fiber to use NUMA first-touch policy
* Reduce code duplication:
* starting a new Client is now done with Client::startNew
* latency measurement and error handling are moved from the clientIteration*
functions in the Client::run functionhttps://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/163[io/Future] take a const reference of the Future::Callback2021-04-12T09:46:23ZFlorian Fischer[io/Future] take a const reference of the Future::Callbackhttps://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/162[gitlab-ci] Update flowdalic/debian-testing-dev to 1.82021-04-12T10:45:44ZFlorian Schmaus[gitlab-ci] Update flowdalic/debian-testing-dev to 1.8https://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/161[IO] fix IoCompleterBehavior::none2021-04-12T09:54:34ZFlorian Fischer[IO] fix IoCompleterBehavior::noneOnly create, initiateTermination and waitUntilFinished if the
completer thread is actually used.
Currently we get a segfault when terminating the Runtime because we call `GlobalIoContext::initiateTermination` without calling `GlobalIoCo...Only create, initiateTermination and waitUntilFinished if the
completer thread is actually used.
Currently we get a segfault when terminating the Runtime because we call `GlobalIoContext::initiateTermination` without calling `GlobalIoContext::startGlobalCompleter`https://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/160Fix race condition in io::Stats2021-04-11T12:47:18ZFlorian SchmausFix race condition in io::StatsThe workerStats std::vector was modified concurrently without being
synchronized. The Stats are constructed together with IoContext and
the worker IoContexts are constructed concurrently at the beginning of
the worker loop. And in the St...The workerStats std::vector was modified concurrently without being
synchronized. The Stats are constructed together with IoContext and
the worker IoContexts are constructed concurrently at the beginning of
the worker loop. And in the Stats constructor the workerStats
std::vector was modified by adding the Stats instance that is
currently being constructed to it.
Turns out, we don't need the workerStats data structure at all. We
simply provide the global and worker IoContexts to
io::Stats::printStats(). This results IMHO in a cleaner design of
printStats() since there is no longer a data structure called
workerStats, that in fact, not only holds the worker's stats. And
while we at it, we rename io::Stats::printWorkerStats() to
io::Stats::printsStats(), since it does, in fact, not only print the
worker stats, but more or less all related worker stats.https://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/159[gitlab-ci] Fix test-with-stats2021-04-12T09:52:13ZFlorian Schmaus[gitlab-ci] Fix test-with-statsThe EMPER option Meson option for stats is called 'stats' not
'worker_stats'.The EMPER option Meson option for stats is called 'stats' not
'worker_stats'.https://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/158Add WorkerLocalData and use it for stats2021-04-13T10:08:45ZFlorian SchmausAdd WorkerLocalData and use it for statshttps://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/157[EchoClient] collect the total amount of unexpected echos2021-04-08T13:28:21ZFlorian Fischer[EchoClient] collect the total amount of unexpected echosMy first test runs don't show unexpected echos. Therefore I think
unexpected echos are rare and their average will always be 0 even if
some sparse unexpected echos occur.My first test runs don't show unexpected echos. Therefore I think
unexpected echos are rare and their average will always be 0 even if
some sparse unexpected echos occur.https://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/156[EchoClient] changes to easier debug our IO evaluation artifact2021-04-08T11:41:16ZFlorian Fischer[EchoClient] changes to easier debug our IO evaluation artifact* Use `emper::lib::math::RunningAverage` instead of doing it manually
* Count the unexpected Echos, print a error to the log and go continue
* Return a duration and error message pair from our `clientIteration*` functions
* Print the cur...* Use `emper::lib::math::RunningAverage` instead of doing it manually
* Count the unexpected Echos, print a error to the log and go continue
* Return a duration and error message pair from our `clientIteration*` functions
* Print the current iteration on error
* Wait 10 seconds after an error before terminatinghttps://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/155[EchoClient] improve unexpected echo output2021-04-08T08:58:48ZFlorian Fischer[EchoClient] improve unexpected echo outputOnly output the least significant byte of the unsigned int cast.
Always use two digits per byte and remove the whitespace.
Should have used printf in the first place.Only output the least significant byte of the unsigned int cast.
Always use two digits per byte and remove the whitespace.
Should have used printf in the first place.https://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/154[EchoClient] output buffers if they do not match2021-04-06T16:18:20ZFlorian Fischer[EchoClient] output buffers if they do not matchhttps://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/153[EchoClient] output the number of all echos done2021-04-03T10:30:24ZFlorian Fischer[EchoClient] output the number of all echos donehttps://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/152[tests] restructure tests directory2021-04-01T13:13:47ZFlorian Fischer[tests] restructure tests directoryMove io and lib related tests into io/ and lib/.
Our test dictionaries not must have a 'source' key with an array
created with files().
This ensures we use always correct paths regardless of the actual directory
we are currently in.
Un...Move io and lib related tests into io/ and lib/.
Our test dictionaries not must have a 'source' key with an array
created with files().
This ensures we use always correct paths regardless of the actual directory
we are currently in.
Until https://github.com/mesonbuild/meson/issues/8608 is not solved
we have to separately specify the tests name in a 'name' key.https://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/151[iwyu] Take load average into consideration on CI2021-04-12T09:57:28ZFlorian Schmaus[iwyu] Take load average into consideration on CIThis required that we backport iwyu_tool.py from
https://github.com/include-what-you-use/include-what-you-use/pull/891
into tools/, which supports --load.This required that we backport iwyu_tool.py from
https://github.com/include-what-you-use/include-what-you-use/pull/891
into tools/, which supports --load.