emper merge requestshttps://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests2022-05-13T12:54:47Zhttps://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/387runtime: do not notify workers about termination if they do not sleep2022-05-13T12:54:47ZFlorian Fischerruntime: do not notify workers about termination if they do not sleepI stumbled upon this because the no-sleep variants in the pulse evaluation
showed notifications in their stats.I stumbled upon this because the no-sleep variants in the pulse evaluation
showed notifications in their stats.https://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/386pulse: only measure time between work creation and dispatching it2022-05-13T13:22:09ZFlorian Fischerpulse: only measure time between work creation and dispatching itWe do not need to measure the work duration which is the known pulse.
This makes comparing the raw data easier because the latency does not
include the pulse duration.We do not need to measure the work duration which is the known pulse.
This makes comparing the raw data easier because the latency does not
include the pulse duration.https://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/385[SemaphoreWorkerSleepStrategy] Improve code comment2022-05-30T13:12:20ZFlorian Schmaus[SemaphoreWorkerSleepStrategy] Improve code commenthttps://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/383add additional mode of introducing work2022-05-04T07:19:03ZFlorian Fischeradd additional mode of introducing workThe default is Burst mode which inserts new work as fast as possible
sleeping until the next pulse.
Even mode distributes the new work evenly in the duration between two
pulses.The default is Burst mode which inserts new work as fast as possible
sleeping until the next pulse.
Even mode distributes the new work evenly in the duration between two
pulses.https://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/382fsearch: add possible semaphore limiting the concurrenct open requests2022-05-10T09:26:29ZFlorian Fischerfsearch: add possible semaphore limiting the concurrenct open requestshttps://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/381Improve pulse evaluation2022-04-26T14:46:36ZFlorian FischerImprove pulse evaluationhttps://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/380[io] Transform assert() to EMPER_ASSERT_MSG() in GlobalIoContext2022-04-25T18:35:29ZFlorian Schmaus[io] Transform assert() to EMPER_ASSERT_MSG() in GlobalIoContextWe recently saw this assert triggering in our CI. Let's yank up its
verbosity.We recently saw this assert triggering in our CI. Let's yank up its
verbosity.https://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/379[apps/FibChildStealing] Improve fibNum arg parsing2022-04-25T15:26:05ZFlorian Schmaus[apps/FibChildStealing] Improve fibNum arg parsingAlso, don't set an unsigned to -1…Also, don't set an unsigned to -1…https://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/378make sleep semaphore threshold configurable2022-04-25T11:21:48ZFlorian Fischermake sleep semaphore threshold configurable~~I think we rashly merged !377.~~
~~Mazstab using the xs problem size reports that using the semaphore greedy `(EMPER threshold == 0)` has significant (at least how interpret the plots no idea about the actual statistics) benefits for 5...~~I think we rashly merged !377.~~
~~Mazstab using the xs problem size reports that using the semaphore greedy `(EMPER threshold == 0)` has significant (at least how interpret the plots no idea about the actual statistics) benefits for 5 out of 7 benchmarks.~~
~~[matmul.pdf](/uploads/387dc2521c5eaf488db2040497ae6cad/matmul.pdf) for example.~~
I don't trust those reasults because my rts-descriptor also uses `sleep_sem_threshold=workerCount` which should exactly be exactly the same as the default configuration.
```yaml
emper:
meson:
options:
sleep_sem_threshold: workerCount
```https://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/377increase the sleep semaphore threshold2022-04-24T10:42:13ZFlorian Fischerincrease the sleep semaphore thresholdWe currently use the semaphore of our sleep strategy very greedy.
This is due to skipping the semaphore's `V()` operation if we are sure
that it does not kill our progress guaranty.
When scheduling new work from within the runtime we sk...We currently use the semaphore of our sleep strategy very greedy.
This is due to skipping the semaphore's `V()` operation if we are sure
that it does not kill our progress guaranty.
When scheduling new work from within the runtime we skip the wakeup if
we observe nobody sleeping. This is fine in terms of progress and
limits the amount of atomic operations on global state to a minimum.
Using a threshold of 0 (observe nobody sleeping) however introduces a
race between inserting new work and going to sleep which harm the
latency when the worker goes to sleep and is not notified about the
new work.
This race is common and can be observed in the pulse micro benchmark.
A emper with a threshold of 0 shows high latency compared to using
an io-based sleep strategy or when increasing the threshold.
```
$ build-release/eval/pulse | python -c "<calc mean>"
Starting pulse evaluation with pulse=1, iterations=30 and utilization=80
mean: 1721970116.425
$ build-increased-sem-threshold/eval/pulse | python -c "<calc mean"
Starting pulse evaluation with pulse=1, iterations=30 and utilization=80
mean: 1000023942.15
$ build-pipe-release/eval/pulse | python -c "<calc mean>
Starting pulse evaluation with pulse=1, iterations=30 and utilization=80
mean: 1000030557.0861111
$ build-pipe-no-completer/eval/pulse | python -c "<calc mean>"
Starting pulse evaluation with pulse=1, iterations=30 and utilization=80
mean: 1000021514.1805556
```
I could not measure any significant overhead due to the more atomics
on my 16 core machine using the fs-eval on a SSD.
```
$ ./eval.py -r 50 -i emper-vanilla emper-inc-sem-threshold emper-pipe emper-pipe-no-completer
...
$ ./summarize.py results/1599f44-dirty-pasture/<date>/ -f '{target}-{median} (+- {std})'
duration_time:u:
emper-vanilla-0.202106189 (+- 0.016075981812486713)
emper-inc-sem-threshold-0.2049344115 (+- 0.015348506939891596)
emper-pipe-0.21689131 (+- 0.015198438371285145)
emper-pipe-no-completer-0.1372724185 (+- 0.005865720218833998)
```https://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/376Pulse: initial pulse evaluation commit2022-04-23T18:35:34ZFlorian FischerPulse: initial pulse evaluation commithttps://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/375fsearch: add more fine grained control about the used fiber throttles2022-04-23T18:34:29ZFlorian Fischerfsearch: add more fine grained control about the used fiber throttleshttps://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/374io: improve recursive_directory_walk2022-04-08T12:44:16ZFlorian Fischerio: improve recursive_directory_walk* Add optional throttle Semaphore pointer to limit the number
of spawned fn as well as directory walk fibers
* Use const references to the passed functions instead of values
* fsearch: Use max_running as fn and recursion throttle* Add optional throttle Semaphore pointer to limit the number
of spawned fn as well as directory walk fibers
* Use const references to the passed functions instead of values
* fsearch: Use max_running as fn and recursion throttlehttps://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/373add io synchronous option2022-04-10T17:00:55ZFlorian Fischeradd io synchronous optionWhen EMPER is build with `-Dio_synchronous` each Future will be
completed synchronously when calling `Future::wait()`.When EMPER is build with `-Dio_synchronous` each Future will be
completed synchronously when calling `Future::wait()`.https://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/372Run clang tidy in parallel2022-04-04T12:26:46ZFlorian SchmausRun clang tidy in parallelhttps://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/371[EchoServers] improve the callback based echoserver2022-04-04T13:54:06ZFlorian Fischer[EchoServers] improve the callback based echoserver* Pass the IO results on the stack instead of storing them in the
client object.
* Terminate the runtime on quit to print stats.
* Free Client objects.* Pass the IO results on the stack instead of storing them in the
client object.
* Terminate the runtime on quit to print stats.
* Free Client objects.https://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/370[tests] Infer test name from first test source file2022-03-30T17:05:41ZFlorian Schmaus[tests] Infer test name from first test source fileThis also fixes a bug in
source = test_dict['source'][0]
so the source(s) used for the test was always only the first source
file. I did/does not matter, as we do not have tests that span
multiple source files (and I am not sure if we ...This also fixes a bug in
source = test_dict['source'][0]
so the source(s) used for the test was always only the first source
file. I did/does not matter, as we do not have tests that span
multiple source files (and I am not sure if we ever will have).https://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/369add assert message in future callback2022-03-30T16:34:53ZFlorian Fischeradd assert message in future callbackAdd a message to help investigate CI failures for single-uring
configurations like:
https://gitlab.cs.fau.de/i4/manycore/emper/-/jobs/574828Add a message to help investigate CI failures for single-uring
configurations like:
https://gitlab.cs.fau.de/i4/manycore/emper/-/jobs/574828https://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/368[meson] Add build_only_emper_dep option2022-03-24T12:36:28ZFlorian Schmaus[meson] Add build_only_emper_dep optionhttps://gitlab.cs.fau.de/i4/manycore/emper/-/merge_requests/367[AbstractWorkStealingScheduler] assert in runtime system in pushBottom2022-03-23T10:55:20ZFlorian Schmaus[AbstractWorkStealingScheduler] assert in runtime system in pushBottom