Towards fixing modernize-avoid-c-arrays, add template_util::getSize()

See merge request i4/manycore/emper!84

[tests] Run tests in parallel if io_uring is disabled

See merge request i4/manycore/emper!85

[IO] fail hard if the globalCompleter would drop a Future

See merge request i4/manycore/emper!83

Improve networking code

See merge request i4/manycore/emper!82

Currently the globalCompleter would drop Futures which he is
unable to submit to its SQ because io_uring_submit returned -EBUSY,
signalizing a full CQ.
For now we fail hard if this hopefully unlikely condition occurs.

Additional small code changes:
* don't wrap assert in if constexpr(DEBUG)
* annotate Future completion in globalCompleter as unlikely
* use prepared Future count as unsigned
* remove redundant parenthesis
* introduce helper function for IoContext* tagging
* introduce Stats::record_io_submit_full_cq default parameter

Use getaddrinfo instead to get the socket information instead of
using always ipv4 sockets.
Pass host and port as strings for use with getaddrinfo.

Start client Fibers in batches to make sure we don't exceed the server's
TCP listen backlog.

Add support for open/openat and a simple fixed string search utility

See merge request i4/manycore/emper!77

Florian Fischer authored 4 years ago and Florian Schmaus committed 4 years ago

Modified-by: Florian Schmaus <flow@cs.fau.de>

Florian Fischer authored 4 years ago and Florian Schmaus committed 4 years ago

Modified-by: Florian Schmaus <flow@cs.fau.de>

[Makefile][CI] Add static-analysis Make target, and gitlab-ci stage

See merge request i4/manycore/emper!81

The 'static-analysis' Make target runs all static analysis we
currently have. This allows to run at least static analysis, in cases
where we are able build a particular EMPER configuration, but no able
to execute it (e.g. because the kernel lacks io_uring support).

Remove linux time types include

See merge request i4/manycore/emper!80

Florian Fischer authored 4 years ago and Florian Schmaus committed 4 years ago

On our bigboxes the build failed because both liburing.h and
linux/time_types.h define struct __kernel_time_spec.
The linux headers were added because of iwyu and are not unnecessary.

Modified-by: Florian Schmaus <flow@cs.fau.de>

fix build with locked work-stealing queues and add test

See merge request i4/manycore/emper!79

Improve default read/write offset

See merge request i4/manycore/emper!78

[test] fix occasional failures of SimpleDiskAndNetworkTest

See merge request i4/manycore/emper!76

Remove feature_flags from test and introduce io test runner

Closes #10

See merge request i4/manycore/emper!75

Add a warning when the kernel does not support IORING_FEAT_NODROP
that maybe IO request are getting lost.

To skip test if emper::IO is false the new io test runner in
tests/test-runner is introduced which checks emper::IO and skips the
test otherwise executes our normal testMain function.

Fixes #10.

The current default offset is 0 which means read/write will not
work like POSIX read/write and update the file position and instead will
always work at the beginning of the file.

Since linux v5.5 (ba04291eb66ed895f194ae5abd3748d72bf8aaea) io_uring
supports offset=-1 which behaves like pvread2/pvwrite2.
This feature is reported by IORING_FEAT_RW_CUR_POS.

By using the default offset -1 emper::io::readFile and emper::io::writeFile
will work like write/read but does also support pwrite/pread.

Concurrently connecting and creating the server socket is racy.
Now the server socket is created before any Fiber is started
removing this race.
Note: The same race is theoretically in SimpleNetworkTest and
ConcurrenctEchoTest too but I haven't observed the failure yet

[gitlab-ci] disable IO since io_uring is not available in our CI

See merge request i4/manycore/emper!74

[clang-tidy] Set HeaderFilterRegex to '.*'

See merge request i4/manycore/emper!73

This matches now all headers in the repository, instead of just the
ones in the emper/ subdirectory, for example the
/tests/fixtures.network.hpp header.

Minor fixes

See merge request i4/manycore/emper!72

Both are semantically equivalent, and it appears that "enum struct" is
a little bit more confusing.

The operation is called 'dequeue', 'deque' is a double-ended queue.

Worker exclusive uring

See merge request i4/manycore/emper!54

The echo client established X connections and start the
echo phase after all sockets are connected.

Each client Fiber measures the time from sending the message until
receiving the echo.

Empers IO design is based on a proactor pattern where each worker
can issue IO requests through its exclusive IoContext object which wraps an
io_uring instance.

IO completions are reaped at 4 places:
1. After a submit to collect inline completions
2. Before dispatching a new Fiber
3. When no new IO can be submitted because the completion queue is full
4. And by a global completer thread which gets notified about completions
   on worker IoContexts through registered eventfds

All IO requests are modeled as Future objects which can be either
instantiated and submitted manually, retrieved by POSIX-like non-blocking
or implicitly used by posix-like blocking functions.

User facing API is exported in the following headers:
* emper/io.hpp (POSIX-like)
* emper.h (POSIX-like)
* emper/io/Future.hpp

Catching short write/reads/sends and resubmitting the request without
unblocking the Fiber is supported.

Using AlarmFuture objects Fibers have a emper-native way to sleep for
a given time.

IO request timeouts with TimeoutWrapper class.
Request Cancellation is supported with Future::cancel() or the
CancelWrapper() Future class.

A proactor design demands that buffers are committed to the kernel
as long as the request is active. To guaranty memory safety Futures
get canceled in their Destructor which will only return after the committed
memory is free to use.

Linking Futures to chains is supported using the Future::SetDependency()
method. Future are submitted when their last Future gets submitted.
A linked Request will start if the previous has finished.
Error or partial completions will cancel the not started tail of a chain.

TODO: Handle possible situations where the CQ of the global completer is full
and no more sqe can be submitted to the SQ.