Fix fsearch by using a sharded file buffer

Each worker thread has it own stringstream per ShardedFileBuffer. All streams are joined and flushed to the file when a ShardedFileBuffer is destructed.

Flushing a ShardedFileBuffer is not thread safe and must not be called concurrently to any modifications to the ShardedFileBuffer.

Use a ShardedFileBuffer to acumulate and print all paths during fsearch.

Closes !351 (closed).

added 1 commit

• 65c8d451 - implement a sharded file buffer

Compare with previous version

added 1 commit

• 5678cc2f - implement a sharded file buffer

Compare with previous version

added 1 commit

• 9cd137e3 - implement a sharded file buffer

Compare with previous version

added 1 commit

• 2531492e - implement a sharded file buffer

Compare with previous version

I prefer this over !351 (closed) even if !351 (closed) is way simpler. Because using buffered results gathering seams to be faster:

Results of the emper-fs-eval (e7d010e)
On faui49big02 running linux 5.16.0 with Jens' io_uring-5.17 patches.

# Haystack on SSD
$ ./eval.py -r 5 -i 'emper-*-vanilla' -hs /srv/scratch/aj46ezos/emper-fs-eval/haystack/
duration_time:u:
median:
        emper-tracked-vanilla-median: 2.353944618
        emper-buffered-vanilla-median: 2.341981383
mean:
        emper-tracked-vanilla-mean: 2.3665241451999997
        emper-buffered-vanilla-mean: 2.346599383

# Haystack on NFS
$ ./eval.py -r 5 -i 'emper-*-vanilla'
duration_time:u:
median:
        emper-tracked-vanilla-median: 2.005457338
        emper-buffered-vanilla-median: 1.950769338
mean:
        emper-tracked-vanilla-mean: 2.0048716464
        emper-buffered-vanilla-mean: 1.9655432080000002

emper-buffered-vanilla is EMPER build with default arguments on !352 (merged).

emper-tracked-vanilla is EMPER build with default arguments on !351 (closed).

And the ShardedFileBuffer code may be useful for other things to.

changed the description

merged

mentioned in commit 31129496