GitLab

The fact that we are worse than golang or tokio when the system is not saturated with work in our echo server benchmark bothers me. That's why I experimented with different ways of suspending and reactivating worker threads.

1. Blocking IO: Read and write on an eventfd opened with EFD_SEMAPHORE
2. Signals: sigwait and pthread_kill

I am not convinced that my implementations are sound. And not produce lost wakeups. If we decide to change the synchronization primitive they should be thoroughly checked.

I did 5 runs of the echo test but haven't done any real statistics. Here are the results: wakeup.pdf

connections    	500            	1000           	5000           	10000         	25000
signal         	220.5737 (+8%) 	257.5492 (+14%)	325.7909 (+12%)	313.9826 (+4%)	288.9554 (+8%)
mutex + condvar	202.7931 (0%)  	225.6365 (0%)  	289.4035 (0%)  	300.4161 (0%) 	267.1373 (0%)
eventfd        	162.5336 (-20%)	209.8668 (-7%) 	319.8507 (+7%) 	311.5511 (+3%)	286.3498 (+7%)

My interpretation

• BlockingIO with eventfd is the worst of the three variants.
• Using signal seams interesting.
• 5000 connections and eventfd has a really huge variance (not visible hear) and is not really trustworthy
• I can't really explain why both new variants perform better in a saturated system. I would expect that they do less in onNewWork which is executed in each dispatchLoop iteration. This would also explain why they behave very similarly.