Skip to content

fix Future::cancel with new Scheduler::scheduleOn(fiber, workerId)

Florian Fischer requested to merge aj46ezos/emper:scheduleOn into master

This function is needed to deal with worker local resources: io_uring requests for example.

Each worker now always has a MPSC inbox queue which was already used in the laws scheduling strategy. Fibers can be scheduled to a specific worker using the new Scheduler::scheduleOn method.

When using MPSC queues which are only accessed by a single worker we have to ensure that work scheduled to an inbox is retrieved and handled. Therefore a sleeping worker must be notified if work is scheduled to its inbox. To do so we pass a FiberHint through the Runtime's onNewWork notification callbacks. And in the runtime we decide if we have to notify a specific worker or anyone as before.

The sleep strategies must support notifySpecific to allow those new inbox queues. Our SemaphoreSleepStrategy has a naive but generic implementation which is now always active if the used semaphore implementation does not provide notify_specific.

The new SpuriousFutex2Semaphore uses futex_waitv(2) introduced in linux v5.16 to support a more efficient way to implement notify_specific. This is achieved by using a second worker exclusive futex to wake or prevent a specific worker from sleeping.

Implement notifySpecific for the PipeSleepStrategy by introducing a worker exclusive state and pipe.

Fix Future::cancel() by submitting the cancel request on the same worker which submitted the future to cancel.

TODO:

  • Add notifySpecific support to PipeSleepStrategy
  • Fix ScheduleOnTest using SemaphoreSleepStrategy generic notifySpecific implementation
  • Fix SemaphoreSleepStrategy::notifySpecific. The current implementation can sleep lock if all workers try to notify someone and awaiting a state change that will never happen.
  • Fix the sleep lock introduced when a request can not be found and thus is not canceled. This is a kernel bug and fixes are scheduled for linux 5.17.

Fixes #31 (closed).

Edited by Florian Fischer

Merge request reports