Skip to content
GitLab
Projects Groups Topics Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • E emper
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributor statistics
    • Graph
    • Compare revisions
  • Issues 15
    • Issues 15
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 12
    • Merge requests 12
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Artifacts
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • i4i4
  • manycore
  • emper
  • Merge requests
  • !260

[RFC] IO-stealing analogue to work-stealing

  • Review changes

  • Download
  • Patches
  • Plain diff
Merged Florian Fischer requested to merge aj46ezos/emper:io-stealing into master Sep 23, 2021
  • Overview 2
  • Commits 6
  • Pipelines 28
  • Changes 25

IO stealing is analog to work-stealing and means that worker thread without work will try to steal IO completions (CQEs) from other worker's IoContexts. The work stealing algorithm is modified to check a victims CQ after findig their work queue empty.

This approach in combination with future additions (global notifications on IO completions, and lock free CQE consumption) are a realistic candidate to replace the completer thread without loosing its benefits.

To allow IO stealing the CQ must be synchronized which is already the case with the IoContext::cq_lock. Currently stealing workers always try to pop a single CQE (this could be configurable). Steal attempts are recorded in the IoContext's Stats object and successfully stolen IO continuations in the AbstractWorkStealingWorkerStats.

I moved the code transforming CQEs into continuation Fibers from reapCompletions into a seperate function to make the rather complicated function more readable and thus easier to understand.

Remove the default CallerEnvironment template arguments to make the code more explicit and prevent easy errors (not propagating the caller environment or forgetting the function takes a caller environment).

io::Stats now need to use atomics because multiple thread may increment them in parallel from EMPER and the OWNER. And since using std::atomic<T*> in std::map is not easily possible we use the compiler __atomic_* builtins.

Add, adjust and fix some comments.

Edited Sep 29, 2021 by Florian Fischer
Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: io-stealing