We don't need to pay the overhead of the atomic operations on each dispatch loop if there is no concurrent access to worker CQs.
