Re: Waiting for requests completions from multiple threads

[Jens Axboe <axboe@kernel.dk>]

On 1/21/20 7:45 PM, Dmitry Sychov wrote:
> Really nice work, I have a question though.
> 
> It is possible to efficiently wait for request completions
> from multiple threads?
> 
> Like, two threads are entering
> " io_uring_enter" both with min_complete=1 while the completion ring
> holds 2 events - will the first one goes to thread 1 and the second
> one to thread 2?
> 
> I just do not understand exactly the best way to scale this api into
> multiple threads... with IOCP for example is is perfectly clear.

You can have two threads waiting on events, and yes, if they each ask to
wait for 1 event and 2 completes, then they will both get woken up. But
the wait side doesn't give you any events, it merely tells you of the
availability of them. When each thread is woken up and goes back to
userspace, it'll have to reap an event from the ring. If each thread
reaps one event from the CQ ring, then you're done.

You need synchronization on the CQ ring side in userspace if you want
two rings to access the CQ ring. That is not needed for entering the
kernel, only when the application reads a CQE (or modifies the ring), if
you can have more than one thread modifying the CQ ring. The exact same
is true on the SQ ring side.

-- 
Jens Axboe