Re: [PATCH 5/6] io_uring: add support for futex wake and wait

[Jens Axboe <axboe@kernel.dk>]

On 6/23/23 1:34?PM, Andres Freund wrote:
> Hi,
> 
> On 2023-06-23 13:07:12 -0600, Jens Axboe wrote:
>> On 6/23/23 1:04?PM, Andres Freund wrote:
>>> Hi,
>>>
>>> I'd been chatting with Jens about this, so obviously I'm interested in the
>>> feature...
>>>
>>> On 2023-06-09 12:31:24 -0600, Jens Axboe wrote:
>>>> Add support for FUTEX_WAKE/WAIT primitives.
>>>>
>>>> IORING_OP_FUTEX_WAKE is mix of FUTEX_WAKE and FUTEX_WAKE_BITSET, as
>>>> it does support passing in a bitset.
>>>>
>>>> Similary, IORING_OP_FUTEX_WAIT is a mix of FUTEX_WAIT and
>>>> FUTEX_WAIT_BITSET.
>>>
>>> One thing I was wondering about is what happens when there are multiple
>>> OP_FUTEX_WAITs queued for the same futex, and that futex gets woken up. I
>>> don't really have an opinion about what would be best, just that it'd be
>>> helpful to specify the behaviour.
>>
>> Not sure I follow the question, can you elaborate?
>>
>> If you have N futex waits on the same futex and someone does a wait
>> (with wakenum >= N), then they'd all wake and post a CQE. If less are
>> woken because the caller asked for less than N, than that number should
>> be woken.
>>
>> IOW, should have the same semantics as "normal" futex waits.
> 
> With a normal futex wait you can't wait multiple times on the same futex in
> one thread. But with the proposed io_uring interface, one can.

Right, but you could have N threads waiting on the same futex.

> Basically, what is the defined behaviour for:
> 
>    sqe = io_uring_get_sqe(ring);
>    io_uring_prep_futex_wait(sqe, futex, 0, FUTEX_BITSET_MATCH_ANY);
> 
>    sqe = io_uring_get_sqe(ring);
>    io_uring_prep_futex_wait(sqe, futex, 0, FUTEX_BITSET_MATCH_ANY);
> 
>    io_uring_submit(ring)
> 
> when someone does:
>    futex(FUTEX_WAKE, futex, 1, 0, 0, 0);
>    or
>    futex(FUTEX_WAKE, futex, INT_MAX, 0, 0, 0);
> 
> or the equivalent io_uring operation.

Waking with num=1 should wake just one of them, which one is really down
to the futex ordering which depends on task priority (which would be the
same here), and ordered after that. So first one should wake the first
sqe queued.

Second one will wake all of them, in that order.

I'll put that in the the test case.

> Is it an error? Will there always be two cqes queued? Will it depend
> on the number of wakeups specified by the waker?  I'd assume the
> latter, but it'd be good to specify that.

It's not an error, I would not want to police that. It will purely
depend on the number of wakes specified by the wake operation. If it's
1, one will be triggered. If it's INT_MAX, then both of them will
trigger. First case will generate one CQE, second one will generate both
CQEs.

No documentation has been written for the io_uring bits yet. But the
current branch is ready for wider posting, so I should get that written
up too...

-- 
Jens Axboe