Re: Sending CQE to a different ring

[Jens Axboe <axboe@kernel.dk>]

On 3/10/22 8:52 AM, Artyom Pavlov wrote:
> 10.03.2022 18:46, Jens Axboe wrote:
>> On 3/10/22 8:43 AM, Jens Axboe wrote:
>>> On 3/10/22 8:36 AM, Artyom Pavlov wrote:
>>>> After thinking about it a bit, I think this approach has one serious
>>>> disadvantage: you lose successful result value of the initial request.
>>>> Imagine we submit IORING_OP_READ and link IORING_OP_WAKEUP_RING to it.
>>>> If the request is completed successfully, both ring1 and ring2 will
>>>> lose number of read bytes.
>>>
>>> But you know what the result is, otherwise you would've gotten a cqe
>>> posted if it wasn't what you asked for.
>>
>> Can also be made more explicit by setting sqe->len to what you set the
>> original request length to, and then pass that back in the cqe->res
>> instead of using the pid from the task that sent it. Then you'd have it
>> immediately available. Might make more sense than the pid, not sure
>> anyone would care about that?
> 
> Maybe I am missing something, but we only know that the request to
> which IORING_OP_WAKEUP_RING was linked completed successfully. How
> exactly do you retrieve the number of read bytes with the linking
> aproach?

Because you'd do:

sqe1 = get_sqe();
prep_sqe(sqe1);
sqe1->len = io_bytes;
sqe1->flags |= IOSQE_IO_LINK | IOSQE_CQE_SKIP_ON_SUCESS;

sqe2 = get_sqe();
prep_msg_ring(sqe2);
sqe2->fd = target_ring_fd;
sqe2->len = io_bytes;
sqe2->flags |= IOSQE_CQE_SKIP_ON_SUCESS;

Then when target_ring gets the cqe for the msg_ring operation, it'll
have sqe->len available. If sqe1 doesn't complete with io_bytes as the
result, the link will be broken and you'll have to handle it locally.
Which should be fine, it's not like you can't handle it locally, you
just prefer to have it handled remotely.

> Yes, passing positive result value would make more sense than PID of
> submitter, which is rarely, if ever, needed. IIUC we would not be able
> to use linking with such approach, since sqe->len has to be set in
> user code based on a received CQE, but I guess it should be fine in
> practice.

Right, and using sqe->len and passing it through makes a lot more sense
in general as you can pass whatever you want there. If you want to use
the pid, you can use it like that. Or for whatever else you'd want. That
gives you both 'len' and 'user_data' as information you can pass between
the rings.

It could also be used as `len` holding a message type, and `user_data`
holding a pointer to a struct. For example.

-- 
Jens Axboe