Re: Sending CQE to a different ring

[Jens Axboe <axboe@kernel.dk>]

On 3/9/22 9:03 PM, Jens Axboe wrote:
> I'll mull over this a bit...

One idea... You issue the request as you normally would for ring1, and
you mark that request A with IOSQE_CQE_SKIP_SUCCESS. Then you link an
IORING_OP_WAKEUP_RING to request A, with the fd for it set to ring2, and
also mark that with IOSQE_CQE_SKIP_SUCCESS.

We'd need to have sqe->addr (or whatever field) in the WAKEUP_RING
request be set to the user_data of request A, so we can propagate it.

The end result is that ring1 won't see any completions for request A or
the linked WAKEUP, unless one of them fails. If that happens, then you
get to process things locally still, but given that this is a weird
error case, shouldn't affect things in practice. ring2 will get the CQE
posted once request A has completed, with the user_data filled in from
request A. Which is really what we care about here, as far as I
understand.

This basically works right now with what I posted, and without needing
to rearchitect a bunch of stuff. And it's pretty efficient. Only thing
we'd need to add is passing in the target cqe user_data for the WAKEUP
request. Would just need to be wrapped in something that allows you to
do this easily, as it would be useful for others too potentially.

-- 
Jens Axboe