Re: Non sequential linked chains and IO_LINK support

["Bhatia, Sumeet" <sumee@amazon.com>]

Thank you for the response!

Unfortunately when the application submits operation_0 it has no way of determining if/when operation_2 would be generated. 

For now I plan to maintain a list of outstanding operations. If operation_2 gets generated while operation_0 is in flight the application will hold its submission until operation_0 is completed. 

I wanted to check whether this would be a generic use case and would warrant native support in iouring?

Thanks,
Sumeet
________________________________________
From: Pavel Begunkov <asml.silence@gmail.com>
Sent: Wednesday, May 6, 2020 6:11 PM
To: Bhatia, Sumeet; io-uring@vger.kernel.org
Cc: Hegde, Pramod; Jens Axboe
Subject: RE: [EXTERNAL] Non sequential linked chains and IO_LINK support

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.



On 07/05/2020 00:46, Bhatia, Sumeet wrote:
> Hello everyone,
>
> I've been exploring iouring to submit disk operations. My application generates disk operations based on some events and operations are unknown until those events occur.  Some of these disk operations are interdependent others are not.
>
> Example: Following operations are generated and submitted before any of them are complete
> operation_0 (independent operation)
> operation_1 (independent operation),​
> operation_2 (to be issued only if operation_0 was successful),
> operation_3 (independent operation),
> operation_4 (to be issued only if operation_1 was successful)
>
> In my example I have two independent link chains, (operation_0, operation_2) and (operation_1, operation_4).  iouring documentation suggests IOSQE_IO_LINK expects link chains to be sequential and will not support my use case.

First of all, there shouldn't be a submission (i.e. io_uring_enter(to_submit>0))
between adding linked requests to a submission queue (SQ). It'd be racy otherwise.

E.g. you can't do:

add_sqe(op0)
submit(op0)
add_sqe(op2, linked)

Though the following is valid, as we don't submit op0:

add_sqe(opX)
add_sqe(op0)
submit(up until opX)
add_sqe(op2, linked)


And that means you can reorder them just before submitting, or filing them into
the SQ in a better order.

Is it helpful? Let's figure out how to cover your case.


> I explored creating new iouring context for each of these linked chains. But it turns out depending on disk size there can be somewhere between 500-1000 such chains. I'm not sure whether it is prudent to create that many iouring contexts.

Then you would need to wait on them (e.g. epoll or 1000 threads), and that would
defeat the whole idea. In any case even with sharing io-wq and having small CQ
and SQ, it'd be wasteful keeping many resources duplicated.

>
> I am reaching out to check whether there would be a generic need to support nonsequential linked chains on a single iouring context. Would love to hear all your thoughts.
>
> Thanks,
> Sumeet
>

--
Pavel Begunkov