Re: Non sequential linked chains and IO_LINK support

[Pavel Begunkov <asml.silence@gmail.com>]

On 07/05/2020 00:46, Bhatia, Sumeet wrote:
> Hello everyone,
> 
> I've been exploring iouring to submit disk operations. My application generates disk operations based on some events and operations are unknown until those events occur.  Some of these disk operations are interdependent others are not. 
> 
> Example: Following operations are generated and submitted before any of them are complete
> operation_0 (independent operation)
> operation_1 (independent operation),​
> operation_2 (to be issued only if operation_0 was successful),
> operation_3 (independent operation),
> operation_4 (to be issued only if operation_1 was successful)
> 
> In my example I have two independent link chains, (operation_0, operation_2) and (operation_1, operation_4).  iouring documentation suggests IOSQE_IO_LINK expects link chains to be sequential and will not support my use case. 

First of all, there shouldn't be a submission (i.e. io_uring_enter(to_submit>0))
between adding linked requests to a submission queue (SQ). It'd be racy otherwise.

E.g. you can't do:

add_sqe(op0)
submit(op0)
add_sqe(op2, linked)

Though the following is valid, as we don't submit op0:

add_sqe(opX)
add_sqe(op0)
submit(up until opX)
add_sqe(op2, linked)


And that means you can reorder them just before submitting, or filing them into
the SQ in a better order.

Is it helpful? Let's figure out how to cover your case.


> I explored creating new iouring context for each of these linked chains. But it turns out depending on disk size there can be somewhere between 500-1000 such chains. I'm not sure whether it is prudent to create that many iouring contexts.

Then you would need to wait on them (e.g. epoll or 1000 threads), and that would
defeat the whole idea. In any case even with sharing io-wq and having small CQ
and SQ, it'd be wasteful keeping many resources duplicated.

> 
> I am reaching out to check whether there would be a generic need to support nonsequential linked chains on a single iouring context. Would love to hear all your thoughts.
> 
> Thanks,
> Sumeet
> 

-- 
Pavel Begunkov