Re: io_uring_prep_openat_direct() and link/drain

[Jens Axboe <axboe@kernel.dk>]

On 3/29/22 12:21 PM, Miklos Szeredi wrote:
> On Tue, 29 Mar 2022 at 19:04, Jens Axboe <axboe@kernel.dk> wrote:
>>
>> On 3/29/22 10:08 AM, Jens Axboe wrote:
>>> On 3/29/22 7:20 AM, Miklos Szeredi wrote:
>>>> Hi,
>>>>
>>>> I'm trying to read multiple files with io_uring and getting stuck,
>>>> because the link and drain flags don't seem to do what they are
>>>> documented to do.
>>>>
>>>> Kernel is v5.17 and liburing is compiled from the git tree at
>>>> 7a3a27b6a384 ("add tests for nonblocking accept sockets").
>>>>
>>>> Without those flags the attached example works some of the time, but
>>>> that's probably accidental since ordering is not ensured.
>>>>
>>>> Adding the drain or link flags make it even worse (fail in casese that
>>>> the unordered one didn't).
>>>>
>>>> What am I missing?
>>>
>>> I don't think you're missing anything, it looks like a bug. What you
>>> want here is:
>>>
>>> prep_open_direct(sqe);
>>> sqe->flags |= IOSQE_IO_LINK;
>>> ...
>>> prep_read(sqe);
> 
> So with the below merge this works.   But if instead I do
> 
> prep_open_direct(sqe);
>  ...
> prep_read(sqe);
> sqe->flags |= IOSQE_IO_DRAIN;
> 
> than it doesn't.  Shouldn't drain have a stronger ordering guarantee than link?

I didn't test that, but I bet it's running into the same kind of issue
wrt prep. Are you getting -EBADF? The drain will indeed ensure that
_execution_ doesn't start until the previous requests have completed,
but it's still prepared before.

For your use case, IO_LINK is what you want and that must work.

I'll check the drain case just in case, it may in fact work if you just
edit the code base you're running now and remove these two lines from
io_init_req():

if (unlikely(!req->file)) {
-        if (!ctx->submit_state.link.head)
-                return -EBADF;
        req->result = fd;
        req->flags |= REQ_F_DEFERRED_FILE;
}

to not make it dependent on link.head. Probably not a bad idea in
general, as the rest of the handlers have been audited for req->file
usage in prep.

-- 
Jens Axboe