public inbox for [email protected]
 help / color / mirror / Atom feed
From: Dmitry Kadashev <[email protected]>
To: Jens Axboe <[email protected]>
Cc: [email protected]
Subject: Re: io_uring's openat doesn't work with large (2G+) files
Date: Mon, 13 Apr 2020 16:20:15 +0700	[thread overview]
Message-ID: <CAOKbgA6JN4oQzyUo0_2y2KUKGX_xuwmDnQsCCABPq_nxms12Aw@mail.gmail.com> (raw)
In-Reply-To: <[email protected]>

On Thu, Apr 9, 2020 at 10:29 PM Jens Axboe <[email protected]> wrote:
>
> On 4/8/20 8:50 PM, Dmitry Kadashev wrote:
> > On Wed, Apr 8, 2020 at 11:26 PM Jens Axboe <[email protected]> wrote:
> >>
> >> On 4/8/20 9:12 AM, Dmitry Kadashev wrote:
> >>> On Wed, Apr 8, 2020 at 10:49 PM Jens Axboe <[email protected]> wrote:
> >>>>
> >>>> On 4/8/20 8:41 AM, Dmitry Kadashev wrote:
> >>>>> On Wed, Apr 8, 2020 at 10:36 PM Jens Axboe <[email protected]> wrote:
> >>>>>>
> >>>>>> On 4/8/20 8:30 AM, Dmitry Kadashev wrote:
> >>>>>>> On Wed, Apr 8, 2020 at 10:19 PM Jens Axboe <[email protected]> wrote:
> >>>>>>>>
> >>>>>>>> On 4/8/20 7:51 AM, Dmitry Kadashev wrote:
> >>>>>>>>> Hi,
> >>>>>>>>>
> >>>>>>>>> io_uring's openat seems to produce FDs that are incompatible with
> >>>>>>>>> large files (>2GB). If a file (smaller than 2GB) is opened using
> >>>>>>>>> io_uring's openat then writes -- both using io_uring and just sync
> >>>>>>>>> pwrite() -- past that threshold fail with EFBIG. If such a file is
> >>>>>>>>> opened with sync openat, then both io_uring's writes and sync writes
> >>>>>>>>> succeed. And if the file is larger than 2GB then io_uring's openat
> >>>>>>>>> fails right away, while the sync one works.
> >>>>>>>>>
> >>>>>>>>> Kernel versions: 5.6.0-rc2, 5.6.0.
> >>>>>>>>>
> >>>>>>>>> A couple of reproducers attached, one demos successful open with
> >>>>>>>>> failed writes afterwards, and another failing open (in comparison with
> >>>>>>>>> sync  calls).
> >>>>>>>>>
> >>>>>>>>> The output of the former one for example:
> >>>>>>>>>
> >>>>>>>>> *** sync openat
> >>>>>>>>> openat succeeded
> >>>>>>>>> sync write at offset 0
> >>>>>>>>> write succeeded
> >>>>>>>>> sync write at offset 4294967296
> >>>>>>>>> write succeeded
> >>>>>>>>>
> >>>>>>>>> *** sync openat
> >>>>>>>>> openat succeeded
> >>>>>>>>> io_uring write at offset 0
> >>>>>>>>> write succeeded
> >>>>>>>>> io_uring write at offset 4294967296
> >>>>>>>>> write succeeded
> >>>>>>>>>
> >>>>>>>>> *** io_uring openat
> >>>>>>>>> openat succeeded
> >>>>>>>>> sync write at offset 0
> >>>>>>>>> write succeeded
> >>>>>>>>> sync write at offset 4294967296
> >>>>>>>>> write failed: File too large
> >>>>>>>>>
> >>>>>>>>> *** io_uring openat
> >>>>>>>>> openat succeeded
> >>>>>>>>> io_uring write at offset 0
> >>>>>>>>> write succeeded
> >>>>>>>>> io_uring write at offset 4294967296
> >>>>>>>>> write failed: File too large
> >>>>>>>>
> >>>>>>>> Can you try with this one? Seems like only openat2 gets it set,
> >>>>>>>> not openat...
> >>>>>>>
> >>>>>>> I've tried specifying O_LARGEFILE explicitly, that did not change the
> >>>>>>> behavior. Is this good enough? Much faster for me to check this way
> >>>>>>> that rebuilding the kernel. But if necessary I can do that.
> >>>>>>
> >>>>>> Not sure O_LARGEFILE settings is going to do it for x86-64, the patch
> >>>>>> should fix it though. Might have worked on 32-bit, though.
> >>>>>
> >>>>> OK, will test.
> >>>>
> >>>> Great, thanks. FWIW, tested here, and it works for me.
> >>>
> >>> Great, will post results tomorrow.
> >>
> >> Thanks!
> >
> > With the patch applied it works perfectly, thanks.
>
> Thanks for testing!

Can I ask if this is going to be merged into 5.6? Since it's a bug
(important enough from my perspective) in existing logic. Thanks.

-- 
Dmitry

  reply	other threads:[~2020-04-13  9:29 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-04-08 14:51 io_uring's openat doesn't work with large (2G+) files Dmitry Kadashev
2020-04-08 15:19 ` Jens Axboe
2020-04-08 15:30   ` Dmitry Kadashev
2020-04-08 15:36     ` Jens Axboe
2020-04-08 15:41       ` Dmitry Kadashev
2020-04-08 15:49         ` Jens Axboe
2020-04-08 16:12           ` Dmitry Kadashev
2020-04-08 16:26             ` Jens Axboe
2020-04-09  3:50               ` Dmitry Kadashev
2020-04-09 15:29                 ` Jens Axboe
2020-04-13  9:20                   ` Dmitry Kadashev [this message]
2020-04-13 10:09                     ` Pavel Begunkov
2020-04-13 10:19                       ` Dmitry Kadashev

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAOKbgA6JN4oQzyUo0_2y2KUKGX_xuwmDnQsCCABPq_nxms12Aw@mail.gmail.com \
    [email protected] \
    [email protected] \
    [email protected] \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox