public inbox for [email protected]
 help / color / mirror / Atom feed
From: Jens Axboe <[email protected]>
To: Miklos Szeredi <[email protected]>
Cc: [email protected]
Subject: Re: io_uring_prep_openat_direct() and link/drain
Date: Thu, 21 Apr 2022 06:41:32 -0600	[thread overview]
Message-ID: <[email protected]> (raw)
In-Reply-To: <CAJfpegvTPc0DR5z80kB6uq=-nMa=+4uxGUqbxiGcOTUiVrR+wg@mail.gmail.com>

On 4/21/22 6:39 AM, Miklos Szeredi wrote:
> On Thu, 21 Apr 2022 at 14:34, Jens Axboe <[email protected]> wrote:
>>
>> On 4/21/22 6:31 AM, Miklos Szeredi wrote:
>>> On Tue, 5 Apr 2022 at 16:44, Jens Axboe <[email protected]> wrote:
>>>>
>>>> On 4/5/22 1:45 AM, Miklos Szeredi wrote:
>>>>> On Sat, 2 Apr 2022 at 03:17, Jens Axboe <[email protected]> wrote:
>>>>>>
>>>>>> On 4/1/22 10:21 AM, Jens Axboe wrote:
>>>>>>> On 4/1/22 10:02 AM, Miklos Szeredi wrote:
>>>>>>>> On Fri, 1 Apr 2022 at 17:36, Jens Axboe <[email protected]> wrote:
>>>>>>>>
>>>>>>>>> I take it you're continually reusing those slots?
>>>>>>>>
>>>>>>>> Yes.
>>>>>>>>
>>>>>>>>>  If you have a test
>>>>>>>>> case that'd be ideal. Agree that it sounds like we just need an
>>>>>>>>> appropriate breather to allow fput/task_work to run. Or it could be the
>>>>>>>>> deferral free of the fixed slot.
>>>>>>>>
>>>>>>>> Adding a breather could make the worst case latency be large.  I think
>>>>>>>> doing the fput synchronously would be better in general.
>>>>>>>
>>>>>>> fput() isn't sync, it'll just offload to task_work. There are some
>>>>>>> dependencies there that would need to be checked. But we'll find a way
>>>>>>> to deal with it.
>>>>>>>
>>>>>>>> I test this on an VM with 8G of memory and run the following:
>>>>>>>>
>>>>>>>> ./forkbomb 14 &
>>>>>>>> # wait till 16k processes are forked
>>>>>>>> for i in `seq 1 100`; do ./procreads u; done
>>>>>>>>
>>>>>>>> You can compare performance with plain reads (./procreads p), the
>>>>>>>> other tests don't work on public kernels.
>>>>>>>
>>>>>>> OK, I'll check up on this, but probably won't have time to do so before
>>>>>>> early next week.
>>>>>>
>>>>>> Can you try with this patch? It's not complete yet, there's actually a
>>>>>> bunch of things we can do to improve the direct descriptor case. But
>>>>>> this one is easy enough to pull off, and I think it'll fix your OOM
>>>>>> case. Not a proposed patch, but it'll prove the theory.
>>>>>
>>>>> Sorry for the delay..
>>>>>
>>>>> Patch works like charm.
>>>>
>>>> OK good, then it is the issue I suspected. Thanks for testing!
>>>
>>> Tested with v5.18-rc3 and performance seems significantly worse than
>>> with the test patch:
>>>
>>> test patch:
>>>         avg     min     max     stdev
>>> real    0.205   0.190   0.266   0.011
>>> user    0.017   0.007   0.029   0.004
>>> sys     0.374   0.336   0.503   0.022
>>>
>>> 5.18.0-rc3-00016-gb253435746d9:
>>>         avg     min     max     stdev
>>> real    0.725   0.200   18.090  2.279
>>> user    0.019   0.005   0.046   0.006
>>> sys     0.454   0.241   1.022   0.199
>>
>> It's been a month and I don't remember details of which patches were
>> tested, when you say "test patch", which one exactly are you referring
>> to and what base was it applied on?
> 
> https://lore.kernel.org/all/[email protected]/
> 
> The base is a good question, it was after the basic fixed slot
> assignment issues were fixed.

Gotcha, ok then this makes sense. The ordering issues were sorted out
for 5.18-rc3, but the direct descriptor optimization is only in the 5.19
branch.

-- 
Jens Axboe


  reply	other threads:[~2022-04-21 12:41 UTC|newest]

Thread overview: 32+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-03-29 13:20 io_uring_prep_openat_direct() and link/drain Miklos Szeredi
2022-03-29 16:08 ` Jens Axboe
2022-03-29 17:04   ` Jens Axboe
2022-03-29 18:21     ` Miklos Szeredi
2022-03-29 18:26       ` Jens Axboe
2022-03-29 18:31         ` Miklos Szeredi
2022-03-29 18:40           ` Jens Axboe
2022-03-29 19:30             ` Miklos Szeredi
2022-03-29 20:03               ` Jens Axboe
2022-03-30  8:18                 ` Miklos Szeredi
2022-03-30 12:35                   ` Jens Axboe
2022-03-30 12:43                     ` Miklos Szeredi
2022-03-30 12:48                       ` Jens Axboe
2022-03-30 12:51                         ` Miklos Szeredi
2022-03-30 14:58                           ` Miklos Szeredi
2022-03-30 15:05                             ` Jens Axboe
2022-03-30 15:12                               ` Miklos Szeredi
2022-03-30 15:17                                 ` Jens Axboe
2022-03-30 15:53                                   ` Jens Axboe
2022-03-30 17:49                                     ` Jens Axboe
2022-04-01  8:40                                       ` Miklos Szeredi
2022-04-01 15:36                                         ` Jens Axboe
2022-04-01 16:02                                           ` Miklos Szeredi
2022-04-01 16:21                                             ` Jens Axboe
2022-04-02  1:17                                               ` Jens Axboe
2022-04-05  7:45                                                 ` Miklos Szeredi
2022-04-05 14:44                                                   ` Jens Axboe
2022-04-21 12:31                                                     ` Miklos Szeredi
2022-04-21 12:34                                                       ` Jens Axboe
2022-04-21 12:39                                                         ` Miklos Szeredi
2022-04-21 12:41                                                           ` Jens Axboe [this message]
2022-04-21 13:10                                                             ` Miklos Szeredi

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox