public inbox for [email protected]
 help / color / mirror / Atom feed
From: Jens Axboe <[email protected]>
To: Andres Freund <[email protected]>
Cc: [email protected], [email protected],
	[email protected], [email protected],
	[email protected]
Subject: Re: [PATCHSET v5 0/12] Add support for async buffered reads
Date: Fri, 5 Jun 2020 08:42:28 -0600	[thread overview]
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>

On 6/3/20 7:04 PM, Jens Axboe wrote:
> On 6/3/20 6:59 PM, Andres Freund wrote:
>> Hi,
>>
>> I was trying to benchmark the benefits of this for the io_uring using
>> postgres I am working on. The initial results where quite promising
>> (reducing cpu usage significantly, to lower than non-uring sync io). But
>> unfortunately trying another workload triggered both panics and before
>> that seemingly returned wrong data.
>>
>> I first saw that problem with b360d424ce02, which was
>> linux-block/async-buffered.6 at the time. After hitting the issue, I
>> updated to the current linux-block/async-buffered.6, but the problem
>> persists.
>>
>> The workload that triggers the bug within a few seconds is postgres
>> doing a parallel sequential scan of a large table (and aggregating the
>> data, but that shouldn't matter). In the triggering case that boils down
>> to 9 processes sequentially reading a number of 1GB files (we chunk
>> tables internally into smaller files). Each process will read a 512kB
>> chunk of the file on its own, and then claim the next 512kB from a
>> shared memory location. Most of the IO will be READV requests, reading
>> 16 * 8kB into postgres' buffer pool (which may or may not be neighboring
>> 8kB pages).
> 
> I'll try and reproduce this, any chance you have a test case that can
> be run so I don't have to write one from scratch? The more detailed
> instructions the better.

Can you try with async-buffered.7? I've rebased it on a new mechanism,
and doing something like what you describe above I haven't been able
to trigger anything bad. I'd try your test case specifically, so do let
know if it's something I can run.

-- 
Jens Axboe


  parent reply	other threads:[~2020-06-05 14:42 UTC|newest]

Thread overview: 63+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-05-26 19:51 [PATCHSET v5 0/12] Add support for async buffered reads Jens Axboe
2020-05-26 19:51 ` [PATCH 01/12] block: read-ahead submission should imply no-wait as well Jens Axboe
2020-05-26 19:51 ` [PATCH 02/12] mm: allow read-ahead with IOCB_NOWAIT set Jens Axboe
2020-05-26 20:45   ` Johannes Weiner
2020-05-26 19:51 ` [PATCH 03/12] mm: abstract out wake_page_match() from wake_page_function() Jens Axboe
2020-05-26 21:02   ` Johannes Weiner
2020-06-01 14:16   ` Matthew Wilcox
2020-05-26 19:51 ` [PATCH 04/12] mm: add support for async page locking Jens Axboe
2020-05-26 21:59   ` Johannes Weiner
2020-05-26 22:01     ` Jens Axboe
2020-05-27 16:02       ` Johannes Weiner
2020-06-01 14:26   ` Matthew Wilcox
2020-06-01 17:15     ` Jens Axboe
2020-05-26 19:51 ` [PATCH 05/12] mm: support async buffered reads in generic_file_buffered_read() Jens Axboe
2020-05-26 22:00   ` Johannes Weiner
2020-05-26 19:51 ` [PATCH 06/12] fs: add FMODE_BUF_RASYNC Jens Axboe
2020-05-26 19:51 ` [PATCH 07/12] ext4: flag as supporting buffered async reads Jens Axboe
2020-05-26 19:51 ` [PATCH 08/12] block: flag block devices as supporting IOCB_WAITQ Jens Axboe
2020-05-26 19:51 ` [PATCH 09/12] xfs: flag files as supporting buffered async reads Jens Axboe
2020-05-28 17:53   ` Darrick J. Wong
2020-05-28 19:23     ` Jens Axboe
2020-05-26 19:51 ` [PATCH 10/12] btrfs: " Jens Axboe
2020-05-26 19:57   ` Chris Mason
2020-05-26 19:51 ` [PATCH 11/12] mm: add kiocb_wait_page_queue_init() helper Jens Axboe
2020-05-26 22:01   ` Johannes Weiner
2020-05-26 19:51 ` [PATCH 12/12] io_uring: support true async buffered reads, if file provides it Jens Axboe
     [not found] ` <CA+icZUWfX+QmroE6j74C7o-BdfMF5=6PdYrA=5W_JCKddqkJgQ@mail.gmail.com>
2020-05-28 17:06   ` [PATCHSET v5 0/12] Add support for async buffered reads Jens Axboe
2020-05-28 17:12     ` Sedat Dilek
2020-05-28 17:14       ` Jens Axboe
2020-05-28 18:20         ` Sedat Dilek
2020-05-29 10:02     ` Sedat Dilek
2020-05-29 11:22       ` Sedat Dilek
2020-05-30 13:36         ` Sedat Dilek
2020-05-30 18:57           ` Sedat Dilek
2020-05-31  1:57             ` Jens Axboe
     [not found]               ` <CA+icZUXxmOA-5+dukCgxfSp4eVHB+QaAHO6tsgq0iioQs3Af-w@mail.gmail.com>
2020-05-31  7:12                 ` Sedat Dilek
2020-06-01 13:35                   ` Sedat Dilek
2020-06-01 14:04                     ` Jens Axboe
2020-06-01 14:13                       ` Sedat Dilek
2020-06-01 14:14                         ` Jens Axboe
2020-06-01 14:35                           ` Jens Axboe
2020-06-01 14:43                             ` Sedat Dilek
2020-06-01 14:46                               ` Jens Axboe
2020-06-01 14:51                                 ` Sedat Dilek
2020-06-01 20:18                                 ` Sedat Dilek
     [not found] ` <[email protected]>
2020-06-04  1:04   ` Jens Axboe
2020-06-04  1:30     ` Andres Freund
2020-06-05 19:56       ` Andres Freund
2020-06-05 14:42     ` Jens Axboe [this message]
2020-06-05 20:20       ` Andres Freund
2020-06-05 20:21         ` Jens Axboe
2020-06-05 20:36         ` Andres Freund
2020-06-05 20:53           ` Jens Axboe
2020-06-05 21:13             ` Jens Axboe
2020-06-05 21:21               ` Jens Axboe
2020-06-05 22:30                 ` Andres Freund
2020-06-05 22:36                   ` Andres Freund
2020-06-05 22:49                     ` Jens Axboe
2020-06-05 22:54                       ` Andres Freund
2020-06-05 22:56                         ` Jens Axboe
2020-06-05 23:02                           ` Andres Freund
2020-06-06  0:33                 ` Sedat Dilek
2020-06-06 16:04                   ` Jens Axboe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox