public inbox for [email protected]
 help / color / mirror / Atom feed
From: Jens Axboe <[email protected]>
To: Keith Busch <[email protected]>
Cc: Kanchan Joshi <[email protected]>,
	[email protected], [email protected],
	[email protected], [email protected],
	[email protected], [email protected], [email protected]
Subject: Re: [PATCH 0/4] iopoll support for io_uring/nvme passthrough
Date: Fri, 5 Aug 2022 12:15:24 -0600	[thread overview]
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>

On 8/5/22 12:11 PM, Keith Busch wrote:
> On Fri, Aug 05, 2022 at 11:18:38AM -0600, Jens Axboe wrote:
>> On 8/5/22 11:04 AM, Jens Axboe wrote:
>>> On 8/5/22 9:42 AM, Kanchan Joshi wrote:
>>>> Hi,
>>>>
>>>> Series enables async polling on io_uring command, and nvme passthrough
>>>> (for io-commands) is wired up to leverage that.
>>>>
>>>> 512b randread performance (KIOP) below:
>>>>
>>>> QD_batch    block    passthru    passthru-poll   block-poll
>>>> 1_1          80        81          158            157
>>>> 8_2         406       470          680            700
>>>> 16_4        620       656          931            920
>>>> 128_32      879       1056        1120            1132
>>>
>>> Curious on why passthru is slower than block-poll? Are we missing
>>> something here?
>>
>> I took a quick peek, running it here. List of items making it slower:
>>
>> - No fixedbufs support for passthru, each each request will go through
>>   get_user_pages() and put_pages() on completion. This is about a 10%
>>   change for me, by itself.
> 
> Enabling fixed buffer support through here looks like it will take a
> little bit of work. The driver needs an opcode or flag to tell it the
> user address is a fixed buffer, and io_uring needs to export its
> registered buffer for a driver like nvme to get to.

Yeah, it's not a straight forward thing. But if this will be used with
recycled buffers, then it'll definitely be worthwhile to look into.

>> - nvme_uring_cmd_io() -> nvme_alloc_user_request() -> blk_rq_map_user()
>>   -> blk_rq_map_user_iov() -> memset() is another ~4% for me.
> 
> Where's the memset() coming from? That should only happen if we need
> to bounce, right? This type of request shouldn't need that unless
> you're using odd user address alignment.

Not sure, need to double check! Hacking up a patch to get rid of the
frivolous alloc+free, we'll see how it stands after that and I'll find
it.

-- 
Jens Axboe


  reply	other threads:[~2022-08-05 18:15 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <CGME20220805155300epcas5p1b98722e20990d0095238964e2be9db34@epcas5p1.samsung.com>
2022-08-05 15:42 ` [PATCH 0/4] iopoll support for io_uring/nvme passthrough Kanchan Joshi
     [not found]   ` <CGME20220805155304epcas5p1bb687a8f9b25317af39def01696626e8@epcas5p1.samsung.com>
2022-08-05 15:42     ` [PATCH 1/4] fs: add file_operations->uring_cmd_iopoll Kanchan Joshi
     [not found]   ` <CGME20220805155307epcas5p4bab3f05dc13d8fc2f03c7a26e9bd8c7c@epcas5p4.samsung.com>
2022-08-05 15:42     ` [PATCH 2/4] io_uring: add iopoll infrastructure for io_uring_cmd Kanchan Joshi
     [not found]   ` <CGME20220805155310epcas5p2bd7ec5b9bee73893958f4bc84038eca0@epcas5p2.samsung.com>
2022-08-05 15:42     ` [PATCH 3/4] block: export blk_rq_is_poll Kanchan Joshi
     [not found]   ` <CGME20220805155313epcas5p2d35d22831bd07ef33fbdc28bd99ae1d0@epcas5p2.samsung.com>
2022-08-05 15:42     ` [PATCH 4/4] nvme: wire up async polling for io passthrough commands Kanchan Joshi
2022-08-05 17:03       ` Jens Axboe
2022-08-05 17:07         ` Kanchan Joshi
2022-08-05 21:22       ` kernel test robot
2022-08-06  0:06       ` kernel test robot
2022-08-06  1:38       ` kernel test robot
2022-08-07 12:25       ` kernel test robot
2022-08-05 17:04   ` [PATCH 0/4] iopoll support for io_uring/nvme passthrough Jens Axboe
2022-08-05 17:13     ` Kanchan Joshi
2022-08-05 17:27       ` Jens Axboe
2022-08-05 17:18     ` Jens Axboe
2022-08-05 18:11       ` Keith Busch
2022-08-05 18:15         ` Jens Axboe [this message]
2022-08-07 17:58           ` Kanchan Joshi
2022-08-07 18:46             ` Jens Axboe
2022-08-05 18:21       ` Jens Axboe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox