From: Jens Axboe <[email protected]>
To: Keith Busch <[email protected]>
Cc: Kanchan Joshi <[email protected]>,
[email protected], [email protected],
[email protected], [email protected],
[email protected], [email protected], [email protected]
Subject: Re: [PATCH 0/4] iopoll support for io_uring/nvme passthrough
Date: Fri, 5 Aug 2022 12:15:24 -0600 [thread overview]
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
On 8/5/22 12:11 PM, Keith Busch wrote:
> On Fri, Aug 05, 2022 at 11:18:38AM -0600, Jens Axboe wrote:
>> On 8/5/22 11:04 AM, Jens Axboe wrote:
>>> On 8/5/22 9:42 AM, Kanchan Joshi wrote:
>>>> Hi,
>>>>
>>>> Series enables async polling on io_uring command, and nvme passthrough
>>>> (for io-commands) is wired up to leverage that.
>>>>
>>>> 512b randread performance (KIOP) below:
>>>>
>>>> QD_batch block passthru passthru-poll block-poll
>>>> 1_1 80 81 158 157
>>>> 8_2 406 470 680 700
>>>> 16_4 620 656 931 920
>>>> 128_32 879 1056 1120 1132
>>>
>>> Curious on why passthru is slower than block-poll? Are we missing
>>> something here?
>>
>> I took a quick peek, running it here. List of items making it slower:
>>
>> - No fixedbufs support for passthru, each each request will go through
>> get_user_pages() and put_pages() on completion. This is about a 10%
>> change for me, by itself.
>
> Enabling fixed buffer support through here looks like it will take a
> little bit of work. The driver needs an opcode or flag to tell it the
> user address is a fixed buffer, and io_uring needs to export its
> registered buffer for a driver like nvme to get to.
Yeah, it's not a straight forward thing. But if this will be used with
recycled buffers, then it'll definitely be worthwhile to look into.
>> - nvme_uring_cmd_io() -> nvme_alloc_user_request() -> blk_rq_map_user()
>> -> blk_rq_map_user_iov() -> memset() is another ~4% for me.
>
> Where's the memset() coming from? That should only happen if we need
> to bounce, right? This type of request shouldn't need that unless
> you're using odd user address alignment.
Not sure, need to double check! Hacking up a patch to get rid of the
frivolous alloc+free, we'll see how it stands after that and I'll find
it.
--
Jens Axboe
next prev parent reply other threads:[~2022-08-05 18:15 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <CGME20220805155300epcas5p1b98722e20990d0095238964e2be9db34@epcas5p1.samsung.com>
2022-08-05 15:42 ` [PATCH 0/4] iopoll support for io_uring/nvme passthrough Kanchan Joshi
[not found] ` <CGME20220805155304epcas5p1bb687a8f9b25317af39def01696626e8@epcas5p1.samsung.com>
2022-08-05 15:42 ` [PATCH 1/4] fs: add file_operations->uring_cmd_iopoll Kanchan Joshi
[not found] ` <CGME20220805155307epcas5p4bab3f05dc13d8fc2f03c7a26e9bd8c7c@epcas5p4.samsung.com>
2022-08-05 15:42 ` [PATCH 2/4] io_uring: add iopoll infrastructure for io_uring_cmd Kanchan Joshi
[not found] ` <CGME20220805155310epcas5p2bd7ec5b9bee73893958f4bc84038eca0@epcas5p2.samsung.com>
2022-08-05 15:42 ` [PATCH 3/4] block: export blk_rq_is_poll Kanchan Joshi
[not found] ` <CGME20220805155313epcas5p2d35d22831bd07ef33fbdc28bd99ae1d0@epcas5p2.samsung.com>
2022-08-05 15:42 ` [PATCH 4/4] nvme: wire up async polling for io passthrough commands Kanchan Joshi
2022-08-05 17:03 ` Jens Axboe
2022-08-05 17:07 ` Kanchan Joshi
2022-08-05 21:22 ` kernel test robot
2022-08-06 0:06 ` kernel test robot
2022-08-06 1:38 ` kernel test robot
2022-08-07 12:25 ` kernel test robot
2022-08-05 17:04 ` [PATCH 0/4] iopoll support for io_uring/nvme passthrough Jens Axboe
2022-08-05 17:13 ` Kanchan Joshi
2022-08-05 17:27 ` Jens Axboe
2022-08-05 17:18 ` Jens Axboe
2022-08-05 18:11 ` Keith Busch
2022-08-05 18:15 ` Jens Axboe [this message]
2022-08-07 17:58 ` Kanchan Joshi
2022-08-07 18:46 ` Jens Axboe
2022-08-05 18:21 ` Jens Axboe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox