public inbox for [email protected]
 help / color / mirror / Atom feed
* [PATCHSET v5 0/6] Enable bio recycling for polled IO
@ 2021-08-12 15:41 Jens Axboe
  2021-08-12 15:41 ` [PATCH 1/6] bio: optimize initialization of a bio Jens Axboe
                   ` (5 more replies)
  0 siblings, 6 replies; 25+ messages in thread
From: Jens Axboe @ 2021-08-12 15:41 UTC (permalink / raw)
  To: io-uring; +Cc: linux-block, hch

Hi,

For v4, see posting here:

https://lore.kernel.org/io-uring/[email protected]/

3.5M+ IOPS per core:

axboe@amd ~/g/fio (master)> sudo taskset -c 0 t/io_uring -b512 -d128 -c32 -s32 -p1 -F1 -B1 /dev/nvme3n1
i 8, argc 9
Added file /dev/nvme3n1 (submitter 0)
sq_ring ptr = 0x0x7fdab1a8d000
sqes ptr    = 0x0x7fdab1a8b000
cq_ring ptr = 0x0x7fdab1a89000
polled=1, fixedbufs=1, register_files=1, buffered=0 QD=128, sq_ring=128, cq_ring=256
submitter=1757
IOPS=3520608, IOS/call=32/31, inflight=47 (47)
IOPS=3514432, IOS/call=32/32, inflight=32 (32)
IOPS=3513440, IOS/call=32/31, inflight=128 (128)
IOPS=3507616, IOS/call=32/32, inflight=32 (32)
IOPS=3505984, IOS/call=32/32, inflight=32 (32)
IOPS=3511328, IOS/call=32/31, inflight=64 (64)
[snip]

Changes can also be bound in my io_uring-bio-cache.5 branch, and sit
on top of for-5.15/io_uring.

 block/bio.c                | 164 +++++++++++++++++++++++++++++++++----
 block/blk-core.c           |   5 +-
 fs/block_dev.c             |   6 +-
 fs/io_uring.c              |   2 +-
 include/linux/bio.h        |  13 +++
 include/linux/blk_types.h  |   1 +
 include/linux/cpuhotplug.h |   1 +
 include/linux/fs.h         |   2 +
 8 files changed, 175 insertions(+), 19 deletions(-)

Changes since v4:

- Kill __bio_init() helper
- Kill __bio_put() helper
- Cleanup bio_alloc_kiocb()
- Expand commit messages
- Various little tweaks
- Add kerneldoc for bio_alloc_kiocb()
- Fix attribution on last patch
- Remove bio.h list manipulation leftovers
- Move cpuhp_dead notifier to end of bio_set
- Rebase on for-5.15/io_uring

-- 
Jens Axboe



^ permalink raw reply	[flat|nested] 25+ messages in thread
* [PATCHSET v4 0/6] Enable bio recycling for polled IO
@ 2021-08-11 19:35 Jens Axboe
  2021-08-11 19:35 ` [PATCH 3/6] bio: add allocation cache abstraction Jens Axboe
  0 siblings, 1 reply; 25+ messages in thread
From: Jens Axboe @ 2021-08-11 19:35 UTC (permalink / raw)
  To: io-uring; +Cc: linux-block, hch

Hi,

This is v4 of this patchset, and Yet Another method of achieving the
same goal. This one moves into the direction of my old cpu-alloc-cache
branch, where the caches are just per-cpu. The trouble with those is
that we need to make this specific to polled IO to lose the IRQ
safety of them, otherwise it's not a real win and we're better off just
using the slab allocator smarts. This is combined with Christoph's idea
to make it per bio_set, and retains the flagging of the kiocb for
having the IO issuer tell the below layer whether the cache can be
safely used or not.

Another change from last is that we can now grossly simplify the
io_uring side, as we don't need locking for the cache and async retries
are no longer interesting there. This is combined with a block layer
change that clears BIO_PERCPU_CACHE if we clear the HIPRI flag.

The tldr; here is that we get about a 10% bump in polled performance with
this patchset, as we can recycle bio structures essentially for free.
Outside of that, explanations in each patch. I've also got an iomap patch,
but trying to keep this single user until there's agreement on the
direction.

Against for-5.15/io_uring, and can also be found in my
io_uring-bio-cache.4 branch.

 block/bio.c                | 170 +++++++++++++++++++++++++++++++++----
 block/blk-core.c           |   5 +-
 fs/block_dev.c             |   6 +-
 fs/io_uring.c              |   2 +-
 include/linux/bio.h        |  23 +++--
 include/linux/blk_types.h  |   1 +
 include/linux/cpuhotplug.h |   1 +
 include/linux/fs.h         |   2 +
 8 files changed, 185 insertions(+), 25 deletions(-)

-- 
Jens Axboe



^ permalink raw reply	[flat|nested] 25+ messages in thread

end of thread, other threads:[~2021-08-13 20:33 UTC | newest]

Thread overview: 25+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-08-12 15:41 [PATCHSET v5 0/6] Enable bio recycling for polled IO Jens Axboe
2021-08-12 15:41 ` [PATCH 1/6] bio: optimize initialization of a bio Jens Axboe
2021-08-12 16:16   ` Christoph Hellwig
2021-08-12 15:41 ` [PATCH 2/6] fs: add kiocb alloc cache flag Jens Axboe
2021-08-12 16:20   ` Christoph Hellwig
2021-08-12 15:41 ` [PATCH 3/6] bio: add allocation cache abstraction Jens Axboe
2021-08-12 16:32   ` Christoph Hellwig
2021-08-12 16:39     ` Jens Axboe
2021-08-12 15:41 ` [PATCH 4/6] block: clear BIO_PERCPU_CACHE flag if polling isn't supported Jens Axboe
2021-08-12 16:33   ` Christoph Hellwig
2021-08-12 17:31   ` Keith Busch
2021-08-12 17:41     ` Jens Axboe
2021-08-13 20:17       ` Keith Busch
2021-08-13 20:19         ` Jens Axboe
2021-08-13 20:31           ` Keith Busch
2021-08-13 20:33             ` Jens Axboe
2021-08-12 15:41 ` [PATCH 5/6] io_uring: enable use of bio alloc cache Jens Axboe
2021-08-12 16:39   ` Christoph Hellwig
2021-08-12 16:46     ` Jens Axboe
2021-08-12 15:41 ` [PATCH 6/6] block: use the percpu bio cache in __blkdev_direct_IO Jens Axboe
  -- strict thread matches above, loose matches on Subject: below --
2021-08-11 19:35 [PATCHSET v4 0/6] Enable bio recycling for polled IO Jens Axboe
2021-08-11 19:35 ` [PATCH 3/6] bio: add allocation cache abstraction Jens Axboe
2021-08-12  7:01   ` Christoph Hellwig
2021-08-12 15:08     ` Jens Axboe
2021-08-12 15:18       ` Christoph Hellwig
2021-08-12 15:26         ` Jens Axboe

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox