[PATCH v8 0/2] io_uring/rsrc: coalescing multi-hugepage registered buffers

[Chenliang Li <cliang01.li@samsung.com>]

Registered buffers are stored and processed in the form of bvec array,
each bvec element typically points to a PAGE_SIZE page but can also work
with hugepages. Specifically, a buffer consisting of a hugepage is
coalesced to use only one hugepage bvec entry during registration.
This coalescing feature helps to save both the space and DMA-mapping time.

However, currently the coalescing feature doesn't work for multi-hugepage
buffers. For a buffer with several 2M hugepages, we still split it into
thousands of 4K page bvec entries while in fact, we can just use a
handful of hugepage bvecs.

This patch series enables coalescing registered buffers with more than
one hugepages. It optimizes the DMA-mapping time and saves memory for 
these kind of buffers.

Testing:

The hugepage fixed buffer I/O can be tested using fio without
modification. The fio command used in the following test is given
in [1]. There's also a liburing testcase in [2]. Also, the system
should have enough hugepages available before testing.

Perf diff of 8M(4 * 2M hugepages) fio randread test:

Before          After           Symbol
.....................................................
5.88%                           [k] __blk_rq_map_sg
3.98%           -3.95%          [k] dma_direct_map_sg
2.47%                           [k] dma_pool_alloc
1.37%           -1.36%          [k] sg_next
                +0.28%          [k] dma_map_page_attrs

Perf diff of 8M fio randwrite test:

Before          After           Symbol
......................................................
2.80%                           [k] __blk_rq_map_sg
1.74%                           [k] dma_direct_map_sg
1.61%                           [k] dma_pool_alloc
0.67%                           [k] sg_next
                +0.04%          [k] dma_map_page_attrs

The first patch prepares for adding the multi-hugepage coalescing
by storing folio_shift and folio_mask into imu, the 2nd patch
enables the feature.

---
Changes since v7:
 - Rebase to io_uring-6.11

v7 : https://lore.kernel.org/io-uring/54ef8248-29ac-4c0e-b224-8515b4c604cc@gmail.com/T/#t

Changes since v6:
 - Remove the restriction on non-border-aligned single hugepage.
 - Code style issue.

v6 : https://lore.kernel.org/io-uring/20240716060807.2707-1-cliang01.li@samsung.com/T/#t

[1]
fio -iodepth=64 -rw=randread -direct=1 -ioengine=io_uring \
-bs=8M -numjobs=1 -group_reporting -mem=shmhuge -fixedbufs -hugepage-size=2M \
-filename=/dev/nvme0n1 -runtime=10s -name=test1

[2]
https://lore.kernel.org/io-uring/20240514051343.582556-1-cliang01.li@samsung.com/T/#u

Chenliang Li (2):
  io_uring/rsrc: store folio shift and mask into imu
  io_uring/rsrc: enable multi-hugepage buffer coalescing

 io_uring/rsrc.c | 149 +++++++++++++++++++++++++++++++++++-------------
 io_uring/rsrc.h |  10 ++++
 2 files changed, 118 insertions(+), 41 deletions(-)


base-commit: c3fca4fb83f7c84cd1e1aa9fe3a0e220ce8f30fb
-- 
2.34.1