public inbox for [email protected]
 help / color / mirror / Atom feed
* [PATCHSET v6 0/8] Improve async iomap DIO performance
@ 2023-07-24 22:55 Jens Axboe
  2023-07-24 22:55 ` [PATCH 1/8] iomap: cleanup up iomap_dio_bio_end_io() Jens Axboe
                   ` (9 more replies)
  0 siblings, 10 replies; 17+ messages in thread
From: Jens Axboe @ 2023-07-24 22:55 UTC (permalink / raw)
  To: io-uring, linux-xfs; +Cc: hch, andres, david, djwong

Hi,

Hi,

This patchset improves async iomap DIO performance, for XFS and ext4.
For full details on this patchset, see the v4 posting:

https://lore.kernel.org/io-uring/[email protected]/

 fs/iomap/direct-io.c | 163 ++++++++++++++++++++++++++++++++-----------
 include/linux/fs.h   |  35 +++++++++-
 io_uring/rw.c        |  26 ++++++-
 3 files changed, 179 insertions(+), 45 deletions(-)

Can also be found here:

https://git.kernel.dk/cgit/linux/log/?h=xfs-async-dio.6

No change in performance since last time, and passes my testing without
complaints.

Changes in v6:
- Drop the polled patch, it's not needed anymore
- Change the "inline is safe" logic based on Dave's suggestions
- Gate HIPRI on INLINE_COMP|CALLER_COMP, so polled IO follows the
  same rules as inline/deferred completions.
- INLINE_COMP is purely for reads, writes can user CALLER_COMP to
  avoid a workqueue punt. This is necessary as we need to invalidate
  pages on write completions, and if we race with a buffered reader
  or writer on the file.

-- 
Jens Axboe



^ permalink raw reply	[flat|nested] 17+ messages in thread
* [PATCHSET v4 0/8] Improve async iomap DIO performance
@ 2023-07-20 18:13 Jens Axboe
  2023-07-20 18:13 ` [PATCH 1/8] iomap: cleanup up iomap_dio_bio_end_io() Jens Axboe
  0 siblings, 1 reply; 17+ messages in thread
From: Jens Axboe @ 2023-07-20 18:13 UTC (permalink / raw)
  To: io-uring, linux-xfs; +Cc: hch, andres, david

Hi,

iomap always punts async dio write completions to a workqueue, which has
a cost in terms of efficiency (now you need an unrelated worker to
process it) and latency (now you're bouncing a completion through an
async worker, which is a classic slowdown scenario).

Even for writes that should, in theory, be able to complete inline,
if we race with truncate or need to invalidate pages post completion,
we cannot sanely be in IRQ context as the locking types don't allow
for that.

io_uring handles IRQ completions via task_work, and for writes that
don't need to do extra IO at completion time, we can safely complete
them inline from that. This patchset adds IOCB_DEFER, which an IO
issuer can set to inform the completion side that any extra work that
needs doing for that completion can be punted to a safe task context.

The iomap dio completion will happen in hard/soft irq context, and we
need a saner context to process these completions. IOCB_DIO_DEFER is
added, which can be set in a struct kiocb->ki_flags by the issuer. If
the completion side of the iocb handling understands this flag, it can
choose to set a kiocb->dio_complete() handler and just call ki_complete
from IRQ context. The issuer must then ensure that this callback is
processed from a task. io_uring punts IRQ completions to task_work
already, so it's trivial wire it up to run more of the completion before
posting a CQE. This is good for up to a 37% improvement in
throughput/latency for low queue depth IO, patch 5 has the details.

If we need to do real work at completion time, iomap will clear the
IOMAP_DIO_DEFER_COMP flag.

This work came about when Andres tested low queue depth dio writes
for postgres and compared it to doing sync dio writes, showing that the
async processing slows us down a lot.

Dave, would appreciate your input on if the logic is right now in
terms of when we can inline complete when DEFER is set!

 fs/iomap/direct-io.c | 154 +++++++++++++++++++++++++++++++++----------
 include/linux/fs.h   |  34 +++++++++-
 io_uring/rw.c        |  27 +++++++-
 3 files changed, 176 insertions(+), 39 deletions(-)

Can also be found in a git branch here:

https://git.kernel.dk/cgit/linux/log/?h=xfs-async-dio.4

Since v3:
- Add two patches for polled IO. One that completes inline if it's set
  at completion time, and one that cleans up the iocb->private handling
  and adds comments as to why they are only relevant on polled IO.
- Rename IOMAP_DIO_WRITE_FUA to IOMAP_DIO_STABLE_WRITE in conjunction
  with treating fua && vwc the same as !vwc.
- Address review comments from Christoph
- Add comments and expand commit messages, where appropriate.

-- 
Jens Axboe



^ permalink raw reply	[flat|nested] 17+ messages in thread
end of thread, other threads:[~2023-08-01 23:35 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-07-24 22:55 [PATCHSET v6 0/8] Improve async iomap DIO performance Jens Axboe
2023-07-24 22:55 ` [PATCH 1/8] iomap: cleanup up iomap_dio_bio_end_io() Jens Axboe
2023-07-24 22:55 ` [PATCH 2/8] iomap: use an unsigned type for IOMAP_DIO_* defines Jens Axboe
2023-07-24 22:55 ` [PATCH 3/8] iomap: treat a write through cache the same as FUA Jens Axboe
2023-07-24 22:55 ` [PATCH 4/8] iomap: only set iocb->private for polled bio Jens Axboe
2023-07-24 22:55 ` [PATCH 5/8] iomap: add IOMAP_DIO_INLINE_COMP Jens Axboe
2023-07-24 22:55 ` [PATCH 6/8] fs: add IOCB flags related to passing back dio completions Jens Axboe
2023-07-24 22:55 ` [PATCH 7/8] io_uring/rw: add write support for IOCB_DIO_CALLER_COMP Jens Axboe
2023-07-24 22:55 ` [PATCH 8/8] iomap: support IOCB_DIO_CALLER_COMP Jens Axboe
2023-07-26 23:07 ` [PATCHSET v6 0/8] Improve async iomap DIO performance Jens Axboe
2023-07-28 21:42   ` Dave Chinner
2023-08-01  0:20     ` Jens Axboe
2023-08-01 22:13 ` Dave Chinner
2023-08-01 23:34   ` Jens Axboe
  -- strict thread matches above, loose matches on Subject: below --
2023-07-20 18:13 [PATCHSET v4 " Jens Axboe
2023-07-20 18:13 ` [PATCH 1/8] iomap: cleanup up iomap_dio_bio_end_io() Jens Axboe
2023-07-21  6:14   ` Christoph Hellwig
2023-07-21 15:13   ` Darrick J. Wong

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox