[RFC PATCH v1 00/18] io-uring/xfs: support async buffered writes

[Stefan Roesch <shr@fb.com>]

This patch series adds support for async buffered writes when using both
xfs and io-uring. Currently io-uring only supports buffered writes in the
slow path, by processing them in the io workers. With this patch series it is
now possible to support buffered writes in the fast path. To be able to use
the fast path the required pages must be in the page cache, the required locks
in xfs can be granted immediately and no additional blocks need to be read
form disk.

Updating the inode can take time. An optimization has been implemented for
the time update. Time updates will be processed in the slow path. While there
is already a time update in process, other write requests for the same file,
can skip the update of the modification time.
  

Performance results:
  For fio the following results have been obtained with a queue depth of
  1 and 4k block size (runtime 600 secs):

                 sequential writes:
                 without patch                 with patch
  iops:              80k                          269k


                 random writes:
                 without patch                 with patch
  iops:              76k                          249k

For an io depth of 1, the new patch improves throughput by over three times
(compared to the exiting behavior, where buffered writes are processed by an
io-worker process) and also the latency is considerably reduced. To achieve the
same or better performance with the exisiting code an io depth of 4 is required.
Increasing the iodepth further does not lead to any further improvements.

Especially for mixed workloads this is a considerable improvement.



Support for async buffered writes:

  Patch 1: block: add check for async buffered writes to generic_write_checks
    Add a new flag FMODE_BUF_WASYNC so filesystems can specify that they support
    async buffered writes and include the flag in the check of the function
    generic_write_checks().
    
  Patch 2: mm: add FGP_ATOMIC flag to __filemap_get_folio()
    This adds the FGP_ATOMIC flag. This allows to specify the gfp flags
    for memory allocations for async buffered writes.
    
  Patch 3: iomap: add iomap_page_create_gfp to allocate iomap_pages
    Add new function to allow specifying gfp flags when allocating and
    initializing the structure iomap_page.

  Patch 4: iomap: use iomap_page_create_gfp() in __iomap_write_begin
    Add a gfp flag to the iomap_page_create function.
  
  Patch 5: iomap: add async buffered write support
    Set IOMAP_NOWAIT flag if IOCB_NOWAIT is set. Also use specific gfp flags if
    the iomap_page structure is allocated for an async buffered page.

  Patch 6: xfs: add iomap async buffered write support
    Add async buffered write support to the xfs iomap layer.

Support for async buffered write support and inode time modification


  Patch 7: fs: split off need_remove_file_privs() do_remove_file_privs()
    Splits of a check and action function, so they can later be invoked
    in the nowait code path.
    
  Patch 8: fs: split off need_file_update_time and do_file_update_time
    Splits of a check and action function, so they can later be invoked
    in the nowait code path.
    
  Patch 9: fs: add pending file update time flag.
    Add new flag so consecutive write requests for the same inode can
    proceed without waiting for the inode modification time to complete.

  Patch 10: xfs: enable async write file modification handling.
    Enable async write handling in xfs for the file modification time
    update. If the file modification update requires logging or removal
    of privileges that needs to wait, -EAGAIN is returned.

  Patch 11: xfs: add async buffered write support
    Take the ilock in nowait mode if async buffered writes are enabled.

  Patch 12: io_uring: add support for async buffered writes
    This enables the async buffered writes optimization in io_uring.
    Buffered writes are enabled for blocks that are already in the page
    cache.

  Patch 13: io_uring: Add tracepoint for short writes

Support for write throttling of async buffered writes:

  Patch 14: sched: add new fields to task_struct
    Add two new fields to the task_struct. These fields store the
    deadline after which writes are no longer throttled.

  Patch 15: mm: support write throttling for async buffered writes
    This changes the balance_dirty_pages function to take an additional
    parameter. When nowait is specified the write throttling code no
    longer waits synchronously for the deadline to expire. Instead
    it sets the fields in task_struct. Once the deadline expires the
    fields are reset.
    
  Patch 16: iomap: User throttling for async buffered writes.
    Enable async buffered write throttling in iomap.

  Patch 17: io_uring: support write throttling for async buffered writes
    Adds support to io_uring for write throttling. When the writes
    are throttled, the write requests are added to the pending io list.
    Once the write throttling deadline expires, the writes are submitted.

Enable async buffered write support in xfs
  Patch 18: xfs: enable async buffered write support
    This enables the flag that enables async buffered writes for xfs.


Testing:
  This patch has been tested with xfstests and fio.


Stefan Roesch (18):
  block: add check for async buffered writes to generic_write_checks
  mm: add FGP_ATOMIC flag to __filemap_get_folio()
  iomap: add iomap_page_create_gfp to allocate iomap_pages
  iomap: use iomap_page_create_gfp() in __iomap_write_begin
  iomap: add async buffered write support
  xfs: add iomap async buffered write support
  fs: split off need_remove_file_privs() do_remove_file_privs()
  fs: split off need_file_update_time and do_file_update_time
  fs: add pending file update time flag.
  xfs: Enable async write file modification handling.
  xfs: add async buffered write support
  io_uring: add support for async buffered writes
  io_uring: add tracepoint for short writes
  sched: add new fields to task_struct
  mm: support write throttling for async buffered writes
  iomap: User throttling for async buffered writes.
  io_uring: support write throttling for async buffered writes
  xfs: enable async buffered write support

 fs/inode.c                      | 127 +++++++++++++++++++++----------
 fs/io_uring.c                   | 131 +++++++++++++++++++++++++++++---
 fs/iomap/buffered-io.c          |  63 +++++++++++++--
 fs/read_write.c                 |   3 +-
 fs/xfs/xfs_file.c               |  51 +++++++++++--
 fs/xfs/xfs_iomap.c              |  33 +++++++-
 include/linux/fs.h              |  14 ++++
 include/linux/pagemap.h         |   1 +
 include/linux/sched.h           |   3 +
 include/linux/writeback.h       |   1 +
 include/trace/events/io_uring.h |  25 ++++++
 kernel/fork.c                   |   1 +
 mm/filemap.c                    |   3 +
 mm/page-writeback.c             |  54 +++++++++----
 14 files changed, 424 insertions(+), 86 deletions(-)


base-commit: af2d861d4cd2a4da5137f795ee3509e6f944a25b
-- 
2.30.2