public inbox for [email protected]
 help / color / mirror / Atom feed
From: "Darrick J. Wong" <[email protected]>
To: Jens Axboe <[email protected]>
Cc: [email protected], [email protected], [email protected],
	[email protected], [email protected]
Subject: Re: [PATCH 7/9] io_uring/rw: add write support for IOCB_DIO_CALLER_COMP
Date: Fri, 21 Jul 2023 09:29:25 -0700	[thread overview]
Message-ID: <20230721162925.GV11352@frogsfrogsfrogs> (raw)
In-Reply-To: <[email protected]>

On Fri, Jul 21, 2023 at 10:16:48AM -0600, Jens Axboe wrote:
> If the filesystem dio handler understands IOCB_DIO_CALLER_COMP, we'll
> get a kiocb->ki_complete() callback with kiocb->dio_complete set. In
> that case, rather than complete the IO directly through task_work, queue
> up an intermediate task_work handler that first processes this callback
> and then immediately completes the request.
> 
> For XFS, this avoids a punt through a workqueue, which is a lot less
> efficient and adds latency to lower queue depth (or sync) O_DIRECT
> writes.
> 
> Only do this for non-polled IO, as polled IO doesn't need this kind
> of deferral as it always completes within the task itself. This then
> avoids a check for deferral in the polled IO completion handler.
> 
> Reviewed-by: Christoph Hellwig <[email protected]>
> Signed-off-by: Jens Axboe <[email protected]>

Looks good!
Reviewed-by: Darrick J. Wong <[email protected]>

--D

> ---
>  io_uring/rw.c | 26 +++++++++++++++++++++++---
>  1 file changed, 23 insertions(+), 3 deletions(-)
> 
> diff --git a/io_uring/rw.c b/io_uring/rw.c
> index 1bce2208b65c..f19f65b3f0ee 100644
> --- a/io_uring/rw.c
> +++ b/io_uring/rw.c
> @@ -105,6 +105,7 @@ int io_prep_rw(struct io_kiocb *req, const struct io_uring_sqe *sqe)
>  	} else {
>  		rw->kiocb.ki_ioprio = get_current_ioprio();
>  	}
> +	rw->kiocb.dio_complete = NULL;
>  
>  	rw->addr = READ_ONCE(sqe->addr);
>  	rw->len = READ_ONCE(sqe->len);
> @@ -285,6 +286,14 @@ static inline int io_fixup_rw_res(struct io_kiocb *req, long res)
>  
>  void io_req_rw_complete(struct io_kiocb *req, struct io_tw_state *ts)
>  {
> +	struct io_rw *rw = io_kiocb_to_cmd(req, struct io_rw);
> +
> +	if (rw->kiocb.dio_complete) {
> +		long res = rw->kiocb.dio_complete(rw->kiocb.private);
> +
> +		io_req_set_res(req, io_fixup_rw_res(req, res), 0);
> +	}
> +
>  	io_req_io_end(req);
>  
>  	if (req->flags & (REQ_F_BUFFER_SELECTED|REQ_F_BUFFER_RING)) {
> @@ -300,9 +309,11 @@ static void io_complete_rw(struct kiocb *kiocb, long res)
>  	struct io_rw *rw = container_of(kiocb, struct io_rw, kiocb);
>  	struct io_kiocb *req = cmd_to_io_kiocb(rw);
>  
> -	if (__io_complete_rw_common(req, res))
> -		return;
> -	io_req_set_res(req, io_fixup_rw_res(req, res), 0);
> +	if (!rw->kiocb.dio_complete) {
> +		if (__io_complete_rw_common(req, res))
> +			return;
> +		io_req_set_res(req, io_fixup_rw_res(req, res), 0);
> +	}
>  	req->io_task_work.func = io_req_rw_complete;
>  	__io_req_task_work_add(req, IOU_F_TWQ_LAZY_WAKE);
>  }
> @@ -916,6 +927,15 @@ int io_write(struct io_kiocb *req, unsigned int issue_flags)
>  	}
>  	kiocb->ki_flags |= IOCB_WRITE;
>  
> +	/*
> +	 * For non-polled IO, set IOCB_DIO_CALLER_COMP, stating that our handler
> +	 * groks deferring the completion to task context. This isn't
> +	 * necessary and useful for polled IO as that can always complete
> +	 * directly.
> +	 */
> +	if (!(kiocb->ki_flags & IOCB_HIPRI))
> +		kiocb->ki_flags |= IOCB_DIO_CALLER_COMP;
> +
>  	if (likely(req->file->f_op->write_iter))
>  		ret2 = call_write_iter(req->file, kiocb, &s->iter);
>  	else if (req->file->f_op->write)
> -- 
> 2.40.1
> 

  reply	other threads:[~2023-07-21 16:32 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-07-21 16:16 [PATCHSET v5 0/9] Improve async iomap DIO performance Jens Axboe
2023-07-21 16:16 ` [PATCH 1/9] iomap: cleanup up iomap_dio_bio_end_io() Jens Axboe
2023-07-21 16:16 ` [PATCH 2/9] iomap: add IOMAP_DIO_INLINE_COMP Jens Axboe
2023-07-21 16:16 ` [PATCH 3/9] iomap: treat a write through cache the same as FUA Jens Axboe
2023-07-21 16:25   ` Darrick J. Wong
2023-07-21 16:27     ` Jens Axboe
2023-07-21 16:47       ` Darrick J. Wong
2023-07-21 16:52         ` Jens Axboe
2023-07-21 16:16 ` [PATCH 4/9] iomap: completed polled IO inline Jens Axboe
2023-07-21 16:16 ` [PATCH 5/9] iomap: only set iocb->private for polled bio Jens Axboe
2023-07-21 16:16 ` [PATCH 6/9] fs: add IOCB flags related to passing back dio completions Jens Axboe
2023-07-21 16:28   ` Darrick J. Wong
2023-07-21 16:30     ` Jens Axboe
2023-07-21 16:43       ` Jens Axboe
2023-07-21 16:16 ` [PATCH 7/9] io_uring/rw: add write support for IOCB_DIO_CALLER_COMP Jens Axboe
2023-07-21 16:29   ` Darrick J. Wong [this message]
2023-07-21 16:16 ` [PATCH 8/9] iomap: support IOCB_DIO_CALLER_COMP Jens Axboe
2023-07-21 16:16 ` [PATCH 9/9] iomap: use an unsigned type for IOMAP_DIO_* defines Jens Axboe
2023-07-21 16:29   ` Darrick J. Wong
2023-07-24 16:36   ` Christoph Hellwig

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230721162925.GV11352@frogsfrogsfrogs \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox