From: David Wei <[email protected]>
To: Jens Axboe <[email protected]>, [email protected]
Subject: Re: [PATCH 3/3] io_uring/msg_ring: improve handling of target CQE posting
Date: Mon, 1 Apr 2024 10:57:25 -0700 [thread overview]
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
On 2024-03-29 13:09, Jens Axboe wrote:
> Use the exported helper for queueing task_work, rather than rolling our
> own.
>
> This improves peak performance of message passing by about 5x in some
> basic testing, with 2 threads just sending messages to each other.
> Before this change, it was capped at around 700K/sec, with the change
> it's at over 4M/sec.
>
> Signed-off-by: Jens Axboe <[email protected]>
> ---
> io_uring/msg_ring.c | 24 ++++++------------------
> 1 file changed, 6 insertions(+), 18 deletions(-)
>
> diff --git a/io_uring/msg_ring.c b/io_uring/msg_ring.c
> index d1f66a40b4b4..af8a5f2947b7 100644
> --- a/io_uring/msg_ring.c
> +++ b/io_uring/msg_ring.c
> @@ -13,7 +13,6 @@
> #include "filetable.h"
> #include "msg_ring.h"
>
> -
> /* All valid masks for MSG_RING */
> #define IORING_MSG_RING_MASK (IORING_MSG_RING_CQE_SKIP | \
> IORING_MSG_RING_FLAGS_PASS)
> @@ -21,7 +20,6 @@
> struct io_msg {
> struct file *file;
> struct file *src_file;
> - struct callback_head tw;
> u64 user_data;
> u32 len;
> u32 cmd;
> @@ -73,26 +71,18 @@ static inline bool io_msg_need_remote(struct io_ring_ctx *target_ctx)
> return current != target_ctx->submitter_task;
> }
>
> -static int io_msg_exec_remote(struct io_kiocb *req, task_work_func_t func)
> +static int io_msg_exec_remote(struct io_kiocb *req, io_req_tw_func_t func)
> {
> struct io_ring_ctx *ctx = req->file->private_data;
> - struct io_msg *msg = io_kiocb_to_cmd(req, struct io_msg);
> - struct task_struct *task = READ_ONCE(ctx->submitter_task);
> -
> - if (unlikely(!task))
> - return -EOWNERDEAD;
> -
> - init_task_work(&msg->tw, func);
> - if (task_work_add(ctx->submitter_task, &msg->tw, TWA_SIGNAL))
> - return -EOWNERDEAD;
>
> + req->io_task_work.func = func;
> + io_req_task_work_add_remote(req, ctx, IOU_F_TWQ_LAZY_WAKE);
> return IOU_ISSUE_SKIP_COMPLETE;
> }
This part looks correct. Now with io_req_task_work_add_remote(),
req->io_task_work.func is added to tctx->task_list, and queued up for
execution on the remote ctx->submitter_task via task_work_add(). The end
result is that the argument @func is executed on the remote
ctx->submitter_task, which is the same outcome as before.
Also, unsure how this hand rolled code interacted with defer taskrun
before but now it is handled properly in io_req_task_work_add_remote().
>
> -static void io_msg_tw_complete(struct callback_head *head)
> +static void io_msg_tw_complete(struct io_kiocb *req, struct io_tw_state *ts)
> {
> - struct io_msg *msg = container_of(head, struct io_msg, tw);
> - struct io_kiocb *req = cmd_to_io_kiocb(msg);
> + struct io_msg *msg = io_kiocb_to_cmd(req, struct io_msg);
> struct io_ring_ctx *target_ctx = req->file->private_data;
> int ret = 0;
>
> @@ -205,10 +195,8 @@ static int io_msg_install_complete(struct io_kiocb *req, unsigned int issue_flag
> return ret;
> }
>
> -static void io_msg_tw_fd_complete(struct callback_head *head)
> +static void io_msg_tw_fd_complete(struct io_kiocb *req, struct io_tw_state *ts)
> {
> - struct io_msg *msg = container_of(head, struct io_msg, tw);
> - struct io_kiocb *req = cmd_to_io_kiocb(msg);
> int ret = -EOWNERDEAD;
>
> if (!(current->flags & PF_EXITING))
next prev parent reply other threads:[~2024-04-01 17:57 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-03-29 20:09 [PATCHSET v2 0/3] Cleanup and improve MSG_RING performance Jens Axboe
2024-03-29 20:09 ` [PATCH 1/3] io_uring: add remote task_work execution helper Jens Axboe
2024-04-01 17:30 ` David Wei
2024-04-01 18:02 ` Jens Axboe
2024-03-29 20:09 ` [PATCH 2/3] io_uring/msg_ring: cleanup posting to IOPOLL vs !IOPOLL ring Jens Axboe
2024-03-29 20:09 ` [PATCH 3/3] io_uring/msg_ring: improve handling of target CQE posting Jens Axboe
2024-04-01 17:57 ` David Wei [this message]
-- strict thread matches above, loose matches on Subject: below --
2024-03-28 18:52 [PATCHSET 0/3] Cleanup and improve MSG_RING performance Jens Axboe
2024-03-28 18:52 ` [PATCH 3/3] io_uring/msg_ring: improve handling of target CQE posting Jens Axboe
2024-03-29 12:54 ` Pavel Begunkov
2024-03-29 13:32 ` Jens Axboe
2024-03-29 15:46 ` Pavel Begunkov
2024-03-29 15:47 ` Jens Axboe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox