public inbox for [email protected]
 help / color / mirror / Atom feed
From: Pavel Begunkov <[email protected]>
To: [email protected]
Cc: [email protected], Jens Axboe <[email protected]>,
	[email protected], Kanchan Joshi <[email protected]>,
	Ming Lei <[email protected]>
Subject: [PATCH v2 08/14] io_uring: force tw ctx locking
Date: Mon, 18 Mar 2024 00:41:53 +0000	[thread overview]
Message-ID: <49af41a9127c06a8133a371b59390e771b39bae4.1710720150.git.asml.silence@gmail.com> (raw)
In-Reply-To: <[email protected]>

We can run normal task_work without locking the ctx, however we try to
lock anyway and most handlers prefer or require it locked. It might have
been interesting to multi-submitter ring with high contention completing
async read/write requests via task_work, however that will still need to
go through io_req_complete_post() and potentially take the lock for
rsrc node putting or some other case.

In other words, it's hard to care about it, so alawys force the locking.
The case described would also because of various io_uring caches.

Signed-off-by: Pavel Begunkov <[email protected]>
Link: https://lore.kernel.org/r/1f7f31f4075e766343055ff0d07482992038d467.1710514702.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <[email protected]>
---
 io_uring/io_uring.c | 21 +++++++++------------
 1 file changed, 9 insertions(+), 12 deletions(-)

diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c
index 9d7bbdea6db5..3184d57f9a35 100644
--- a/io_uring/io_uring.c
+++ b/io_uring/io_uring.c
@@ -1191,8 +1191,9 @@ struct llist_node *io_handle_tw_list(struct llist_node *node,
 		if (req->ctx != ctx) {
 			ctx_flush_and_put(ctx, &ts);
 			ctx = req->ctx;
-			/* if not contended, grab and improve batching */
-			ts.locked = mutex_trylock(&ctx->uring_lock);
+
+			ts.locked = true;
+			mutex_lock(&ctx->uring_lock);
 			percpu_ref_get(&ctx->refs);
 		}
 		INDIRECT_CALL_2(req->io_task_work.func,
@@ -1453,11 +1454,9 @@ static int __io_run_local_work(struct io_ring_ctx *ctx, struct io_tw_state *ts,
 
 	if (io_run_local_work_continue(ctx, ret, min_events))
 		goto again;
-	if (ts->locked) {
-		io_submit_flush_completions(ctx);
-		if (io_run_local_work_continue(ctx, ret, min_events))
-			goto again;
-	}
+	io_submit_flush_completions(ctx);
+	if (io_run_local_work_continue(ctx, ret, min_events))
+		goto again;
 
 	trace_io_uring_local_work_run(ctx, ret, loops);
 	return ret;
@@ -1481,14 +1480,12 @@ static inline int io_run_local_work_locked(struct io_ring_ctx *ctx,
 
 static int io_run_local_work(struct io_ring_ctx *ctx, int min_events)
 {
-	struct io_tw_state ts = {};
+	struct io_tw_state ts = { .locked = true };
 	int ret;
 
-	ts.locked = mutex_trylock(&ctx->uring_lock);
+	mutex_lock(&ctx->uring_lock);
 	ret = __io_run_local_work(ctx, &ts, min_events);
-	if (ts.locked)
-		mutex_unlock(&ctx->uring_lock);
-
+	mutex_unlock(&ctx->uring_lock);
 	return ret;
 }
 
-- 
2.44.0


  parent reply	other threads:[~2024-03-18  0:43 UTC|newest]

Thread overview: 41+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-03-18  0:41 [PATCH v2 00/14] remove aux CQE caches Pavel Begunkov
2024-03-18  0:41 ` [PATCH v2 01/14] io_uring/cmd: kill one issue_flags to tw conversion Pavel Begunkov
2024-03-18  0:41 ` [PATCH v2 02/14] io_uring/cmd: fix tw <-> issue_flags conversion Pavel Begunkov
2024-03-18  2:23   ` Ming Lei
2024-03-18  2:25     ` Jens Axboe
2024-03-18  2:32       ` Pavel Begunkov
2024-03-18  2:40         ` Jens Axboe
2024-03-18  2:43           ` Pavel Begunkov
2024-03-18  2:46             ` Jens Axboe
2024-03-18  2:47           ` Ming Lei
2024-03-18  3:11             ` Jens Axboe
2024-03-18  3:24               ` Pavel Begunkov
2024-03-18  6:59               ` Ming Lei
2024-03-18 11:45                 ` Pavel Begunkov
2024-03-18  0:41 ` [PATCH v2 03/14] io_uring/cmd: make io_uring_cmd_done irq safe Pavel Begunkov
2024-03-18  8:10   ` Ming Lei
2024-03-18 11:50     ` Pavel Begunkov
2024-03-18 11:59       ` Ming Lei
2024-03-18 12:46         ` Pavel Begunkov
2024-03-18 13:09           ` Ming Lei
2024-03-18  0:41 ` [PATCH v2 04/14] io_uring/cmd: introduce io_uring_cmd_complete Pavel Begunkov
2024-03-18  0:41 ` [PATCH v2 05/14] ublk: don't hard code IO_URING_F_UNLOCKED Pavel Begunkov
2024-03-18  8:16   ` Ming Lei
2024-03-18 12:52     ` Pavel Begunkov
2024-03-18 13:37       ` Pavel Begunkov
2024-03-18 14:32         ` Pavel Begunkov
2024-03-18 14:39           ` Ming Lei
2024-03-18 14:34       ` Ming Lei
2024-03-18 15:08         ` Pavel Begunkov
2024-03-18 15:16           ` Ming Lei
2024-03-18  0:41 ` [PATCH v2 06/14] nvme/io_uring: " Pavel Begunkov
2024-03-18 13:26   ` Kanchan Joshi
2024-03-18 13:38     ` Pavel Begunkov
2024-03-18  0:41 ` [PATCH v2 07/14] io_uring/rw: avoid punting to io-wq directly Pavel Begunkov
2024-03-18  0:41 ` Pavel Begunkov [this message]
2024-03-18  0:41 ` [PATCH v2 09/14] io_uring: remove struct io_tw_state::locked Pavel Begunkov
2024-03-18  0:41 ` [PATCH v2 10/14] io_uring: refactor io_fill_cqe_req_aux Pavel Begunkov
2024-03-18  0:41 ` [PATCH v2 11/14] io_uring: get rid of intermediate aux cqe caches Pavel Begunkov
2024-03-18  0:41 ` [PATCH v2 12/14] io_uring: remove current check from complete_post Pavel Begunkov
2024-03-18  0:41 ` [PATCH v2 13/14] io_uring: refactor io_req_complete_post() Pavel Begunkov
2024-03-18  0:41 ` [PATCH v2 14/14] io_uring: clean up io_lockdep_assert_cq_locked Pavel Begunkov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=49af41a9127c06a8133a371b59390e771b39bae4.1710720150.git.asml.silence@gmail.com \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox