public inbox for [email protected]
 help / color / mirror / Atom feed
From: Pavel Begunkov <[email protected]>
To: [email protected]
Cc: Jens Axboe <[email protected]>, [email protected]
Subject: [RFC 03/11] io_uring: ban tw queue for exiting processes
Date: Thu, 21 Apr 2022 14:44:16 +0100	[thread overview]
Message-ID: <4c8fc40551c5b991fc6560cba2ffe37f47375a1e.1650548192.git.asml.silence@gmail.com> (raw)
In-Reply-To: <[email protected]>

We rely on PF_EXITING and task_work infrastructure for preventing adding
new task_work items to a dying task, which is a bit more convoluted than
desired.

Ban new tw items earlier in io_uring_cancel_generic() by relying on
->in_idle. io_req_task_work_add() will check the flag, set REQ_F_FAIL
and push requests to the fallback path. task_work handlers will find it
and cancel requests just as it was with PF_EXITING.

Signed-off-by: Pavel Begunkov <[email protected]>
---
 fs/io_uring.c | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/fs/io_uring.c b/fs/io_uring.c
index 272a180ab7ee..ec5fe55ab265 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -1306,7 +1306,7 @@ static inline void req_ref_get(struct io_kiocb *req)
 
 static inline bool io_should_fail_tw(struct io_kiocb *req)
 {
-	return unlikely(req->task->flags & PF_EXITING);
+	return unlikely(req->flags & REQ_F_FAIL);
 }
 
 static inline void io_submit_flush_completions(struct io_ring_ctx *ctx)
@@ -2577,10 +2577,6 @@ static void tctx_task_work(struct callback_head *cb)
 	}
 
 	ctx_flush_and_put(ctx, &uring_locked);
-
-	/* relaxed read is enough as only the task itself sets ->in_idle */
-	if (unlikely(atomic_read(&tctx->in_idle)))
-		io_uring_drop_tctx_refs(current);
 }
 
 static void io_req_task_work_add(struct io_kiocb *req, bool priority)
@@ -2600,6 +2596,9 @@ static void io_req_task_work_add(struct io_kiocb *req, bool priority)
 	spin_lock_irqsave(&tctx->task_lock, flags);
 	list = priority ? &tctx->prior_task_list : &tctx->task_list;
 	wq_list_add_tail(&req->io_task_work.node, list);
+	if (unlikely(atomic_read(&tctx->in_idle)))
+		goto cancel_locked;
+
 	running = tctx->task_running;
 	if (!running)
 		tctx->task_running = true;
@@ -2623,12 +2622,13 @@ static void io_req_task_work_add(struct io_kiocb *req, bool priority)
 	}
 
 	spin_lock_irqsave(&tctx->task_lock, flags);
-	tctx->task_running = false;
+cancel_locked:
 	node = wq_list_merge(&tctx->prior_task_list, &tctx->task_list);
 	spin_unlock_irqrestore(&tctx->task_lock, flags);
 
 	while (node) {
 		req = container_of(node, struct io_kiocb, io_task_work.node);
+		req_set_fail(req);
 		node = node->next;
 		if (llist_add(&req->io_task_work.fallback_node,
 			      &req->ctx->fallback_llist))
@@ -10352,7 +10352,10 @@ static __cold void io_uring_cancel_generic(bool cancel_all,
 	if (tctx->io_wq)
 		io_wq_exit_start(tctx->io_wq);
 
+	spin_lock_irq(&tctx->task_lock);
 	atomic_inc(&tctx->in_idle);
+	spin_unlock_irq(&tctx->task_lock);
+
 	do {
 		io_uring_drop_tctx_refs(current);
 		/* read completions before cancelations */
-- 
2.36.0


  parent reply	other threads:[~2022-04-21 13:45 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-04-21 13:44 [RFC 00/11] io_uring specific task_work infra Pavel Begunkov
2022-04-21 13:44 ` [RFC 01/11] io_uring: optimise io_req_task_work_add Pavel Begunkov
2022-04-21 13:44 ` [RFC 02/11] io_uringg: add io_should_fail_tw() helper Pavel Begunkov
2022-04-21 13:44 ` Pavel Begunkov [this message]
2022-04-21 13:44 ` [RFC 04/11] io_uring: don't take ctx refs in tctx_task_work() Pavel Begunkov
2022-04-21 13:44 ` [RFC 05/11] io_uring: add dummy io_uring_task_work_run() Pavel Begunkov
2022-04-21 13:44 ` [RFC 06/11] task_work: add helper for signalling a task Pavel Begunkov
2022-04-21 13:44 ` [RFC 07/11] io_uring: run io_uring task_works on TIF_NOTIFY_SIGNAL Pavel Begunkov
2022-04-21 13:44 ` [RFC 08/11] io_uring: wire io_uring specific task work Pavel Begunkov
2022-04-21 13:44 ` [RFC 09/11] io_uring: refactor io_run_task_work() Pavel Begunkov
2022-04-21 13:44 ` [RFC 10/11] io_uring: remove priority tw list Pavel Begunkov
2022-04-21 13:44 ` [RFC 11/11] io_uring: lock-free task_work stack Pavel Begunkov
2022-04-21 14:50 ` [RFC 00/11] io_uring specific task_work infra Pavel Begunkov
2022-04-22  8:45   ` Hao Xu
2022-04-22 11:54     ` Pavel Begunkov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=4c8fc40551c5b991fc6560cba2ffe37f47375a1e.1650548192.git.asml.silence@gmail.com \
    [email protected] \
    [email protected] \
    [email protected] \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox