public inbox for [email protected]
 help / color / mirror / Atom feed
From: Pavel Begunkov <[email protected]>
To: [email protected]
Cc: Jens Axboe <[email protected]>,
	[email protected], Oleg Nesterov <[email protected]>,
	Andrew Morton <[email protected]>,
	Christian Brauner <[email protected]>,
	Tycho Andersen <[email protected]>,
	Thomas Gleixner <[email protected]>,
	[email protected], Julian Orth <[email protected]>
Subject: [PATCH 1/2] io_uring/io-wq: limit retrying worker initialisation
Date: Sun,  7 Jul 2024 17:32:10 +0100	[thread overview]
Message-ID: <1af2a697b568f8cf77c5d527d183fa0ffe8fd9cc.1720368770.git.asml.silence@gmail.com> (raw)
In-Reply-To: <[email protected]>

If io-wq worker creation fails, we retry it by queueing up a task_work.
tasK_work is needed because it should be done from the user process
context. The problem is that retries are not limited, and if queueing a
task_work is the reason for the failure, we might get into an infinite
loop.

It doesn't seem to happen now but it will with the following patch
executing task_work in the freezing loop. For now, arbitrarily limit the
number of attempts to create a worker.

Cc: [email protected]
Fixes: 3146cba99aa28 ("io-wq: make worker creation resilient against signals")
Reported-by: Julian Orth <[email protected]>
Signed-off-by: Pavel Begunkov <[email protected]>
---
 io_uring/io-wq.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/io_uring/io-wq.c b/io_uring/io-wq.c
index 913c92249522..f1e7c670add8 100644
--- a/io_uring/io-wq.c
+++ b/io_uring/io-wq.c
@@ -23,6 +23,7 @@
 #include "io_uring.h"
 
 #define WORKER_IDLE_TIMEOUT	(5 * HZ)
+#define WORKER_INIT_LIMIT	3
 
 enum {
 	IO_WORKER_F_UP		= 0,	/* up and active */
@@ -58,6 +59,7 @@ struct io_worker {
 
 	unsigned long create_state;
 	struct callback_head create_work;
+	int init_retries;
 
 	union {
 		struct rcu_head rcu;
@@ -745,7 +747,7 @@ static bool io_wq_work_match_all(struct io_wq_work *work, void *data)
 	return true;
 }
 
-static inline bool io_should_retry_thread(long err)
+static inline bool io_should_retry_thread(struct io_worker *worker, long err)
 {
 	/*
 	 * Prevent perpetual task_work retry, if the task (or its group) is
@@ -753,6 +755,8 @@ static inline bool io_should_retry_thread(long err)
 	 */
 	if (fatal_signal_pending(current))
 		return false;
+	if (worker->init_retries++ >= WORKER_INIT_LIMIT)
+		return false;
 
 	switch (err) {
 	case -EAGAIN:
@@ -779,7 +783,7 @@ static void create_worker_cont(struct callback_head *cb)
 		io_init_new_worker(wq, worker, tsk);
 		io_worker_release(worker);
 		return;
-	} else if (!io_should_retry_thread(PTR_ERR(tsk))) {
+	} else if (!io_should_retry_thread(worker, PTR_ERR(tsk))) {
 		struct io_wq_acct *acct = io_wq_get_acct(worker);
 
 		atomic_dec(&acct->nr_running);
@@ -846,7 +850,7 @@ static bool create_io_worker(struct io_wq *wq, int index)
 	tsk = create_io_thread(io_wq_worker, worker, NUMA_NO_NODE);
 	if (!IS_ERR(tsk)) {
 		io_init_new_worker(wq, worker, tsk);
-	} else if (!io_should_retry_thread(PTR_ERR(tsk))) {
+	} else if (!io_should_retry_thread(worker, PTR_ERR(tsk))) {
 		kfree(worker);
 		goto fail;
 	} else {
-- 
2.44.0


  reply	other threads:[~2024-07-07 16:32 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-07-07 16:32 [PATCH 0/2] fix task_work interation with freezing Pavel Begunkov
2024-07-07 16:32 ` Pavel Begunkov [this message]
2024-07-07 16:32 ` [PATCH 2/2] kernel: rerun task_work while freezing in get_signal() Pavel Begunkov
2024-07-08 10:42   ` Oleg Nesterov
2024-07-08 15:40     ` Pavel Begunkov
2024-07-08 18:48       ` Peter Zijlstra
2024-07-09 10:36       ` Oleg Nesterov
2024-07-09 14:05         ` Pavel Begunkov
2024-07-09 16:39           ` Tejun Heo
2024-07-09 19:07             ` Oleg Nesterov
2024-07-09 19:26               ` Pavel Begunkov
2024-07-09 19:38                 ` Oleg Nesterov
2024-07-09 19:55                   ` Pavel Begunkov
2024-07-10  0:54                     ` Tejun Heo
2024-07-10 17:53                       ` Pavel Begunkov
2024-07-10 19:10                         ` Oleg Nesterov
2024-07-10 19:20                           ` Tejun Heo
2024-07-10 21:34                             ` Oleg Nesterov
2024-07-10 22:01                               ` Tejun Heo
2024-07-10 22:17                                 ` Oleg Nesterov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1af2a697b568f8cf77c5d527d183fa0ffe8fd9cc.1720368770.git.asml.silence@gmail.com \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox