public inbox for [email protected]
 help / color / mirror / Atom feed
From: Jens Axboe <[email protected]>
To: [email protected]
Cc: Jens Axboe <[email protected]>,
	[email protected]
Subject: [PATCH 33/33] io-wq: ensure all pending work is canceled on exit
Date: Wed,  3 Mar 2021 17:27:00 -0700	[thread overview]
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>

If we race on shutting down the io-wq, then we should ensure that any
work that was queued after workers shutdown is canceled. Harden the
add work check a bit too, checking for IO_WQ_BIT_EXIT and cancel if
it's set.

Add a WARN_ON() for having any work before we kill the io-wq context.

Reported-by: [email protected]
Signed-off-by: Jens Axboe <[email protected]>
---
 fs/io-wq.c | 42 +++++++++++++++++++++++++++++++++---------
 1 file changed, 33 insertions(+), 9 deletions(-)

diff --git a/fs/io-wq.c b/fs/io-wq.c
index a44bd22c045e..6d57f9b80213 100644
--- a/fs/io-wq.c
+++ b/fs/io-wq.c
@@ -129,6 +129,17 @@ struct io_wq {
 
 static enum cpuhp_state io_wq_online;
 
+struct io_cb_cancel_data {
+	work_cancel_fn *fn;
+	void *data;
+	int nr_running;
+	int nr_pending;
+	bool cancel_all;
+};
+
+static void io_wqe_cancel_pending_work(struct io_wqe *wqe,
+				       struct io_cb_cancel_data *match);
+
 static bool io_worker_get(struct io_worker *worker)
 {
 	return refcount_inc_not_zero(&worker->ref);
@@ -713,6 +724,23 @@ static void io_wq_check_workers(struct io_wq *wq)
 	}
 }
 
+static bool io_wq_work_match_all(struct io_wq_work *work, void *data)
+{
+	return true;
+}
+
+static void io_wq_cancel_pending(struct io_wq *wq)
+{
+	struct io_cb_cancel_data match = {
+		.fn		= io_wq_work_match_all,
+		.cancel_all	= true,
+	};
+	int node;
+
+	for_each_node(node)
+		io_wqe_cancel_pending_work(wq->wqes[node], &match);
+}
+
 /*
  * Manager thread. Tasked with creating new workers, if we need them.
  */
@@ -748,6 +776,8 @@ static int io_wq_manager(void *data)
 	/* we might not ever have created any workers */
 	if (atomic_read(&wq->worker_refs))
 		wait_for_completion(&wq->worker_done);
+
+	io_wq_cancel_pending(wq);
 	complete(&wq->exited);
 	do_exit(0);
 }
@@ -809,7 +839,8 @@ static void io_wqe_enqueue(struct io_wqe *wqe, struct io_wq_work *work)
 	unsigned long flags;
 
 	/* Can only happen if manager creation fails after exec */
-	if (unlikely(io_wq_fork_manager(wqe->wq))) {
+	if (io_wq_fork_manager(wqe->wq) ||
+	    test_bit(IO_WQ_BIT_EXIT, &wqe->wq->state)) {
 		work->flags |= IO_WQ_WORK_CANCEL;
 		wqe->wq->do_work(work);
 		return;
@@ -845,14 +876,6 @@ void io_wq_hash_work(struct io_wq_work *work, void *val)
 	work->flags |= (IO_WQ_WORK_HASHED | (bit << IO_WQ_HASH_SHIFT));
 }
 
-struct io_cb_cancel_data {
-	work_cancel_fn *fn;
-	void *data;
-	int nr_running;
-	int nr_pending;
-	bool cancel_all;
-};
-
 static bool io_wq_worker_cancel(struct io_worker *worker, void *data)
 {
 	struct io_cb_cancel_data *match = data;
@@ -1086,6 +1109,7 @@ static void io_wq_destroy(struct io_wq *wq)
 		struct io_wqe *wqe = wq->wqes[node];
 
 		list_del_init(&wqe->wait.entry);
+		WARN_ON_ONCE(!wq_list_empty(&wqe->work_list));
 		kfree(wqe);
 	}
 	spin_unlock_irq(&wq->hash->wait.lock);
-- 
2.30.1


      parent reply	other threads:[~2021-03-04  1:10 UTC|newest]

Thread overview: 36+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-03-04  0:26 [PATCHSET 0/33] Fixes queued up for 5.12 Jens Axboe
2021-03-04  0:26 ` [PATCH 01/33] io-wq: wait for worker startup when forking a new one Jens Axboe
2021-03-04  0:26 ` [PATCH 02/33] io-wq: have manager wait for all workers to exit Jens Axboe
2021-03-04  0:26 ` [PATCH 03/33] io-wq: don't ask for a new worker if we're exiting Jens Axboe
2021-03-04  0:26 ` [PATCH 04/33] io-wq: rename wq->done completion to wq->started Jens Axboe
2021-03-04  0:26 ` [PATCH 05/33] io-wq: wait for manager exit on wq destroy Jens Axboe
2021-03-04  0:26 ` [PATCH 06/33] io-wq: fix double put of 'wq' in error path Jens Axboe
2021-03-04  0:26 ` [PATCH 07/33] io_uring: SQPOLL stop error handling fixes Jens Axboe
2021-03-04  0:26 ` [PATCH 08/33] io_uring: run fallback on cancellation Jens Axboe
2021-03-04  0:26 ` [PATCH 09/33] io_uring: don't use complete_all() on SQPOLL thread exit Jens Axboe
2021-03-04  0:26 ` [PATCH 10/33] io-wq: provide an io_wq_put_and_exit() helper Jens Axboe
2021-03-04  0:26 ` [PATCH 11/33] io_uring: fix race condition in task_work add and clear Jens Axboe
2021-03-04  0:26 ` [PATCH 12/33] io_uring: signal worker thread unshare Jens Axboe
2021-03-04 12:15   ` Stefan Metzmacher
2021-03-04 14:05     ` Jens Axboe
2021-03-04  0:26 ` [PATCH 13/33] io_uring: warn on not destroyed io-wq Jens Axboe
2021-03-04  0:26 ` [PATCH 14/33] io_uring: destroy io-wq on exec Jens Axboe
2021-03-04  0:26 ` [PATCH 15/33] io_uring: remove unused argument 'tsk' from io_req_caches_free() Jens Axboe
2021-03-04  0:26 ` [PATCH 16/33] io_uring: kill unnecessary REQ_F_WORK_INITIALIZED checks Jens Axboe
2021-03-04  0:26 ` [PATCH 17/33] io_uring: move cred assignment into io_issue_sqe() Jens Axboe
2021-03-04  0:26 ` [PATCH 18/33] io_uring: kill unnecessary io_run_ctx_fallback() in io_ring_exit_work() Jens Axboe
2021-03-04  0:26 ` [PATCH 19/33] io_uring: kill io_uring_flush() Jens Axboe
2021-03-04  0:26 ` [PATCH 20/33] io_uring: fix __tctx_task_work() ctx race Jens Axboe
2021-03-04  0:26 ` [PATCH 21/33] io_uring: replace cmpxchg in fallback with xchg Jens Axboe
2021-03-04  0:26 ` [PATCH 22/33] io_uring: ensure that SQPOLL thread is started for exit Jens Axboe
2021-03-04  0:26 ` [PATCH 23/33] io_uring: ignore double poll add on the same waitqueue head Jens Axboe
2021-03-04  0:26 ` [PATCH 24/33] io_uring: kill sqo_dead and sqo submission halting Jens Axboe
2021-03-04  0:26 ` [PATCH 25/33] io_uring: remove sqo_task Jens Axboe
2021-03-04  0:26 ` [PATCH 26/33] io-wq: fix error path leak of buffered write hash map Jens Axboe
2021-03-04  0:26 ` [PATCH 27/33] io_uring: fix -EAGAIN retry with IOPOLL Jens Axboe
2021-03-04  0:26 ` [PATCH 28/33] io_uring: choose right tctx->io_wq for try cancel Jens Axboe
2021-03-04  0:26 ` [PATCH 29/33] io_uring: inline io_req_clean_work() Jens Axboe
2021-03-04  0:26 ` [PATCH 30/33] io_uring: inline __io_queue_async_work() Jens Axboe
2021-03-04  0:26 ` [PATCH 31/33] io_uring: remove extra in_idle wake up Jens Axboe
2021-03-04  0:26 ` [PATCH 32/33] io_uring: ensure that threads freeze on suspend Jens Axboe
2021-03-04  0:27 ` Jens Axboe [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox