From: Jens Axboe <[email protected]>
To: [email protected]
Cc: Pavel Begunkov <[email protected]>, Jens Axboe <[email protected]>
Subject: [PATCH 15/27] io_uring: clean R_DISABLED startup mess
Date: Wed, 10 Mar 2021 15:43:46 -0700 [thread overview]
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
From: Pavel Begunkov <[email protected]>
There are enough of problems with IORING_SETUP_R_DISABLED, including the
burden of checking and kicking off the SQO task all over the codebase --
for exit/cancel/etc.
Rework it, always start the thread but don't do submit unless the flag
is gone, that's much easier.
Signed-off-by: Pavel Begunkov <[email protected]>
Signed-off-by: Jens Axboe <[email protected]>
---
fs/io_uring.c | 25 ++++++-------------------
1 file changed, 6 insertions(+), 19 deletions(-)
diff --git a/fs/io_uring.c b/fs/io_uring.c
index d4f018f5838d..3f6db813d670 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -6606,7 +6606,8 @@ static int __io_sq_thread(struct io_ring_ctx *ctx, bool cap_entries)
if (!list_empty(&ctx->iopoll_list))
io_do_iopoll(ctx, &nr_events, 0);
- if (to_submit && likely(!percpu_ref_is_dying(&ctx->refs)))
+ if (to_submit && likely(!percpu_ref_is_dying(&ctx->refs)) &&
+ !(ctx->flags & IORING_SETUP_R_DISABLED))
ret = io_submit_sqes(ctx, to_submit);
mutex_unlock(&ctx->uring_lock);
}
@@ -7861,6 +7862,7 @@ static int io_sq_offload_create(struct io_ring_ctx *ctx,
wake_up_new_task(tsk);
if (ret)
goto err;
+ complete(&sqd->startup);
} else if (p->flags & IORING_SETUP_SQ_AFF) {
/* Can't have SQ_AFF without SQPOLL */
ret = -EINVAL;
@@ -7873,15 +7875,6 @@ static int io_sq_offload_create(struct io_ring_ctx *ctx,
return ret;
}
-static void io_sq_offload_start(struct io_ring_ctx *ctx)
-{
- struct io_sq_data *sqd = ctx->sq_data;
-
- ctx->flags &= ~IORING_SETUP_R_DISABLED;
- if (ctx->flags & IORING_SETUP_SQPOLL)
- complete(&sqd->startup);
-}
-
static inline void __io_unaccount_mem(struct user_struct *user,
unsigned long nr_pages)
{
@@ -8742,11 +8735,6 @@ static void io_uring_cancel_task_requests(struct io_ring_ctx *ctx,
struct task_struct *task = current;
if ((ctx->flags & IORING_SETUP_SQPOLL) && ctx->sq_data) {
- /* never started, nothing to cancel */
- if (ctx->flags & IORING_SETUP_R_DISABLED) {
- io_sq_offload_start(ctx);
- return;
- }
io_sq_thread_park(ctx->sq_data);
task = ctx->sq_data->thread;
if (task)
@@ -9449,9 +9437,6 @@ static int io_uring_create(unsigned entries, struct io_uring_params *p,
if (ret)
goto err;
- if (!(p->flags & IORING_SETUP_R_DISABLED))
- io_sq_offload_start(ctx);
-
memset(&p->sq_off, 0, sizeof(p->sq_off));
p->sq_off.head = offsetof(struct io_rings, sq.head);
p->sq_off.tail = offsetof(struct io_rings, sq.tail);
@@ -9668,7 +9653,9 @@ static int io_register_enable_rings(struct io_ring_ctx *ctx)
if (ctx->restrictions.registered)
ctx->restricted = 1;
- io_sq_offload_start(ctx);
+ ctx->flags &= ~IORING_SETUP_R_DISABLED;
+ if (ctx->sq_data && wq_has_sleeper(&ctx->sq_data->wait))
+ wake_up(&ctx->sq_data->wait);
return 0;
}
--
2.30.2
next prev parent reply other threads:[~2021-03-10 22:45 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-10 22:43 [PATCHSET for 5.12] Patches queued up for 5.12 Jens Axboe
2021-03-10 22:43 ` [PATCH 01/27] io-wq: fix race in freeing 'wq' and worker access Jens Axboe
2021-03-10 22:43 ` [PATCH 02/27] io-wq: always track creds for async issue Jens Axboe
2021-03-10 22:43 ` [PATCH 03/27] io_uring: make del_task_file more forgiving Jens Axboe
2021-03-10 22:43 ` [PATCH 04/27] io_uring: introduce ctx to tctx back map Jens Axboe
2021-03-10 22:43 ` [PATCH 05/27] io_uring: do ctx initiated file note removal Jens Axboe
2021-03-10 22:43 ` [PATCH 06/27] io_uring: don't take task ring-file notes Jens Axboe
2021-03-10 22:43 ` [PATCH 07/27] io_uring: index io_uring->xa by ctx not file Jens Axboe
2021-03-10 22:43 ` [PATCH 08/27] io_uring: warn when ring exit takes too long Jens Axboe
2021-03-10 22:43 ` [PATCH 09/27] io_uring: cancel reqs of all iowq's on ring exit Jens Axboe
2021-03-10 22:43 ` [PATCH 10/27] io-wq: warn on creating manager while exiting Jens Axboe
2021-03-10 22:43 ` [PATCH 11/27] io_uring: run __io_sq_thread() with the initial creds from io_uring_setup() Jens Axboe
2021-03-10 22:43 ` [PATCH 12/27] io_uring: kill io_sq_thread_fork() and return -EOWNERDEAD if the sq_thread is gone Jens Axboe
2021-03-10 22:43 ` [PATCH 13/27] io_uring: SQPOLL parking fixes Jens Axboe
2021-03-10 22:43 ` [PATCH 14/27] io_uring: fix unrelated ctx reqs cancellation Jens Axboe
2021-03-10 22:43 ` Jens Axboe [this message]
2021-03-10 22:43 ` [PATCH 16/27] io_uring: Convert personality_idr to XArray Jens Axboe
2021-03-10 22:43 ` [PATCH 17/27] io-wq: remove unused 'user' member of io_wq Jens Axboe
2021-03-10 22:43 ` [PATCH 18/27] io_uring: fix io_sq_offload_create error handling Jens Axboe
2021-03-10 22:43 ` [PATCH 19/27] io_uring: add io_disarm_next() helper Jens Axboe
2021-03-10 22:43 ` [PATCH 20/27] io_uring: fix complete_post races for linked req Jens Axboe
2021-03-10 22:43 ` [PATCH 21/27] io-wq: fix ref leak for req in case of exit cancelations Jens Axboe
2021-03-10 22:43 ` [PATCH 22/27] io_uring: move all io_kiocb init early in io_init_req() Jens Axboe
2021-03-10 22:43 ` [PATCH 23/27] io_uring: remove unneeded variable 'ret' Jens Axboe
2021-03-10 22:43 ` [PATCH 24/27] io_uring: always wait for sqd exited when stopping SQPOLL thread Jens Axboe
2021-03-10 22:43 ` [PATCH 25/27] kernel: make IO threads unfreezable by default Jens Axboe
2021-03-10 22:43 ` [PATCH 26/27] io_uring: fix invalid ctx->sq_thread_idle Jens Axboe
2021-03-10 22:43 ` [PATCH 27/27] io_uring: remove indirect ctx into sqo injection Jens Axboe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox