From: Pavel Begunkov <[email protected]>
To: [email protected]
Cc: Jens Axboe <[email protected]>, [email protected]
Subject: [PATCH for-next 2/7] io_uring: don't check overflow flush failures
Date: Mon, 5 Dec 2022 02:44:26 +0000 [thread overview]
Message-ID: <e64f98e7468525a2e475243cc8c0e67b2ecb9127.1670207706.git.asml.silence@gmail.com> (raw)
In-Reply-To: <[email protected]>
The only way to fail overflowed CQEs flush is for CQ to be fully packed.
There is one place checking for flush failures, i.e. io_cqring_wait(),
but we limit the number to be waited for by the CQ size, so getting a
failure automatically means that we're done with waiting.
Don't check for failures, rarely but they might spuriously fail CQ
waiting with -EBUSY.
Signed-off-by: Pavel Begunkov <[email protected]>
---
io_uring/io_uring.c | 24 ++++++------------------
1 file changed, 6 insertions(+), 18 deletions(-)
diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c
index 4721ff6cafaa..7239776a9d4b 100644
--- a/io_uring/io_uring.c
+++ b/io_uring/io_uring.c
@@ -629,13 +629,12 @@ static void io_cqring_overflow_kill(struct io_ring_ctx *ctx)
}
/* Returns true if there are no backlogged entries after the flush */
-static bool __io_cqring_overflow_flush(struct io_ring_ctx *ctx)
+static void __io_cqring_overflow_flush(struct io_ring_ctx *ctx)
{
- bool all_flushed;
size_t cqe_size = sizeof(struct io_uring_cqe);
if (__io_cqring_events(ctx) == ctx->cq_entries)
- return false;
+ return;
if (ctx->flags & IORING_SETUP_CQE32)
cqe_size <<= 1;
@@ -654,30 +653,23 @@ static bool __io_cqring_overflow_flush(struct io_ring_ctx *ctx)
kfree(ocqe);
}
- all_flushed = list_empty(&ctx->cq_overflow_list);
- if (all_flushed) {
+ if (list_empty(&ctx->cq_overflow_list)) {
clear_bit(IO_CHECK_CQ_OVERFLOW_BIT, &ctx->check_cq);
atomic_andnot(IORING_SQ_CQ_OVERFLOW, &ctx->rings->sq_flags);
}
-
io_cq_unlock_post(ctx);
- return all_flushed;
}
-static bool io_cqring_overflow_flush(struct io_ring_ctx *ctx)
+static void io_cqring_overflow_flush(struct io_ring_ctx *ctx)
{
- bool ret = true;
-
if (test_bit(IO_CHECK_CQ_OVERFLOW_BIT, &ctx->check_cq)) {
/* iopoll syncs against uring_lock, not completion_lock */
if (ctx->flags & IORING_SETUP_IOPOLL)
mutex_lock(&ctx->uring_lock);
- ret = __io_cqring_overflow_flush(ctx);
+ __io_cqring_overflow_flush(ctx);
if (ctx->flags & IORING_SETUP_IOPOLL)
mutex_unlock(&ctx->uring_lock);
}
-
- return ret;
}
void __io_put_task(struct task_struct *task, int nr)
@@ -2505,11 +2497,7 @@ static int io_cqring_wait(struct io_ring_ctx *ctx, int min_events,
trace_io_uring_cqring_wait(ctx, min_events);
do {
- /* if we can't even flush overflow, don't wait for more */
- if (!io_cqring_overflow_flush(ctx)) {
- ret = -EBUSY;
- break;
- }
+ io_cqring_overflow_flush(ctx);
prepare_to_wait_exclusive(&ctx->cq_wait, &iowq.wq,
TASK_INTERRUPTIBLE);
ret = io_cqring_wait_schedule(ctx, &iowq, timeout);
--
2.38.1
next prev parent reply other threads:[~2022-12-05 2:45 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-12-05 2:44 [PATCH for-next 0/7] CQ locking optimisation Pavel Begunkov
2022-12-05 2:44 ` [PATCH for-next 1/7] io_uring: skip overflow CQE posting for dying ring Pavel Begunkov
2022-12-05 2:44 ` Pavel Begunkov [this message]
2022-12-05 2:44 ` [PATCH for-next 3/7] io_uring: complete all requests in task context Pavel Begunkov
2022-12-05 2:44 ` [PATCH for-next 4/7] io_uring: force multishot CQEs into " Pavel Begunkov
2022-12-05 2:44 ` [PATCH for-next 5/7] io_uring: post msg_ring CQE in " Pavel Begunkov
2022-12-05 11:53 ` Jens Axboe
2022-12-05 15:12 ` Dylan Yudaken
2022-12-05 15:18 ` Jens Axboe
2022-12-06 10:42 ` Pavel Begunkov
2022-12-06 16:06 ` Jens Axboe
2022-12-07 3:59 ` Pavel Begunkov
2022-12-07 15:42 ` Jens Axboe
2022-12-05 2:44 ` [PATCH for-next 6/7] io_uring: use tw for putting rsrc Pavel Begunkov
2022-12-05 2:44 ` [PATCH for-next 7/7] io_uring: skip spinlocking for ->task_complete Pavel Begunkov
2022-12-06 16:53 ` [PATCH for-next 0/7] CQ locking optimisation Jens Axboe
2022-12-06 17:17 ` Jens Axboe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e64f98e7468525a2e475243cc8c0e67b2ecb9127.1670207706.git.asml.silence@gmail.com \
[email protected] \
[email protected] \
[email protected] \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox