From: Pavel Begunkov <[email protected]>
To: Jens Axboe <[email protected]>, [email protected]
Subject: [PATCH 7/9] io_uring: don't fail overflow on in_idle
Date: Tue, 13 Apr 2021 02:58:44 +0100 [thread overview]
Message-ID: <d873b7dab75c7f3039ead9628a745bea01f2cfd2.1618278933.git.asml.silence@gmail.com> (raw)
In-Reply-To: <[email protected]>
As CQE overflows are now untied from requests and so don't hold any
ref, we don't need to handle exiting/exec'ing cases there anymore.
Moreover, it's much nicer in regards to userspace to save overflowed
CQEs whenever possible, so remove failing on in_idle.
Signed-off-by: Pavel Begunkov <[email protected]>
---
fs/io_uring.c | 44 ++++++++++++++++++++------------------------
1 file changed, 20 insertions(+), 24 deletions(-)
diff --git a/fs/io_uring.c b/fs/io_uring.c
index 4407ebc8f8d3..cc6a44533802 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -1515,32 +1515,28 @@ static bool io_cqring_event_overflow(struct io_kiocb *req, long res,
unsigned int cflags)
{
struct io_ring_ctx *ctx = req->ctx;
+ struct io_overflow_cqe *ocqe;
- if (!atomic_read(&req->task->io_uring->in_idle)) {
- struct io_overflow_cqe *ocqe;
-
- ocqe = kmalloc(sizeof(*ocqe), GFP_ATOMIC | __GFP_ACCOUNT);
- if (!ocqe)
- goto overflow;
- if (list_empty(&ctx->cq_overflow_list)) {
- set_bit(0, &ctx->sq_check_overflow);
- set_bit(0, &ctx->cq_check_overflow);
- ctx->rings->sq_flags |= IORING_SQ_CQ_OVERFLOW;
- }
- ocqe->cqe.user_data = req->user_data;
- ocqe->cqe.res = res;
- ocqe->cqe.flags = cflags;
- list_add_tail(&ocqe->list, &ctx->cq_overflow_list);
- return true;
+ ocqe = kmalloc(sizeof(*ocqe), GFP_ATOMIC | __GFP_ACCOUNT);
+ if (!ocqe) {
+ /*
+ * If we're in ring overflow flush mode, or in task cancel mode,
+ * or cannot allocate an overflow entry, then we need to drop it
+ * on the floor.
+ */
+ WRITE_ONCE(ctx->rings->cq_overflow, ++ctx->cached_cq_overflow);
+ return false;
}
-overflow:
- /*
- * If we're in ring overflow flush mode, or in task cancel mode,
- * or cannot allocate an overflow entry, then we need to drop it
- * on the floor.
- */
- WRITE_ONCE(ctx->rings->cq_overflow, ++ctx->cached_cq_overflow);
- return false;
+ if (list_empty(&ctx->cq_overflow_list)) {
+ set_bit(0, &ctx->sq_check_overflow);
+ set_bit(0, &ctx->cq_check_overflow);
+ ctx->rings->sq_flags |= IORING_SQ_CQ_OVERFLOW;
+ }
+ ocqe->cqe.user_data = req->user_data;
+ ocqe->cqe.res = res;
+ ocqe->cqe.flags = cflags;
+ list_add_tail(&ocqe->list, &ctx->cq_overflow_list);
+ return true;
}
static inline bool __io_cqring_fill_event(struct io_kiocb *req, long res,
--
2.24.0
next prev parent reply other threads:[~2021-04-13 2:03 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-13 1:58 [PATCH 5.13 0/9] another 5.13 pack Pavel Begunkov
2021-04-13 1:58 ` [PATCH 1/9] io_uring: fix leaking reg files on exit Pavel Begunkov
2021-04-13 1:58 ` [PATCH 2/9] io_uring: fix uninit old data for poll event upd Pavel Begunkov
2021-04-13 1:58 ` [PATCH 3/9] io_uring: split poll and poll update structures Pavel Begunkov
2021-04-13 17:14 ` Pavel Begunkov
2021-04-13 1:58 ` [PATCH 4/9] io_uring: add timeout completion_lock annotation Pavel Begunkov
2021-04-13 1:58 ` [PATCH 5/9] io_uring: refactor hrtimer_try_to_cancel uses Pavel Begunkov
2021-04-13 1:58 ` [PATCH 6/9] io_uring: clean up io_poll_remove_waitqs() Pavel Begunkov
2021-04-13 1:58 ` Pavel Begunkov [this message]
2021-04-13 1:58 ` [PATCH 8/9] io_uring: skip futile iopoll iterations Pavel Begunkov
2021-04-13 1:58 ` [PATCH 9/9] io_uring: inline io_iopoll_getevents() Pavel Begunkov
2021-04-13 15:38 ` [PATCH 5.13 0/9] another 5.13 pack Jens Axboe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=d873b7dab75c7f3039ead9628a745bea01f2cfd2.1618278933.git.asml.silence@gmail.com \
[email protected] \
[email protected] \
[email protected] \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox