From: Pavel Begunkov <[email protected]>
To: [email protected]
Cc: Jens Axboe <[email protected]>, [email protected]
Subject: [PATCH 5/9] io_uring: helper for empty req cache checks
Date: Tue, 12 Apr 2022 15:09:47 +0100 [thread overview]
Message-ID: <b18662389f3fb483d0bd07906647f65f6037475a.1649771823.git.asml.silence@gmail.com> (raw)
In-Reply-To: <[email protected]>
Add io_req_cache_empty(), which checks if there are requests in the
inline req cache or not. It'll be needed in the future, but also nicely
cleans up a few spots poking into ->free_list directly.
Signed-off-by: Pavel Begunkov <[email protected]>
---
fs/io_uring.c | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
diff --git a/fs/io_uring.c b/fs/io_uring.c
index 73422af2dd79..3ccc13acb498 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -2211,6 +2211,11 @@ static void io_flush_cached_locked_reqs(struct io_ring_ctx *ctx,
spin_unlock(&ctx->completion_lock);
}
+static inline bool io_req_cache_empty(struct io_ring_ctx *ctx)
+{
+ return !ctx->submit_state.free_list.next;
+}
+
/*
* A request might get retired back into the request caches even before opcode
* handlers and io_issue_sqe() are done with it, e.g. inline completion path.
@@ -2232,7 +2237,7 @@ static __cold bool __io_alloc_req_refill(struct io_ring_ctx *ctx)
*/
if (READ_ONCE(ctx->locked_free_nr) > IO_COMPL_BATCH) {
io_flush_cached_locked_reqs(ctx, &ctx->submit_state);
- if (state->free_list.next)
+ if (!io_req_cache_empty(ctx))
return true;
}
@@ -2261,7 +2266,7 @@ static __cold bool __io_alloc_req_refill(struct io_ring_ctx *ctx)
static inline bool io_alloc_req_refill(struct io_ring_ctx *ctx)
{
- if (unlikely(!ctx->submit_state.free_list.next))
+ if (unlikely(io_req_cache_empty(ctx)))
return __io_alloc_req_refill(ctx);
return true;
}
@@ -9790,7 +9795,7 @@ static void io_req_caches_free(struct io_ring_ctx *ctx)
mutex_lock(&ctx->uring_lock);
io_flush_cached_locked_reqs(ctx, state);
- while (state->free_list.next) {
+ while (!io_req_cache_empty(ctx)) {
struct io_wq_work_node *node;
struct io_kiocb *req;
--
2.35.1
next prev parent reply other threads:[~2022-04-12 14:10 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-04-12 14:09 [PATCH next 0/9] for-next clean ups and micro optimisation Pavel Begunkov
2022-04-12 14:09 ` [PATCH 1/9] io_uring: explicitly keep a CQE in io_kiocb Pavel Begunkov
2022-04-12 14:09 ` [PATCH 2/9] io_uring: memcpy CQE from req Pavel Begunkov
2022-04-12 14:09 ` [PATCH 3/9] io_uring: shrink final link flush Pavel Begunkov
2022-04-12 14:09 ` [PATCH 4/9] io_uring: inline io_flush_cached_reqs Pavel Begunkov
2022-04-12 14:09 ` Pavel Begunkov [this message]
2022-04-12 14:09 ` [PATCH 6/9] io_uring: add helper to return req to cache list Pavel Begunkov
2022-04-12 14:09 ` [PATCH 7/9] io_uring: optimise submission loop invariant Pavel Begunkov
2022-04-12 14:09 ` [PATCH 8/9] io_uring: optimise submission left counting Pavel Begunkov
2022-04-12 14:09 ` [PATCH 9/9] io_uring: optimise io_get_cqe() Pavel Begunkov
2022-04-12 16:06 ` Florian Schmaus
2022-04-12 16:15 ` Pavel Begunkov
2022-04-12 16:25 ` Jens Axboe
2022-04-12 15:05 ` [PATCH next 0/9] for-next clean ups and micro optimisation Jens Axboe
2022-04-12 15:12 ` Jens Axboe
2022-04-12 16:47 ` Jens Axboe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=b18662389f3fb483d0bd07906647f65f6037475a.1649771823.git.asml.silence@gmail.com \
[email protected] \
[email protected] \
[email protected] \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox