From: Pavel Begunkov <[email protected]>
To: Jens Axboe <[email protected]>, [email protected]
Subject: [PATCH 2/8] io_uring: consolidate putting reqs task
Date: Mon, 25 Jan 2021 11:42:21 +0000 [thread overview]
Message-ID: <aea8455c4d80fcabde71d28fd78a42c375ebd8ab.1611573970.git.asml.silence@gmail.com> (raw)
In-Reply-To: <[email protected]>
We grab a task for each request and while putting it it also have to do
extra work like inflight accounting and waking up that task. This
sequence is duplicated several time, it's good time to add a helper.
More to that, the helper generates better code due to better locality
and so not failing alias analysis.
Signed-off-by: Pavel Begunkov <[email protected]>
---
fs/io_uring.c | 34 ++++++++++++++--------------------
1 file changed, 14 insertions(+), 20 deletions(-)
diff --git a/fs/io_uring.c b/fs/io_uring.c
index 8be7b4c6d304..fa421555ab8b 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -2013,17 +2013,22 @@ static void io_dismantle_req(struct io_kiocb *req)
io_req_clean_work(req);
}
+static inline void io_put_task(struct task_struct *task, int nr)
+{
+ struct io_uring_task *tctx = task->io_uring;
+
+ percpu_counter_sub(&tctx->inflight, nr);
+ if (unlikely(atomic_read(&tctx->in_idle)))
+ wake_up(&tctx->wait);
+ put_task_struct_many(task, nr);
+}
+
static void __io_free_req(struct io_kiocb *req)
{
- struct io_uring_task *tctx = req->task->io_uring;
struct io_ring_ctx *ctx = req->ctx;
io_dismantle_req(req);
-
- percpu_counter_dec(&tctx->inflight);
- if (atomic_read(&tctx->in_idle))
- wake_up(&tctx->wait);
- put_task_struct(req->task);
+ io_put_task(req->task, 1);
if (likely(!io_is_fallback_req(req)))
kmem_cache_free(req_cachep, req);
@@ -2277,12 +2282,7 @@ static void io_req_free_batch_finish(struct io_ring_ctx *ctx,
if (rb->to_free)
__io_req_free_batch_flush(ctx, rb);
if (rb->task) {
- struct io_uring_task *tctx = rb->task->io_uring;
-
- percpu_counter_sub(&tctx->inflight, rb->task_refs);
- if (atomic_read(&tctx->in_idle))
- wake_up(&tctx->wait);
- put_task_struct_many(rb->task, rb->task_refs);
+ io_put_task(rb->task, rb->task_refs);
rb->task = NULL;
}
}
@@ -2296,14 +2296,8 @@ static void io_req_free_batch(struct req_batch *rb, struct io_kiocb *req)
io_queue_next(req);
if (req->task != rb->task) {
- if (rb->task) {
- struct io_uring_task *tctx = rb->task->io_uring;
-
- percpu_counter_sub(&tctx->inflight, rb->task_refs);
- if (atomic_read(&tctx->in_idle))
- wake_up(&tctx->wait);
- put_task_struct_many(rb->task, rb->task_refs);
- }
+ if (rb->task)
+ io_put_task(rb->task, rb->task_refs);
rb->task = req->task;
rb->task_refs = 0;
}
--
2.24.0
next prev parent reply other threads:[~2021-01-26 5:10 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-25 11:42 [PATCH 0/8] second part of 5.12 patches Pavel Begunkov
2021-01-25 11:42 ` [PATCH 1/8] io_uring: ensure only sqo_task has file notes Pavel Begunkov
2021-01-25 11:42 ` Pavel Begunkov [this message]
2021-01-25 11:42 ` [PATCH 3/8] io_uring: don't keep submit_state on stack Pavel Begunkov
2021-01-25 16:00 ` Jens Axboe
2021-01-25 16:25 ` Pavel Begunkov
2021-01-25 16:31 ` Jens Axboe
2021-01-25 16:52 ` Pavel Begunkov
2021-01-25 11:42 ` [PATCH 4/8] io_uring: remove ctx from comp_state Pavel Begunkov
2021-01-25 11:42 ` [PATCH 5/8] io_uring: don't reinit submit state every time Pavel Begunkov
2021-01-25 11:42 ` [PATCH 6/8] io_uring: replace list with array for compl batch Pavel Begunkov
2021-01-25 11:42 ` [PATCH 7/8] io_uring: submit-completion free batching Pavel Begunkov
2021-01-25 11:42 ` [PATCH 8/8] io_uring: keep interrupts on on submit completion Pavel Begunkov
2021-01-25 16:02 ` Jens Axboe
2021-01-25 16:08 ` [PATCH 0/8] second part of 5.12 patches Jens Axboe
2021-01-25 16:56 ` Pavel Begunkov
2021-01-25 17:04 ` Jens Axboe
2021-01-25 17:05 ` Pavel Begunkov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=aea8455c4d80fcabde71d28fd78a42c375ebd8ab.1611573970.git.asml.silence@gmail.com \
[email protected] \
[email protected] \
[email protected] \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox