public inbox for [email protected]
 help / color / mirror / Atom feed
From: Pavel Begunkov <[email protected]>
To: [email protected]
Cc: Jens Axboe <[email protected]>, [email protected]
Subject: [PATCH for-next 03/10] io_uring: better caching for ctx timeout fields
Date: Wed, 15 Jun 2022 17:33:49 +0100	[thread overview]
Message-ID: <4b163793072840de53b3cb66e0c2995e7226ff78.1655310733.git.asml.silence@gmail.com> (raw)
In-Reply-To: <[email protected]>

Following timeout fields access patterns, move all of them into a
separate cache line inside ctx, so they don't intervene with normal
completion caching, especially since timeout removals and completion
are separated and the later is done via tw.

It also sheds some bytes from io_ring_ctx, 1216B -> 1152B

Signed-off-by: Pavel Begunkov <[email protected]>
---
 io_uring/io_uring_types.h | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/io_uring/io_uring_types.h b/io_uring/io_uring_types.h
index bff73107f0f3..e050c3c4a7ac 100644
--- a/io_uring/io_uring_types.h
+++ b/io_uring/io_uring_types.h
@@ -179,8 +179,6 @@ struct io_ring_ctx {
 		struct xarray		io_bl_xa;
 		struct list_head	io_buffers_cache;
 
-		struct list_head	timeout_list;
-		struct list_head	ltimeout_list;
 		struct list_head	cq_overflow_list;
 		struct list_head	apoll_cache;
 		struct xarray		personalities;
@@ -213,15 +211,11 @@ struct io_ring_ctx {
 		struct io_ev_fd	__rcu	*io_ev_fd;
 		struct wait_queue_head	cq_wait;
 		unsigned		cq_extra;
-		atomic_t		cq_timeouts;
-		unsigned		cq_last_tm_flush;
 	} ____cacheline_aligned_in_smp;
 
 	struct {
 		spinlock_t		completion_lock;
 
-		spinlock_t		timeout_lock;
-
 		/*
 		 * ->iopoll_list is protected by the ctx->uring_lock for
 		 * io_uring instances that don't use IORING_SETUP_SQPOLL.
@@ -253,6 +247,15 @@ struct io_ring_ctx {
 		struct list_head	io_buffers_pages;
 	};
 
+	/* timeouts */
+	struct {
+		spinlock_t		timeout_lock;
+		atomic_t		cq_timeouts;
+		struct list_head	timeout_list;
+		struct list_head	ltimeout_list;
+		unsigned		cq_last_tm_flush;
+	} ____cacheline_aligned_in_smp;
+
 	/* Keep this last, we don't need it for the fast path */
 	struct {
 		#if defined(CONFIG_UNIX)
-- 
2.36.1



  parent reply	other threads:[~2022-06-15 16:34 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-06-15 16:33 [PATCH for-next 00/10] 5.20 cleanups Pavel Begunkov
2022-06-15 16:33 ` [PATCH for-next 01/10] io_uring: make reg buf init consistent Pavel Begunkov
2022-06-15 16:33 ` [PATCH for-next 02/10] io_uring: move defer_list to slow data Pavel Begunkov
2022-06-15 16:33 ` Pavel Begunkov [this message]
2022-06-15 16:33 ` [PATCH for-next 04/10] io_uring: refactor ctx slow data placement Pavel Begunkov
2022-06-15 16:33 ` [PATCH for-next 05/10] io_uring: move small helpers to headers Pavel Begunkov
2022-06-15 16:33 ` [PATCH for-next 06/10] io_uring: explain io_wq_work::cancel_seq placement Pavel Begunkov
2022-06-15 16:33 ` [PATCH for-next 07/10] io_uring: inline ->registered_rings Pavel Begunkov
2022-06-15 16:33 ` [PATCH for-next 08/10] io_uring: never defer-complete multi-apoll Pavel Begunkov
2022-06-15 16:33 ` [PATCH for-next 09/10] io_uring: remove check_cq checking from hot paths Pavel Begunkov
2022-06-15 16:33 ` [PATCH for-next 10/10] io_uring: don't set REQ_F_COMPLETE_INLINE in tw Pavel Begunkov
2022-06-15 21:30 ` [PATCH for-next 00/10] 5.20 cleanups Jens Axboe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=4b163793072840de53b3cb66e0c2995e7226ff78.1655310733.git.asml.silence@gmail.com \
    [email protected] \
    [email protected] \
    [email protected] \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox