public inbox for [email protected]
 help / color / mirror / Atom feed
From: Stefan Roesch <[email protected]>
To: <[email protected]>, <[email protected]>,
	<[email protected]>
Cc: <[email protected]>, <[email protected]>
Subject: [PATCH v3 06/12] io_uring: modify io_get_cqe for CQE32
Date: Mon, 25 Apr 2022 11:25:24 -0700	[thread overview]
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>

Modify accesses to the CQE array to take large CQE's into account. The
index needs to be shifted by one for large CQE's.

Signed-off-by: Stefan Roesch <[email protected]>
Signed-off-by: Jens Axboe <[email protected]>
---
 fs/io_uring.c | 19 +++++++++++++++++--
 1 file changed, 17 insertions(+), 2 deletions(-)

diff --git a/fs/io_uring.c b/fs/io_uring.c
index f300130fd9f0..726238dc65dc 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -1909,8 +1909,12 @@ static noinline struct io_uring_cqe *__io_get_cqe(struct io_ring_ctx *ctx)
 {
 	struct io_rings *rings = ctx->rings;
 	unsigned int off = ctx->cached_cq_tail & (ctx->cq_entries - 1);
+	unsigned int shift = 0;
 	unsigned int free, queued, len;
 
+	if (ctx->flags & IORING_SETUP_CQE32)
+		shift = 1;
+
 	/* userspace may cheat modifying the tail, be safe and do min */
 	queued = min(__io_cqring_events(ctx), ctx->cq_entries);
 	free = ctx->cq_entries - queued;
@@ -1922,15 +1926,26 @@ static noinline struct io_uring_cqe *__io_get_cqe(struct io_ring_ctx *ctx)
 	ctx->cached_cq_tail++;
 	ctx->cqe_cached = &rings->cqes[off];
 	ctx->cqe_sentinel = ctx->cqe_cached + len;
-	return ctx->cqe_cached++;
+	ctx->cqe_cached++;
+	return &rings->cqes[off << shift];
 }
 
 static inline struct io_uring_cqe *io_get_cqe(struct io_ring_ctx *ctx)
 {
 	if (likely(ctx->cqe_cached < ctx->cqe_sentinel)) {
+		struct io_uring_cqe *cqe = ctx->cqe_cached;
+
+		if (ctx->flags & IORING_SETUP_CQE32) {
+			unsigned int off = ctx->cqe_cached - ctx->rings->cqes;
+
+			cqe += off;
+		}
+
 		ctx->cached_cq_tail++;
-		return ctx->cqe_cached++;
+		ctx->cqe_cached++;
+		return cqe;
 	}
+
 	return __io_get_cqe(ctx);
 }
 
-- 
2.30.2


  parent reply	other threads:[~2022-04-25 18:26 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <CGME20220425182557epcas5p2e1b72edf0fcc4c21b2b96a32910a2736@epcas5p2.samsung.com>
2022-04-25 18:25 ` [PATCH v3 00/12] add large CQE support for io-uring Stefan Roesch
2022-04-25 18:25   ` [PATCH v3 01/12] io_uring: support CQE32 in io_uring_cqe Stefan Roesch
2022-04-26  5:22     ` Kanchan Joshi
2022-04-25 18:25   ` [PATCH v3 02/12] io_uring: wire up inline completion path for CQE32 Stefan Roesch
2022-04-26  5:45     ` Kanchan Joshi
2022-04-25 18:25   ` [PATCH v3 03/12] io_uring: change ring size calculation " Stefan Roesch
2022-04-25 18:25   ` [PATCH v3 04/12] io_uring: add CQE32 setup processing Stefan Roesch
2022-04-25 18:25   ` [PATCH v3 05/12] io_uring: add CQE32 completion processing Stefan Roesch
2022-04-25 18:25   ` Stefan Roesch [this message]
2022-04-25 18:25   ` [PATCH v3 07/12] io_uring: flush completions for CQE32 Stefan Roesch
2022-04-25 18:25   ` [PATCH v3 08/12] io_uring: overflow processing " Stefan Roesch
2022-04-26  6:28     ` Kanchan Joshi
2022-04-26 12:53       ` Jens Axboe
2022-04-25 18:25   ` [PATCH v3 09/12] io_uring: add tracing for additional CQE32 fields Stefan Roesch
2022-04-25 18:25   ` [PATCH v3 10/12] io_uring: support CQE32 in /proc info Stefan Roesch
2022-04-25 18:25   ` [PATCH v3 11/12] io_uring: enable CQE32 Stefan Roesch
2022-04-25 18:25   ` [PATCH v3 12/12] io_uring: support CQE32 for nop operation Stefan Roesch
2022-04-25 18:38   ` [PATCH v3 00/12] add large CQE support for io-uring Jens Axboe
2022-04-26 11:37   ` Kanchan Joshi
2022-04-26 12:54     ` Jens Axboe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox