public inbox for [email protected]
 help / color / mirror / Atom feed
From: Stefan Roesch <[email protected]>
To: <[email protected]>, <[email protected]>,
	<[email protected]>
Cc: <[email protected]>, <[email protected]>
Subject: [PATCH v3 04/12] io_uring: add CQE32 setup processing
Date: Mon, 25 Apr 2022 11:25:22 -0700	[thread overview]
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>

This adds two new function to setup and fill the CQE32 result structure.

Signed-off-by: Stefan Roesch <[email protected]>
Signed-off-by: Jens Axboe <[email protected]>
---
 fs/io_uring.c | 58 +++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 58 insertions(+)

diff --git a/fs/io_uring.c b/fs/io_uring.c
index 9712483d3a17..8cb51676d38d 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -2175,12 +2175,70 @@ static inline bool __io_fill_cqe_req_filled(struct io_ring_ctx *ctx,
 					req->cqe.res, req->cqe.flags);
 }
 
+static inline bool __io_fill_cqe32_req_filled(struct io_ring_ctx *ctx,
+					      struct io_kiocb *req)
+{
+	struct io_uring_cqe *cqe;
+	u64 extra1 = req->extra1;
+	u64 extra2 = req->extra2;
+
+	trace_io_uring_complete(req->ctx, req, req->cqe.user_data,
+				req->cqe.res, req->cqe.flags);
+
+	/*
+	 * If we can't get a cq entry, userspace overflowed the
+	 * submission (by quite a lot). Increment the overflow count in
+	 * the ring.
+	 */
+	cqe = io_get_cqe(ctx);
+	if (likely(cqe)) {
+		memcpy(cqe, &req->cqe, sizeof(struct io_uring_cqe));
+		cqe->big_cqe[0] = extra1;
+		cqe->big_cqe[1] = extra2;
+		return true;
+	}
+
+	return io_cqring_event_overflow(ctx, req->cqe.user_data,
+					req->cqe.res, req->cqe.flags);
+}
+
 static inline bool __io_fill_cqe_req(struct io_kiocb *req, s32 res, u32 cflags)
 {
 	trace_io_uring_complete(req->ctx, req, req->cqe.user_data, res, cflags);
 	return __io_fill_cqe(req->ctx, req->cqe.user_data, res, cflags);
 }
 
+static inline void __io_fill_cqe32_req(struct io_kiocb *req, s32 res, u32 cflags,
+				u64 extra1, u64 extra2)
+{
+	struct io_ring_ctx *ctx = req->ctx;
+	struct io_uring_cqe *cqe;
+
+	if (WARN_ON_ONCE(!(ctx->flags & IORING_SETUP_CQE32)))
+		return;
+	if (req->flags & REQ_F_CQE_SKIP)
+		return;
+
+	trace_io_uring_complete(ctx, req, req->cqe.user_data, res, cflags);
+
+	/*
+	 * If we can't get a cq entry, userspace overflowed the
+	 * submission (by quite a lot). Increment the overflow count in
+	 * the ring.
+	 */
+	cqe = io_get_cqe(ctx);
+	if (likely(cqe)) {
+		WRITE_ONCE(cqe->user_data, req->cqe.user_data);
+		WRITE_ONCE(cqe->res, res);
+		WRITE_ONCE(cqe->flags, cflags);
+		WRITE_ONCE(cqe->big_cqe[0], extra1);
+		WRITE_ONCE(cqe->big_cqe[1], extra2);
+		return;
+	}
+
+	io_cqring_event_overflow(ctx, req->cqe.user_data, res, cflags);
+}
+
 static noinline bool io_fill_cqe_aux(struct io_ring_ctx *ctx, u64 user_data,
 				     s32 res, u32 cflags)
 {
-- 
2.30.2


  parent reply	other threads:[~2022-04-25 18:25 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <CGME20220425182557epcas5p2e1b72edf0fcc4c21b2b96a32910a2736@epcas5p2.samsung.com>
2022-04-25 18:25 ` [PATCH v3 00/12] add large CQE support for io-uring Stefan Roesch
2022-04-25 18:25   ` [PATCH v3 01/12] io_uring: support CQE32 in io_uring_cqe Stefan Roesch
2022-04-26  5:22     ` Kanchan Joshi
2022-04-25 18:25   ` [PATCH v3 02/12] io_uring: wire up inline completion path for CQE32 Stefan Roesch
2022-04-26  5:45     ` Kanchan Joshi
2022-04-25 18:25   ` [PATCH v3 03/12] io_uring: change ring size calculation " Stefan Roesch
2022-04-25 18:25   ` Stefan Roesch [this message]
2022-04-25 18:25   ` [PATCH v3 05/12] io_uring: add CQE32 completion processing Stefan Roesch
2022-04-25 18:25   ` [PATCH v3 06/12] io_uring: modify io_get_cqe for CQE32 Stefan Roesch
2022-04-25 18:25   ` [PATCH v3 07/12] io_uring: flush completions " Stefan Roesch
2022-04-25 18:25   ` [PATCH v3 08/12] io_uring: overflow processing " Stefan Roesch
2022-04-26  6:28     ` Kanchan Joshi
2022-04-26 12:53       ` Jens Axboe
2022-04-25 18:25   ` [PATCH v3 09/12] io_uring: add tracing for additional CQE32 fields Stefan Roesch
2022-04-25 18:25   ` [PATCH v3 10/12] io_uring: support CQE32 in /proc info Stefan Roesch
2022-04-25 18:25   ` [PATCH v3 11/12] io_uring: enable CQE32 Stefan Roesch
2022-04-25 18:25   ` [PATCH v3 12/12] io_uring: support CQE32 for nop operation Stefan Roesch
2022-04-25 18:38   ` [PATCH v3 00/12] add large CQE support for io-uring Jens Axboe
2022-04-26 11:37   ` Kanchan Joshi
2022-04-26 12:54     ` Jens Axboe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox