public inbox for [email protected]
 help / color / mirror / Atom feed
From: Pavel Begunkov <[email protected]>
To: [email protected], [email protected],
	[email protected]
Cc: "David S . Miller" <[email protected]>,
	Jakub Kicinski <[email protected]>,
	Jonathan Lemon <[email protected]>,
	Willem de Bruijn <[email protected]>,
	Jens Axboe <[email protected]>, David Ahern <[email protected]>,
	[email protected], Pavel Begunkov <[email protected]>
Subject: [PATCH net-next v5 22/27] io_uring: sendzc with fixed buffers
Date: Tue, 12 Jul 2022 21:52:46 +0100	[thread overview]
Message-ID: <e1d8bd1b5934e541d90c1824eb4020ae3f5f43f3.1657643355.git.asml.silence@gmail.com> (raw)
In-Reply-To: <[email protected]>

Allow zerocopy sends to use fixed buffers. There is an optimisation for
this case, the network layer don't need to reference the pages, see
SKBFL_MANAGED_FRAG_REFS, so io_uring have to ensure validity of fixed
buffers until the notifier is released.

Signed-off-by: Pavel Begunkov <[email protected]>
---
 include/uapi/linux/io_uring.h |  6 +++++-
 io_uring/net.c                | 29 ++++++++++++++++++++++++-----
 2 files changed, 29 insertions(+), 6 deletions(-)

diff --git a/include/uapi/linux/io_uring.h b/include/uapi/linux/io_uring.h
index 9303bf5236f7..3f2305bc5c79 100644
--- a/include/uapi/linux/io_uring.h
+++ b/include/uapi/linux/io_uring.h
@@ -269,9 +269,13 @@ enum io_uring_op {
  * IORING_RECV_MULTISHOT	Multishot recv. Sets IORING_CQE_F_MORE if
  *				the handler will continue to report
  *				CQEs on behalf of the same SQE.
+ *
+ * IORING_RECVSEND_FIXED_BUF	Use registered buffers, the index is stored in
+ *				the buf_index field.
  */
 #define IORING_RECVSEND_POLL_FIRST	(1U << 0)
-#define IORING_RECV_MULTISHOT	(1U << 1)
+#define IORING_RECV_MULTISHOT		(1U << 1)
+#define IORING_RECVSEND_FIXED_BUF	(1U << 2)
 
 /*
  * accept flags stored in sqe->ioprio
diff --git a/io_uring/net.c b/io_uring/net.c
index 2172cf3facd8..0259fbbad591 100644
--- a/io_uring/net.c
+++ b/io_uring/net.c
@@ -14,6 +14,7 @@
 #include "kbuf.h"
 #include "net.h"
 #include "notif.h"
+#include "rsrc.h"
 
 #if defined(CONFIG_NET)
 struct io_shutdown {
@@ -667,13 +668,23 @@ int io_recv(struct io_kiocb *req, unsigned int issue_flags)
 int io_sendzc_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
 {
 	struct io_sendzc *zc = io_kiocb_to_cmd(req);
+	struct io_ring_ctx *ctx = req->ctx;
 
 	if (READ_ONCE(sqe->__pad2[0]) || READ_ONCE(sqe->addr3))
 		return -EINVAL;
 
 	zc->flags = READ_ONCE(sqe->ioprio);
-	if (zc->flags & ~IORING_RECVSEND_POLL_FIRST)
+	if (zc->flags & ~(IORING_RECVSEND_POLL_FIRST | IORING_RECVSEND_FIXED_BUF))
 		return -EINVAL;
+	if (zc->flags & IORING_RECVSEND_FIXED_BUF) {
+		unsigned idx = READ_ONCE(sqe->buf_index);
+
+		if (unlikely(idx >= ctx->nr_user_bufs))
+			return -EFAULT;
+		idx = array_index_nospec(idx, ctx->nr_user_bufs);
+		req->imu = READ_ONCE(ctx->user_bufs[idx]);
+		io_req_set_rsrc_node(req, ctx, 0);
+	}
 
 	zc->buf = u64_to_user_ptr(READ_ONCE(sqe->addr));
 	zc->len = READ_ONCE(sqe->len);
@@ -727,10 +738,18 @@ int io_sendzc(struct io_kiocb *req, unsigned int issue_flags)
 	msg.msg_controllen = 0;
 	msg.msg_namelen = 0;
 
-	ret = import_single_range(WRITE, zc->buf, zc->len, &iov, &msg.msg_iter);
-	if (unlikely(ret))
-		return ret;
-	mm_account_pinned_pages(&notif->uarg.mmp, zc->len);
+	if (zc->flags & IORING_RECVSEND_FIXED_BUF) {
+		ret = io_import_fixed(WRITE, &msg.msg_iter, req->imu,
+					(u64)zc->buf, zc->len);
+		if (unlikely(ret))
+				return ret;
+	} else {
+		ret = import_single_range(WRITE, zc->buf, zc->len, &iov,
+					  &msg.msg_iter);
+		if (unlikely(ret))
+			return ret;
+		mm_account_pinned_pages(&notif->uarg.mmp, zc->len);
+	}
 
 	if (zc->addr) {
 		ret = move_addr_to_kernel(zc->addr, zc->addr_len, &address);
-- 
2.37.0


  parent reply	other threads:[~2022-07-12 20:55 UTC|newest]

Thread overview: 34+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-07-12 20:52 [PATCH net-next v5 00/27] io_uring zerocopy send Pavel Begunkov
2022-07-12 20:52 ` [PATCH net-next v5 01/27] ipv4: avoid partial copy for zc Pavel Begunkov
2022-07-19  1:54   ` Jakub Kicinski
2022-07-19  9:35     ` Willem de Bruijn
2022-07-21 10:03       ` Pavel Begunkov
2022-07-12 20:52 ` [PATCH net-next v5 02/27] ipv6: " Pavel Begunkov
2022-07-12 20:52 ` [PATCH net-next v5 03/27] skbuff: don't mix ubuf_info from different sources Pavel Begunkov
2022-07-12 20:52 ` [PATCH net-next v5 04/27] skbuff: add SKBFL_DONT_ORPHAN flag Pavel Begunkov
2022-07-12 20:52 ` [PATCH net-next v5 05/27] skbuff: carry external ubuf_info in msghdr Pavel Begunkov
2022-07-12 20:52 ` [PATCH net-next v5 06/27] net: Allow custom iter handler " Pavel Begunkov
2022-07-12 20:52 ` [PATCH net-next v5 07/27] net: introduce managed frags infrastructure Pavel Begunkov
2022-07-12 20:52 ` [PATCH net-next v5 08/27] net: introduce __skb_fill_page_desc_noacc Pavel Begunkov
2022-07-12 20:52 ` [PATCH net-next v5 09/27] ipv4/udp: support externally provided ubufs Pavel Begunkov
2022-07-12 20:52 ` [PATCH net-next v5 10/27] ipv6/udp: " Pavel Begunkov
2022-07-12 20:52 ` [PATCH net-next v5 11/27] tcp: " Pavel Begunkov
2022-07-12 20:52 ` [PATCH net-next v5 12/27] io_uring: initialise msghdr::msg_ubuf Pavel Begunkov
2022-07-12 20:52 ` [PATCH net-next v5 13/27] io_uring: export io_put_task() Pavel Begunkov
2022-07-12 20:52 ` [PATCH net-next v5 14/27] io_uring: add zc notification infrastructure Pavel Begunkov
2022-07-12 20:52 ` [PATCH net-next v5 15/27] io_uring: cache struct io_notif Pavel Begunkov
2022-07-12 20:52 ` [PATCH net-next v5 16/27] io_uring: complete notifiers in tw Pavel Begunkov
2022-07-12 20:52 ` [PATCH net-next v5 17/27] io_uring: add rsrc referencing for notifiers Pavel Begunkov
2022-07-12 20:52 ` [PATCH net-next v5 18/27] io_uring: add notification slot registration Pavel Begunkov
2022-07-12 20:52 ` [PATCH net-next v5 19/27] io_uring: wire send zc request type Pavel Begunkov
2022-07-12 20:52 ` [PATCH net-next v5 20/27] io_uring: account locked pages for non-fixed zc Pavel Begunkov
2022-07-12 20:52 ` [PATCH net-next v5 21/27] io_uring: allow to pass addr into sendzc Pavel Begunkov
2022-07-12 20:52 ` Pavel Begunkov [this message]
2022-07-12 20:52 ` [PATCH net-next v5 23/27] io_uring: flush notifiers after sendzc Pavel Begunkov
2022-07-12 20:52 ` [PATCH net-next v5 24/27] io_uring: rename IORING_OP_FILES_UPDATE Pavel Begunkov
2022-07-12 20:52 ` [PATCH net-next v5 25/27] io_uring: add zc notification flush requests Pavel Begunkov
2022-07-12 20:52 ` [PATCH net-next v5 26/27] io_uring: enable managed frags with register buffers Pavel Begunkov
2022-07-12 20:52 ` [PATCH net-next v5 27/27] selftests/io_uring: test zerocopy send Pavel Begunkov
2022-07-27  8:01   ` dust.li
2022-07-27  9:18     ` Pavel Begunkov
2022-07-20 12:46 ` (subset) [PATCH net-next v5 00/27] io_uring " Jens Axboe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=e1d8bd1b5934e541d90c1824eb4020ae3f5f43f3.1657643355.git.asml.silence@gmail.com \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox