[PATCH] io_uring/net: Allow to do vectorized send

[PATCH] io_uring/net: Allow to do vectorized send

[norman.maurer]

From: Norman Maurer <norman_maurer@apple.com>

At the moment you have to use sendmsg for vectorized send. While this works it's suboptimal as it also means you need to allocate a struct msghdr that needs to be kept alive until a submission happens. We can remove this limitation by just allowing to use send directly.

Signed-off-by: Norman Maurer <norman_maurer@apple.com>
---
 include/uapi/linux/io_uring.h | 4 ++++
 io_uring/net.c                | 7 +++++++
 2 files changed, 11 insertions(+)

diff --git a/include/uapi/linux/io_uring.h b/include/uapi/linux/io_uring.h
index b8a0e70ee2fd..6957dc539d83 100644
--- a/include/uapi/linux/io_uring.h
+++ b/include/uapi/linux/io_uring.h
@@ -392,12 +392,16 @@ enum io_uring_op {
  *				the starting buffer ID in cqe->flags as per
  *				usual for provided buffer usage. The buffers
  *				will be	contiguous from the starting buffer ID.
+ *
+ * IORING_SEND_VECTORIZED	If set, SEND[_ZC] will take a pointer to a io_vec
+ * 				to allow vectorized send operations.
  */
 #define IORING_RECVSEND_POLL_FIRST	(1U << 0)
 #define IORING_RECV_MULTISHOT		(1U << 1)
 #define IORING_RECVSEND_FIXED_BUF	(1U << 2)
 #define IORING_SEND_ZC_REPORT_USAGE	(1U << 3)
 #define IORING_RECVSEND_BUNDLE		(1U << 4)
+#define IORING_SEND_VECTORIZED		(1U << 5)
 
 /*
  * cqe.res for IORING_CQE_F_NOTIF if
diff --git a/io_uring/net.c b/io_uring/net.c
index ba2d0abea349..d4a59f5461ed 100644
--- a/io_uring/net.c
+++ b/io_uring/net.c
@@ -382,6 +382,10 @@ static int io_send_setup(struct io_kiocb *req, const struct io_uring_sqe *sqe)
 	}
 	if (req->flags & REQ_F_BUFFER_SELECT)
 		return 0;
+
+	if (sr->flags & IORING_SEND_VECTORIZED)
+               return io_net_import_vec(req, kmsg, sr->buf, sr->len, ITER_SOURCE);
+
 	return import_ubuf(ITER_SOURCE, sr->buf, sr->len, &kmsg->msg.msg_iter);
 }
 
@@ -420,6 +424,9 @@ int io_sendmsg_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
 	sr->flags = READ_ONCE(sqe->ioprio);
 	if (sr->flags & ~SENDMSG_FLAGS)
 		return -EINVAL;
+        if (req->opcode != IORING_OP_SEND && req->opcode != IORING_OP_SEND_ZC && sr->flags & IORING_SEND_VECTORIZED)
+                return -EINVAL;
+
 	sr->msg_flags = READ_ONCE(sqe->msg_flags) | MSG_NOSIGNAL;
 	if (sr->msg_flags & MSG_DONTWAIT)
 		req->flags |= REQ_F_NOWAIT;
-- 
2.39.5 (Apple Git-154)

Re: [PATCH] io_uring/net: Allow to do vectorized send

[Jens Axboe]

On 7/23/25 11:16 PM, norman.maurer@googlemail.com wrote:
> From: Norman Maurer <norman_maurer@apple.com>
> 
> At the moment you have to use sendmsg for vectorized send. While this
> works it's suboptimal as it also means you need to allocate a struct
> msghdr that needs to be kept alive until a submission happens. We can
> remove this limitation by just allowing to use send directly.

Looks pretty clean, just a few minor comments below. For the commit
message above, you should wrap it at ~72 chars.

> diff --git a/include/uapi/linux/io_uring.h b/include/uapi/linux/io_uring.h
> index b8a0e70ee2fd..6957dc539d83 100644
> --- a/include/uapi/linux/io_uring.h
> +++ b/include/uapi/linux/io_uring.h
> @@ -392,12 +392,16 @@ enum io_uring_op {
>   *				the starting buffer ID in cqe->flags as per
>   *				usual for provided buffer usage. The buffers
>   *				will be	contiguous from the starting buffer ID.
> + *
> + * IORING_SEND_VECTORIZED	If set, SEND[_ZC] will take a pointer to a io_vec
> + * 				to allow vectorized send operations.
>   */
>  #define IORING_RECVSEND_POLL_FIRST	(1U << 0)
>  #define IORING_RECV_MULTISHOT		(1U << 1)
>  #define IORING_RECVSEND_FIXED_BUF	(1U << 2)
>  #define IORING_SEND_ZC_REPORT_USAGE	(1U << 3)
>  #define IORING_RECVSEND_BUNDLE		(1U << 4)
> +#define IORING_SEND_VECTORIZED		(1U << 5)

Do we want to support this on the recv side too? I guess that can be
added later and IORING_RECV_VECTORIZED can just be defined to
IORING_SEND_VECTORIZED in that case.

> diff --git a/io_uring/net.c b/io_uring/net.c
> index ba2d0abea349..d4a59f5461ed 100644
> --- a/io_uring/net.c
> +++ b/io_uring/net.c
> @@ -420,6 +424,9 @@ int io_sendmsg_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
>  	sr->flags = READ_ONCE(sqe->ioprio);
>  	if (sr->flags & ~SENDMSG_FLAGS)
>  		return -EINVAL;
> +        if (req->opcode != IORING_OP_SEND && req->opcode != IORING_OP_SEND_ZC && sr->flags & IORING_SEND_VECTORIZED)
> +                return -EINVAL;
> +

	if (req->opcode == IORING_OP_SENDMSG && sr->flags & IORING_SEND_VECTORIZED)
		return -EINVAL;

?

-- 
Jens Axboe

Re: [PATCH] io_uring/net: Allow to do vectorized send

[Norman Maurer]

> On Jul 24, 2025, at 4:49 AM, Jens Axboe <axboe@kernel.dk> wrote:
> 
> On 7/23/25 11:16 PM, norman.maurer@googlemail.com wrote:
>> From: Norman Maurer <norman_maurer@apple.com>
>> 
>> At the moment you have to use sendmsg for vectorized send. While this
>> works it's suboptimal as it also means you need to allocate a struct
>> msghdr that needs to be kept alive until a submission happens. We can
>> remove this limitation by just allowing to use send directly.
> 
> Looks pretty clean, just a few minor comments below. For the commit
> message above, you should wrap it at ~72 chars.

Will do in a v2…

> 
>> diff --git a/include/uapi/linux/io_uring.h b/include/uapi/linux/io_uring.h
>> index b8a0e70ee2fd..6957dc539d83 100644
>> --- a/include/uapi/linux/io_uring.h
>> +++ b/include/uapi/linux/io_uring.h
>> @@ -392,12 +392,16 @@ enum io_uring_op {
>>  * the starting buffer ID in cqe->flags as per
>>  * usual for provided buffer usage. The buffers
>>  * will be contiguous from the starting buffer ID.
>> + *
>> + * IORING_SEND_VECTORIZED If set, SEND[_ZC] will take a pointer to a io_vec
>> + * to allow vectorized send operations.
>>  */
>> #define IORING_RECVSEND_POLL_FIRST (1U << 0)
>> #define IORING_RECV_MULTISHOT (1U << 1)
>> #define IORING_RECVSEND_FIXED_BUF (1U << 2)
>> #define IORING_SEND_ZC_REPORT_USAGE (1U << 3)
>> #define IORING_RECVSEND_BUNDLE (1U << 4)
>> +#define IORING_SEND_VECTORIZED (1U << 5)
> 
> Do we want to support this on the recv side too? I guess that can be
> added later and IORING_RECV_VECTORIZED can just be defined to
> IORING_SEND_VECTORIZED in that case.

I think we can do this a follow if we want too..

> 
>> diff --git a/io_uring/net.c b/io_uring/net.c
>> index ba2d0abea349..d4a59f5461ed 100644
>> --- a/io_uring/net.c
>> +++ b/io_uring/net.c
>> @@ -420,6 +424,9 @@ int io_sendmsg_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
>> sr->flags = READ_ONCE(sqe->ioprio);
>> if (sr->flags & ~SENDMSG_FLAGS)
>> return -EINVAL;
>> +        if (req->opcode != IORING_OP_SEND && req->opcode != IORING_OP_SEND_ZC && sr->flags & IORING_SEND_VECTORIZED)
>> +                return -EINVAL;
>> +
> 
> if (req->opcode == IORING_OP_SENDMSG && sr->flags & IORING_SEND_VECTORIZED)
> return -EINVAL;
> 
> ?

Sure works for me… Just wanted to make it more explicit but I can change it as part of a v2.

> 
> -- 
> Jens Axboe