public inbox for [email protected]
 help / color / mirror / Atom feed
* [PATCH for-next 0/3] sendmsg/recvmsg cleanup
@ 2021-02-05  0:57 Pavel Begunkov
  2021-02-05  0:57 ` [PATCH 1/3] io_uring: set msg_name on msg fixup Pavel Begunkov
                   ` (3 more replies)
  0 siblings, 4 replies; 11+ messages in thread
From: Pavel Begunkov @ 2021-02-05  0:57 UTC (permalink / raw)
  To: Jens Axboe, io-uring

Reincarnated cleanups of sendmsg/recvmsg managing and copying of iov.

Pavel Begunkov (3):
  io_uring: set msg_name on msg fixup
  io_uring: clean iov usage for recvmsg buf select
  io_uring: refactor sendmsg/recvmsg iov managing

 fs/io_uring.c | 68 +++++++++++++++++++++++----------------------------
 1 file changed, 30 insertions(+), 38 deletions(-)

-- 
2.24.0


^ permalink raw reply	[flat|nested] 11+ messages in thread

* [PATCH 1/3] io_uring: set msg_name on msg fixup
  2021-02-05  0:57 [PATCH for-next 0/3] sendmsg/recvmsg cleanup Pavel Begunkov
@ 2021-02-05  0:57 ` Pavel Begunkov
  2021-02-05  0:57 ` [PATCH 2/3] io_uring: clean iov usage for recvmsg buf select Pavel Begunkov
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 11+ messages in thread
From: Pavel Begunkov @ 2021-02-05  0:57 UTC (permalink / raw)
  To: Jens Axboe, io-uring

io_setup_async_msg() should fully prepare io_async_msghdr, let it also
handle assigning msg_name and don't hand code it in [send,recv]msg().

Signed-off-by: Pavel Begunkov <[email protected]>
---
 fs/io_uring.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/fs/io_uring.c b/fs/io_uring.c
index b740a39110d6..39bc1df9bb64 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -4558,6 +4558,7 @@ static int io_setup_async_msg(struct io_kiocb *req,
 	async_msg = req->async_data;
 	req->flags |= REQ_F_NEED_CLEANUP;
 	memcpy(async_msg, kmsg, sizeof(*kmsg));
+	async_msg->msg.msg_name = &async_msg->addr;
 	return -EAGAIN;
 }
 
@@ -4610,7 +4611,6 @@ static int io_sendmsg(struct io_kiocb *req, bool force_nonblock,
 
 	if (req->async_data) {
 		kmsg = req->async_data;
-		kmsg->msg.msg_name = &kmsg->addr;
 		/* if iov is set, it's allocated already */
 		if (!kmsg->iov)
 			kmsg->iov = kmsg->fast_iov;
@@ -4839,7 +4839,6 @@ static int io_recvmsg(struct io_kiocb *req, bool force_nonblock,
 
 	if (req->async_data) {
 		kmsg = req->async_data;
-		kmsg->msg.msg_name = &kmsg->addr;
 		/* if iov is set, it's allocated already */
 		if (!kmsg->iov)
 			kmsg->iov = kmsg->fast_iov;
-- 
2.24.0


^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 2/3] io_uring: clean iov usage for recvmsg buf select
  2021-02-05  0:57 [PATCH for-next 0/3] sendmsg/recvmsg cleanup Pavel Begunkov
  2021-02-05  0:57 ` [PATCH 1/3] io_uring: set msg_name on msg fixup Pavel Begunkov
@ 2021-02-05  0:57 ` Pavel Begunkov
  2021-02-05  0:58 ` [PATCH 3/3] io_uring: refactor sendmsg/recvmsg iov managing Pavel Begunkov
  2021-02-05 14:46 ` [PATCH for-next 0/3] sendmsg/recvmsg cleanup Jens Axboe
  3 siblings, 0 replies; 11+ messages in thread
From: Pavel Begunkov @ 2021-02-05  0:57 UTC (permalink / raw)
  To: Jens Axboe, io-uring

Don't pretend we don't know that REQ_F_BUFFER_SELECT for recvmsg always
uses fast_iov -- clean up confusing intermixing kmsg->iov and
kmsg->fast_iov for buffer select.

Also don't init iter with garbage in __io_recvmsg_copy_hdr() only for it
to be set shortly after in io_recvmsg().

Signed-off-by: Pavel Begunkov <[email protected]>
---
 fs/io_uring.c | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

diff --git a/fs/io_uring.c b/fs/io_uring.c
index 39bc1df9bb64..e07a7fa15cfa 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -4701,11 +4701,9 @@ static int __io_recvmsg_copy_hdr(struct io_kiocb *req,
 	if (req->flags & REQ_F_BUFFER_SELECT) {
 		if (iov_len > 1)
 			return -EINVAL;
-		if (copy_from_user(iomsg->iov, uiov, sizeof(*uiov)))
+		if (copy_from_user(iomsg->fast_iov, uiov, sizeof(*uiov)))
 			return -EFAULT;
-		sr->len = iomsg->iov[0].iov_len;
-		iov_iter_init(&iomsg->msg.msg_iter, READ, iomsg->iov, 1,
-				sr->len);
+		sr->len = iomsg->fast_iov[0].iov_len;
 		iomsg->iov = NULL;
 	} else {
 		ret = __import_iovec(READ, uiov, iov_len, UIO_FASTIOV,
@@ -4748,7 +4746,6 @@ static int __io_compat_recvmsg_copy_hdr(struct io_kiocb *req,
 		if (clen < 0)
 			return -EINVAL;
 		sr->len = clen;
-		iomsg->iov[0].iov_len = clen;
 		iomsg->iov = NULL;
 	} else {
 		ret = __import_iovec(READ, (struct iovec __user *)uiov, len,
@@ -4855,7 +4852,8 @@ static int io_recvmsg(struct io_kiocb *req, bool force_nonblock,
 		if (IS_ERR(kbuf))
 			return PTR_ERR(kbuf);
 		kmsg->fast_iov[0].iov_base = u64_to_user_ptr(kbuf->addr);
-		iov_iter_init(&kmsg->msg.msg_iter, READ, kmsg->iov,
+		kmsg->fast_iov[0].iov_len = req->sr_msg.len;
+		iov_iter_init(&kmsg->msg.msg_iter, READ, kmsg->fast_iov,
 				1, req->sr_msg.len);
 	}
 
-- 
2.24.0


^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 3/3] io_uring: refactor sendmsg/recvmsg iov managing
  2021-02-05  0:57 [PATCH for-next 0/3] sendmsg/recvmsg cleanup Pavel Begunkov
  2021-02-05  0:57 ` [PATCH 1/3] io_uring: set msg_name on msg fixup Pavel Begunkov
  2021-02-05  0:57 ` [PATCH 2/3] io_uring: clean iov usage for recvmsg buf select Pavel Begunkov
@ 2021-02-05  0:58 ` Pavel Begunkov
  2021-02-05  7:17   ` Stefan Metzmacher
  2021-02-05 14:46 ` [PATCH for-next 0/3] sendmsg/recvmsg cleanup Jens Axboe
  3 siblings, 1 reply; 11+ messages in thread
From: Pavel Begunkov @ 2021-02-05  0:58 UTC (permalink / raw)
  To: Jens Axboe, io-uring

Current iov handling with recvmsg/sendmsg may be confusing. First make a
rule for msg->iov: either it points to an allocated iov that have to be
kfree()'d later, or it's NULL and we use fast_iov. That's much better
than current 3-state (also can point to fast_iov). And rename it into
free_iov for uniformity with read/write.

Also, instead of after struct io_async_msghdr copy fixing up of
msg.msg_iter.iov has been happening in io_recvmsg()/io_sendmsg(). Move
it into io_setup_async_msg(), that's the right place.

Signed-off-by: Pavel Begunkov <[email protected]>
---
 fs/io_uring.c | 55 +++++++++++++++++++++++----------------------------
 1 file changed, 25 insertions(+), 30 deletions(-)

diff --git a/fs/io_uring.c b/fs/io_uring.c
index e07a7fa15cfa..7008e0c7e05d 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -594,7 +594,8 @@ struct io_async_connect {
 
 struct io_async_msghdr {
 	struct iovec			fast_iov[UIO_FASTIOV];
-	struct iovec			*iov;
+	/* points to an allocated iov, if NULL we use fast_iov instead */
+	struct iovec			*free_iov;
 	struct sockaddr __user		*uaddr;
 	struct msghdr			msg;
 	struct sockaddr_storage		addr;
@@ -4551,24 +4552,27 @@ static int io_setup_async_msg(struct io_kiocb *req,
 	if (async_msg)
 		return -EAGAIN;
 	if (io_alloc_async_data(req)) {
-		if (kmsg->iov != kmsg->fast_iov)
-			kfree(kmsg->iov);
+		kfree(kmsg->free_iov);
 		return -ENOMEM;
 	}
 	async_msg = req->async_data;
 	req->flags |= REQ_F_NEED_CLEANUP;
 	memcpy(async_msg, kmsg, sizeof(*kmsg));
 	async_msg->msg.msg_name = &async_msg->addr;
+	/* if were using fast_iov, set it to the new one */
+	if (!async_msg->free_iov)
+		async_msg->msg.msg_iter.iov = async_msg->fast_iov;
+
 	return -EAGAIN;
 }
 
 static int io_sendmsg_copy_hdr(struct io_kiocb *req,
 			       struct io_async_msghdr *iomsg)
 {
-	iomsg->iov = iomsg->fast_iov;
 	iomsg->msg.msg_name = &iomsg->addr;
+	iomsg->free_iov = iomsg->fast_iov;
 	return sendmsg_copy_msghdr(&iomsg->msg, req->sr_msg.umsg,
-				   req->sr_msg.msg_flags, &iomsg->iov);
+				   req->sr_msg.msg_flags, &iomsg->free_iov);
 }
 
 static int io_sendmsg_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
@@ -4609,13 +4613,8 @@ static int io_sendmsg(struct io_kiocb *req, bool force_nonblock,
 	if (unlikely(!sock))
 		return -ENOTSOCK;
 
-	if (req->async_data) {
-		kmsg = req->async_data;
-		/* if iov is set, it's allocated already */
-		if (!kmsg->iov)
-			kmsg->iov = kmsg->fast_iov;
-		kmsg->msg.msg_iter.iov = kmsg->iov;
-	} else {
+	kmsg = req->async_data;
+	if (!kmsg) {
 		ret = io_sendmsg_copy_hdr(req, &iomsg);
 		if (ret)
 			return ret;
@@ -4634,8 +4633,8 @@ static int io_sendmsg(struct io_kiocb *req, bool force_nonblock,
 	if (ret == -ERESTARTSYS)
 		ret = -EINTR;
 
-	if (kmsg->iov != kmsg->fast_iov)
-		kfree(kmsg->iov);
+	if (kmsg->free_iov)
+		kfree(kmsg->free_iov);
 	req->flags &= ~REQ_F_NEED_CLEANUP;
 	if (ret < 0)
 		req_set_fail_links(req);
@@ -4704,10 +4703,11 @@ static int __io_recvmsg_copy_hdr(struct io_kiocb *req,
 		if (copy_from_user(iomsg->fast_iov, uiov, sizeof(*uiov)))
 			return -EFAULT;
 		sr->len = iomsg->fast_iov[0].iov_len;
-		iomsg->iov = NULL;
+		iomsg->free_iov = NULL;
 	} else {
+		iomsg->free_iov = iomsg->fast_iov;
 		ret = __import_iovec(READ, uiov, iov_len, UIO_FASTIOV,
-				     &iomsg->iov, &iomsg->msg.msg_iter,
+				     &iomsg->free_iov, &iomsg->msg.msg_iter,
 				     false);
 		if (ret > 0)
 			ret = 0;
@@ -4746,10 +4746,11 @@ static int __io_compat_recvmsg_copy_hdr(struct io_kiocb *req,
 		if (clen < 0)
 			return -EINVAL;
 		sr->len = clen;
-		iomsg->iov = NULL;
+		iomsg->free_iov = NULL;
 	} else {
+		iomsg->free_iov = iomsg->fast_iov;
 		ret = __import_iovec(READ, (struct iovec __user *)uiov, len,
-				   UIO_FASTIOV, &iomsg->iov,
+				   UIO_FASTIOV, &iomsg->free_iov,
 				   &iomsg->msg.msg_iter, true);
 		if (ret < 0)
 			return ret;
@@ -4763,7 +4764,6 @@ static int io_recvmsg_copy_hdr(struct io_kiocb *req,
 			       struct io_async_msghdr *iomsg)
 {
 	iomsg->msg.msg_name = &iomsg->addr;
-	iomsg->iov = iomsg->fast_iov;
 
 #ifdef CONFIG_COMPAT
 	if (req->ctx->compat)
@@ -4834,13 +4834,8 @@ static int io_recvmsg(struct io_kiocb *req, bool force_nonblock,
 	if (unlikely(!sock))
 		return -ENOTSOCK;
 
-	if (req->async_data) {
-		kmsg = req->async_data;
-		/* if iov is set, it's allocated already */
-		if (!kmsg->iov)
-			kmsg->iov = kmsg->fast_iov;
-		kmsg->msg.msg_iter.iov = kmsg->iov;
-	} else {
+	kmsg = req->async_data;
+	if (!kmsg) {
 		ret = io_recvmsg_copy_hdr(req, &iomsg);
 		if (ret)
 			return ret;
@@ -4872,8 +4867,8 @@ static int io_recvmsg(struct io_kiocb *req, bool force_nonblock,
 
 	if (req->flags & REQ_F_BUFFER_SELECTED)
 		cflags = io_put_recv_kbuf(req);
-	if (kmsg->iov != kmsg->fast_iov)
-		kfree(kmsg->iov);
+	if (kmsg->free_iov)
+		kfree(kmsg->free_iov);
 	req->flags &= ~REQ_F_NEED_CLEANUP;
 	if (ret < 0)
 		req_set_fail_links(req);
@@ -6166,8 +6161,8 @@ static void __io_clean_op(struct io_kiocb *req)
 		case IORING_OP_RECVMSG:
 		case IORING_OP_SENDMSG: {
 			struct io_async_msghdr *io = req->async_data;
-			if (io->iov != io->fast_iov)
-				kfree(io->iov);
+
+			kfree(io->free_iov);
 			break;
 			}
 		case IORING_OP_SPLICE:
-- 
2.24.0


^ permalink raw reply related	[flat|nested] 11+ messages in thread

* Re: [PATCH 3/3] io_uring: refactor sendmsg/recvmsg iov managing
  2021-02-05  0:58 ` [PATCH 3/3] io_uring: refactor sendmsg/recvmsg iov managing Pavel Begunkov
@ 2021-02-05  7:17   ` Stefan Metzmacher
  2021-02-05  9:48     ` Pavel Begunkov
  0 siblings, 1 reply; 11+ messages in thread
From: Stefan Metzmacher @ 2021-02-05  7:17 UTC (permalink / raw)
  To: Pavel Begunkov, Jens Axboe, io-uring


[-- Attachment #1.1: Type: text/plain, Size: 1781 bytes --]

Hi Pavel,

>  static int io_sendmsg_copy_hdr(struct io_kiocb *req,
>  			       struct io_async_msghdr *iomsg)
>  {
> -	iomsg->iov = iomsg->fast_iov;
>  	iomsg->msg.msg_name = &iomsg->addr;
> +	iomsg->free_iov = iomsg->fast_iov;

Why this? Isn't the idea of this patch that free_iov is never == fast_iov?


> @@ -4704,10 +4703,11 @@ static int __io_recvmsg_copy_hdr(struct io_kiocb *req,
>  		if (copy_from_user(iomsg->fast_iov, uiov, sizeof(*uiov)))
>  			return -EFAULT;
>  		sr->len = iomsg->fast_iov[0].iov_len;
> -		iomsg->iov = NULL;
> +		iomsg->free_iov = NULL;
>  	} else {
> +		iomsg->free_iov = iomsg->fast_iov;

The same here...

>  		ret = __import_iovec(READ, uiov, iov_len, UIO_FASTIOV,
> -				     &iomsg->iov, &iomsg->msg.msg_iter,
> +				     &iomsg->free_iov, &iomsg->msg.msg_iter,
>  				     false);
>  		if (ret > 0)
>  			ret = 0;
> @@ -4746,10 +4746,11 @@ static int __io_compat_recvmsg_copy_hdr(struct io_kiocb *req,
>  		if (clen < 0)
>  			return -EINVAL;
>  		sr->len = clen;
> -		iomsg->iov = NULL;
> +		iomsg->free_iov = NULL;
>  	} else {
> +		iomsg->free_iov = iomsg->fast_iov;

And here...

>  		ret = __import_iovec(READ, (struct iovec __user *)uiov, len,
> -				   UIO_FASTIOV, &iomsg->iov,
> +				   UIO_FASTIOV, &iomsg->free_iov,
>  				   &iomsg->msg.msg_iter, true);
>  		if (ret < 0)
>  			return ret;

> @@ -4872,8 +4867,8 @@ static int io_recvmsg(struct io_kiocb *req, bool force_nonblock,
>  
>  	if (req->flags & REQ_F_BUFFER_SELECTED)
>  		cflags = io_put_recv_kbuf(req);
> -	if (kmsg->iov != kmsg->fast_iov)
> -		kfree(kmsg->iov);
> +	if (kmsg->free_iov)
> +		kfree(kmsg->free_iov);

kfree() handles NULL, or is this a hot path and we want to avoid a function call?

metze



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH 3/3] io_uring: refactor sendmsg/recvmsg iov managing
  2021-02-05  7:17   ` Stefan Metzmacher
@ 2021-02-05  9:48     ` Pavel Begunkov
  2021-02-05  9:58       ` Stefan Metzmacher
  0 siblings, 1 reply; 11+ messages in thread
From: Pavel Begunkov @ 2021-02-05  9:48 UTC (permalink / raw)
  To: Stefan Metzmacher, Jens Axboe, io-uring

On 05/02/2021 07:17, Stefan Metzmacher wrote:
> Hi Pavel,
> 
>>  static int io_sendmsg_copy_hdr(struct io_kiocb *req,
>>  			       struct io_async_msghdr *iomsg)
>>  {
>> -	iomsg->iov = iomsg->fast_iov;
>>  	iomsg->msg.msg_name = &iomsg->addr;
>> +	iomsg->free_iov = iomsg->fast_iov;
> 
> Why this? Isn't the idea of this patch that free_iov is never == fast_iov?

That's a part of __import_iovec() and sendmsg_copy_msghdr() API, you pass
fast_iov as such and get back NULL or a newly allocated one in it.

> 
> 
>> @@ -4704,10 +4703,11 @@ static int __io_recvmsg_copy_hdr(struct io_kiocb *req,
>>  		if (copy_from_user(iomsg->fast_iov, uiov, sizeof(*uiov)))
>>  			return -EFAULT;
>>  		sr->len = iomsg->fast_iov[0].iov_len;
>> -		iomsg->iov = NULL;
>> +		iomsg->free_iov = NULL;
>>  	} else {
>> +		iomsg->free_iov = iomsg->fast_iov;
> 
> The same here...
> 
>>  		ret = __import_iovec(READ, uiov, iov_len, UIO_FASTIOV,
>> -				     &iomsg->iov, &iomsg->msg.msg_iter,
>> +				     &iomsg->free_iov, &iomsg->msg.msg_iter,
>>  				     false);
>>  		if (ret > 0)
>>  			ret = 0;
>> @@ -4746,10 +4746,11 @@ static int __io_compat_recvmsg_copy_hdr(struct io_kiocb *req,
>>  		if (clen < 0)
>>  			return -EINVAL;
>>  		sr->len = clen;
>> -		iomsg->iov = NULL;
>> +		iomsg->free_iov = NULL;
>>  	} else {
>> +		iomsg->free_iov = iomsg->fast_iov;
> 
> And here...
> 
>>  		ret = __import_iovec(READ, (struct iovec __user *)uiov, len,
>> -				   UIO_FASTIOV, &iomsg->iov,
>> +				   UIO_FASTIOV, &iomsg->free_iov,
>>  				   &iomsg->msg.msg_iter, true);
>>  		if (ret < 0)
>>  			return ret;
> 
>> @@ -4872,8 +4867,8 @@ static int io_recvmsg(struct io_kiocb *req, bool force_nonblock,
>>  
>>  	if (req->flags & REQ_F_BUFFER_SELECTED)
>>  		cflags = io_put_recv_kbuf(req);
>> -	if (kmsg->iov != kmsg->fast_iov)
>> -		kfree(kmsg->iov);
>> +	if (kmsg->free_iov)
>> +		kfree(kmsg->free_iov);
> 
> kfree() handles NULL, or is this a hot path and we want to avoid a function call?

Yes, the hot path here is not having iov allocated, and Jens told before
that he had observed overhead for a similar place in io_[read,write].

-- 
Pavel Begunkov

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH 3/3] io_uring: refactor sendmsg/recvmsg iov managing
  2021-02-05  9:48     ` Pavel Begunkov
@ 2021-02-05  9:58       ` Stefan Metzmacher
  2021-02-05 10:06         ` Pavel Begunkov
  0 siblings, 1 reply; 11+ messages in thread
From: Stefan Metzmacher @ 2021-02-05  9:58 UTC (permalink / raw)
  To: Pavel Begunkov, Jens Axboe, io-uring


[-- Attachment #1.1: Type: text/plain, Size: 1293 bytes --]

Hi Pavel,

>>>  static int io_sendmsg_copy_hdr(struct io_kiocb *req,
>>>  			       struct io_async_msghdr *iomsg)
>>>  {
>>> -	iomsg->iov = iomsg->fast_iov;
>>>  	iomsg->msg.msg_name = &iomsg->addr;
>>> +	iomsg->free_iov = iomsg->fast_iov;
>>
>> Why this? Isn't the idea of this patch that free_iov is never == fast_iov?
> 
> That's a part of __import_iovec() and sendmsg_copy_msghdr() API, you pass
> fast_iov as such and get back NULL or a newly allocated one in it.
I think that should at least get a comment to make this clear and
maybe a temporary variable like this:

tmp_iov = iomsg->fast_iov;
__import_iovec(..., &tmp_iov, ...);
iomsg->free_iov = tmp_iov;

>>> @@ -4872,8 +4867,8 @@ static int io_recvmsg(struct io_kiocb *req, bool force_nonblock,
>>>  
>>>  	if (req->flags & REQ_F_BUFFER_SELECTED)
>>>  		cflags = io_put_recv_kbuf(req);
>>> -	if (kmsg->iov != kmsg->fast_iov)
>>> -		kfree(kmsg->iov);
>>> +	if (kmsg->free_iov)
>>> +		kfree(kmsg->free_iov);
>>
>> kfree() handles NULL, or is this a hot path and we want to avoid a function call?
> 
> Yes, the hot path here is not having iov allocated, and Jens told before
> that he had observed overhead for a similar place in io_[read,write].

Ok, a comment would also help here...

metze



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH 3/3] io_uring: refactor sendmsg/recvmsg iov managing
  2021-02-05  9:58       ` Stefan Metzmacher
@ 2021-02-05 10:06         ` Pavel Begunkov
  2021-02-05 14:45           ` Jens Axboe
  0 siblings, 1 reply; 11+ messages in thread
From: Pavel Begunkov @ 2021-02-05 10:06 UTC (permalink / raw)
  To: Stefan Metzmacher, Jens Axboe, io-uring

On 05/02/2021 09:58, Stefan Metzmacher wrote:
> Hi Pavel,
> 
>>>>  static int io_sendmsg_copy_hdr(struct io_kiocb *req,
>>>>  			       struct io_async_msghdr *iomsg)
>>>>  {
>>>> -	iomsg->iov = iomsg->fast_iov;
>>>>  	iomsg->msg.msg_name = &iomsg->addr;
>>>> +	iomsg->free_iov = iomsg->fast_iov;
>>>
>>> Why this? Isn't the idea of this patch that free_iov is never == fast_iov?
>>
>> That's a part of __import_iovec() and sendmsg_copy_msghdr() API, you pass
>> fast_iov as such and get back NULL or a newly allocated one in it.
> I think that should at least get a comment to make this clear and
> maybe a temporary variable like this:
> 
> tmp_iov = iomsg->fast_iov;
> __import_iovec(..., &tmp_iov, ...);
> iomsg->free_iov = tmp_iov;

I'd rather disagree. It's a well known (ish) API, and I deliberately
placed such assignments right before import_iovec/etc.

We have been using the same pattern has been used for reads/writes and
io_import_iovec() for ages, but seems it haven't got its attention.

> 
>>>> @@ -4872,8 +4867,8 @@ static int io_recvmsg(struct io_kiocb *req, bool force_nonblock,
>>>>  
>>>>  	if (req->flags & REQ_F_BUFFER_SELECTED)
>>>>  		cflags = io_put_recv_kbuf(req);
>>>> -	if (kmsg->iov != kmsg->fast_iov)
>>>> -		kfree(kmsg->iov);
>>>> +	if (kmsg->free_iov)
>>>> +		kfree(kmsg->free_iov);
>>>
>>> kfree() handles NULL, or is this a hot path and we want to avoid a function call?
>>
>> Yes, the hot path here is not having iov allocated, and Jens told before
>> that he had observed overhead for a similar place in io_[read,write].
> 
> Ok, a comment would also help here...
> 
> metze
> 
> 

-- 
Pavel Begunkov

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH 3/3] io_uring: refactor sendmsg/recvmsg iov managing
  2021-02-05 10:06         ` Pavel Begunkov
@ 2021-02-05 14:45           ` Jens Axboe
  2021-02-05 14:57             ` Pavel Begunkov
  0 siblings, 1 reply; 11+ messages in thread
From: Jens Axboe @ 2021-02-05 14:45 UTC (permalink / raw)
  To: Pavel Begunkov, Stefan Metzmacher, io-uring

On 2/5/21 3:06 AM, Pavel Begunkov wrote:
> On 05/02/2021 09:58, Stefan Metzmacher wrote:
>> Hi Pavel,
>>
>>>>>  static int io_sendmsg_copy_hdr(struct io_kiocb *req,
>>>>>  			       struct io_async_msghdr *iomsg)
>>>>>  {
>>>>> -	iomsg->iov = iomsg->fast_iov;
>>>>>  	iomsg->msg.msg_name = &iomsg->addr;
>>>>> +	iomsg->free_iov = iomsg->fast_iov;
>>>>
>>>> Why this? Isn't the idea of this patch that free_iov is never == fast_iov?
>>>
>>> That's a part of __import_iovec() and sendmsg_copy_msghdr() API, you pass
>>> fast_iov as such and get back NULL or a newly allocated one in it.
>> I think that should at least get a comment to make this clear and
>> maybe a temporary variable like this:
>>
>> tmp_iov = iomsg->fast_iov;
>> __import_iovec(..., &tmp_iov, ...);
>> iomsg->free_iov = tmp_iov;
> 
> I'd rather disagree. It's a well known (ish) API, and I deliberately
> placed such assignments right before import_iovec/etc.

Agree on that, it's kind of a weird idiom, but it's used throughout the
kernel. However:

>>>> kfree() handles NULL, or is this a hot path and we want to avoid a function call?
>>>
>>> Yes, the hot path here is not having iov allocated, and Jens told before
>>> that he had observed overhead for a similar place in io_[read,write].
>>
>> Ok, a comment would also help here...

I do agree on that one, since otherwise we get patches for it as has
been proven by the few other spots... I'll add then when queueing this
up.

-- 
Jens Axboe


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH for-next 0/3] sendmsg/recvmsg cleanup
  2021-02-05  0:57 [PATCH for-next 0/3] sendmsg/recvmsg cleanup Pavel Begunkov
                   ` (2 preceding siblings ...)
  2021-02-05  0:58 ` [PATCH 3/3] io_uring: refactor sendmsg/recvmsg iov managing Pavel Begunkov
@ 2021-02-05 14:46 ` Jens Axboe
  3 siblings, 0 replies; 11+ messages in thread
From: Jens Axboe @ 2021-02-05 14:46 UTC (permalink / raw)
  To: Pavel Begunkov, io-uring

On 2/4/21 5:57 PM, Pavel Begunkov wrote:
> Reincarnated cleanups of sendmsg/recvmsg managing and copying of iov.
> 
> Pavel Begunkov (3):
>   io_uring: set msg_name on msg fixup
>   io_uring: clean iov usage for recvmsg buf select
>   io_uring: refactor sendmsg/recvmsg iov managing
> 
>  fs/io_uring.c | 68 +++++++++++++++++++++++----------------------------
>  1 file changed, 30 insertions(+), 38 deletions(-)

Applied, thanks.

-- 
Jens Axboe


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH 3/3] io_uring: refactor sendmsg/recvmsg iov managing
  2021-02-05 14:45           ` Jens Axboe
@ 2021-02-05 14:57             ` Pavel Begunkov
  0 siblings, 0 replies; 11+ messages in thread
From: Pavel Begunkov @ 2021-02-05 14:57 UTC (permalink / raw)
  To: Jens Axboe, Stefan Metzmacher, io-uring

On 05/02/2021 14:45, Jens Axboe wrote:
> On 2/5/21 3:06 AM, Pavel Begunkov wrote:
>> On 05/02/2021 09:58, Stefan Metzmacher wrote:
>>> Hi Pavel,
>>>
>>>>>>  static int io_sendmsg_copy_hdr(struct io_kiocb *req,
>>>>>>  			       struct io_async_msghdr *iomsg)
>>>>>>  {
>>>>>> -	iomsg->iov = iomsg->fast_iov;
>>>>>>  	iomsg->msg.msg_name = &iomsg->addr;
>>>>>> +	iomsg->free_iov = iomsg->fast_iov;
>>>>>
>>>>> Why this? Isn't the idea of this patch that free_iov is never == fast_iov?
>>>>
>>>> That's a part of __import_iovec() and sendmsg_copy_msghdr() API, you pass
>>>> fast_iov as such and get back NULL or a newly allocated one in it.
>>> I think that should at least get a comment to make this clear and
>>> maybe a temporary variable like this:
>>>
>>> tmp_iov = iomsg->fast_iov;
>>> __import_iovec(..., &tmp_iov, ...);
>>> iomsg->free_iov = tmp_iov;
>>
>> I'd rather disagree. It's a well known (ish) API, and I deliberately
>> placed such assignments right before import_iovec/etc.
> 
> Agree on that, it's kind of a weird idiom, but it's used throughout the
> kernel. However:

Maybe someday someone will refactor it and make it accepting inline_vecs
directly...

> 
>>>>> kfree() handles NULL, or is this a hot path and we want to avoid a function call?
>>>>
>>>> Yes, the hot path here is not having iov allocated, and Jens told before
>>>> that he had observed overhead for a similar place in io_[read,write].
>>>
>>> Ok, a comment would also help here...
> 
> I do agree on that one, since otherwise we get patches for it as has
> been proven by the few other spots... I'll add then when queueing this
> up.

I didn't disagree with that, just forgot to throw one to protect ourselves
from static analysis tools. Thanks for fixing up

-- 
Pavel Begunkov

^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2021-02-05 23:23 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-02-05  0:57 [PATCH for-next 0/3] sendmsg/recvmsg cleanup Pavel Begunkov
2021-02-05  0:57 ` [PATCH 1/3] io_uring: set msg_name on msg fixup Pavel Begunkov
2021-02-05  0:57 ` [PATCH 2/3] io_uring: clean iov usage for recvmsg buf select Pavel Begunkov
2021-02-05  0:58 ` [PATCH 3/3] io_uring: refactor sendmsg/recvmsg iov managing Pavel Begunkov
2021-02-05  7:17   ` Stefan Metzmacher
2021-02-05  9:48     ` Pavel Begunkov
2021-02-05  9:58       ` Stefan Metzmacher
2021-02-05 10:06         ` Pavel Begunkov
2021-02-05 14:45           ` Jens Axboe
2021-02-05 14:57             ` Pavel Begunkov
2021-02-05 14:46 ` [PATCH for-next 0/3] sendmsg/recvmsg cleanup Jens Axboe

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox