* Re: [PATCH] fuse: add zero-copy to fuse-over-io_uring
2025-12-04 8:25 ` [PATCH] fuse: add zero-copy to fuse-over-io_uring Xiaobing Li
@ 2025-12-04 22:37 ` kernel test robot
2025-12-05 0:26 ` Joanne Koong
2025-12-12 16:50 ` kernel test robot
2 siblings, 0 replies; 4+ messages in thread
From: kernel test robot @ 2025-12-04 22:37 UTC (permalink / raw)
To: Xiaobing Li, miklos, axboe
Cc: oe-kbuild-all, io-uring, linux-fsdevel, bschubert, asml.silence,
joannelkoong, dw, josef, kbusch, peiwei.li, joshi.k, Xiaobing Li
Hi Xiaobing,
kernel test robot noticed the following build warnings:
[auto build test WARNING on mszeredi-fuse/for-next]
[also build test WARNING on linus/master v6.18 next-20251204]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Xiaobing-Li/fuse-add-zero-copy-to-fuse-over-io_uring/20251204-165924
base: https://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse.git for-next
patch link: https://lore.kernel.org/r/20251204082536.17349-1-xiaobing.li%40samsung.com
patch subject: [PATCH] fuse: add zero-copy to fuse-over-io_uring
config: um-randconfig-r052-20251205 (https://download.01.org/0day-ci/archive/20251205/202512050506.gwZpnWio-lkp@intel.com/config)
compiler: gcc-13 (Debian 13.3.0-16) 13.3.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20251205/202512050506.gwZpnWio-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202512050506.gwZpnWio-lkp@intel.com/
All warnings (new ones prefixed by >>):
In file included from fs/fuse/dev_uring.c:8:
fs/fuse/dev_uring_i.h:43:25: error: field 'payload_iter' has incomplete type
43 | struct iov_iter payload_iter;
| ^~~~~~~~~~~~
fs/fuse/dev_uring.c: In function 'fuse_uring_create_ring_ent':
>> fs/fuse/dev_uring.c:1086:49: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
1086 | err = io_uring_cmd_import_fixed((u64)ent->payload, payload_size, ITER_DEST,
| ^
vim +1086 fs/fuse/dev_uring.c
1042
1043 static struct fuse_ring_ent *
1044 fuse_uring_create_ring_ent(struct io_uring_cmd *cmd,
1045 struct fuse_ring_queue *queue)
1046 {
1047 struct fuse_ring *ring = queue->ring;
1048 struct fuse_ring_ent *ent;
1049 size_t payload_size;
1050 struct iovec iov[FUSE_URING_IOV_SEGS];
1051 int err;
1052
1053 err = fuse_uring_get_iovec_from_sqe(cmd->sqe, iov);
1054 if (err) {
1055 pr_info_ratelimited("Failed to get iovec from sqe, err=%d\n",
1056 err);
1057 return ERR_PTR(err);
1058 }
1059
1060 err = -EINVAL;
1061 if (iov[0].iov_len < sizeof(struct fuse_uring_req_header)) {
1062 pr_info_ratelimited("Invalid header len %zu\n", iov[0].iov_len);
1063 return ERR_PTR(err);
1064 }
1065
1066 payload_size = iov[1].iov_len;
1067 if (payload_size < ring->max_payload_sz) {
1068 pr_info_ratelimited("Invalid req payload len %zu\n",
1069 payload_size);
1070 return ERR_PTR(err);
1071 }
1072
1073 err = -ENOMEM;
1074 ent = kzalloc(sizeof(*ent), GFP_KERNEL_ACCOUNT);
1075 if (!ent)
1076 return ERR_PTR(err);
1077
1078 INIT_LIST_HEAD(&ent->list);
1079
1080 ent->queue = queue;
1081 ent->headers = iov[0].iov_base;
1082 ent->payload = iov[1].iov_base;
1083
1084 if (READ_ONCE(cmd->sqe->uring_cmd_flags) & IORING_URING_CMD_FIXED) {
1085 ent->zero_copy = true;
> 1086 err = io_uring_cmd_import_fixed((u64)ent->payload, payload_size, ITER_DEST,
1087 &ent->payload_iter, cmd, 0);
1088
1089 if (err) {
1090 kfree(ent);
1091 return ERR_PTR(err);
1092 }
1093 }
1094
1095 atomic_inc(&ring->queue_refs);
1096 return ent;
1097 }
1098
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: [PATCH] fuse: add zero-copy to fuse-over-io_uring
2025-12-04 8:25 ` [PATCH] fuse: add zero-copy to fuse-over-io_uring Xiaobing Li
2025-12-04 22:37 ` kernel test robot
@ 2025-12-05 0:26 ` Joanne Koong
2025-12-12 16:50 ` kernel test robot
2 siblings, 0 replies; 4+ messages in thread
From: Joanne Koong @ 2025-12-05 0:26 UTC (permalink / raw)
To: Xiaobing Li
Cc: miklos, axboe, io-uring, linux-fsdevel, bschubert, asml.silence,
dw, josef, kbusch, peiwei.li, joshi.k
On Thu, Dec 4, 2025 at 12:57 AM Xiaobing Li <xiaobing.li@samsung.com> wrote:
>
> Joanne has submitted a patch for adding a zero-copy solution.
>
> We have also done some research and testing before, and
> the test data shows improved performance. This patch is
> submitted for discussion.
Hi Xiaobing,
I think the logic in this patch uses fixed buffers for the request
payloads to reduce the per-I/O overhead of pinning/unpinning the user
pages for the payload but I don't think this helps with zero-copying,
as the pages still need to get copied from the registered buffer the
server provided back to the client's pages (or vice versa). In the
implementation in [1], the client's pages are registered into the
sparse buffer by the kernel such that the server reads/writes to those
client pages directly.
I think your patch is doing a similar thing to what the patch from
last month for registered buffers [2] did, except [2] also implemented
fixed buffers for the headers to eliminate that per-I/O overhead too.
However, when I thought about this design some more, I realized if we
used kernel-managed ring buffers instead, we pretty much get the same
wins on reducing I/O overhead but also can significantly reduce the
memory footprint used by each queue, by allowing the buffers to be
incrementally consumed and thus sharable across requests, as well as
allowing more flexibility in the future if we want to do things like
add multiple ring-pools for different categories of requests, etc.
Some more thoughts on this can be found on this thread here [3].
Thanks,
Joanne
[1] https://lore.kernel.org/linux-fsdevel/20251203003526.2889477-1-joannelkoong@gmail.com/
[2] https://lore.kernel.org/linux-fsdevel/20251027222808.2332692-9-joannelkoong@gmail.com/
[3] https://lore.kernel.org/linux-fsdevel/CAJnrk1bG7fAX8MfwJL_D2jzMNv5Rj9=1cgQvVpqC1=mGaeAwOg@mail.gmail.com/
>
> libfuse section:
> https://github.com/lreeze123/libfuse/tree/zero-copy
>
> Signed-off-by: Xiaobing Li <xiaobing.li@samsung.com>
> ---
> fs/fuse/dev_uring.c | 44 +++++++++++++++++++++++++++++++------------
> fs/fuse/dev_uring_i.h | 4 ++++
> 2 files changed, 36 insertions(+), 12 deletions(-)
>
> diff --git a/fs/fuse/dev_uring.c b/fs/fuse/dev_uring.c
> index f6b12aebb8bb..23790ae78853 100644
> --- a/fs/fuse/dev_uring.c
> +++ b/fs/fuse/dev_uring.c
> @@ -584,15 +584,20 @@ static int fuse_uring_copy_from_ring(struct fuse_ring *ring,
> int err;
> struct fuse_uring_ent_in_out ring_in_out;
>
> - err = copy_from_user(&ring_in_out, &ent->headers->ring_ent_in_out,
> - sizeof(ring_in_out));
> - if (err)
> - return -EFAULT;
> + if (ent->zero_copy) {
> + iter = ent->payload_iter;
> + ring_in_out.payload_sz = ent->cmd->sqe->len;
> + } else {
> + err = copy_from_user(&ring_in_out, &ent->headers->ring_ent_in_out,
> + sizeof(ring_in_out));
> + if (err)
> + return -EFAULT;
>
> - err = import_ubuf(ITER_SOURCE, ent->payload, ring->max_payload_sz,
> - &iter);
> - if (err)
> - return err;
> + err = import_ubuf(ITER_SOURCE, ent->payload, ring->max_payload_sz,
> + &iter);
> + if (err)
> + return err;
> + }
>
> fuse_copy_init(&cs, false, &iter);
> cs.is_uring = true;
> @@ -618,10 +623,14 @@ static int fuse_uring_args_to_ring(struct fuse_ring *ring, struct fuse_req *req,
> .commit_id = req->in.h.unique,
> };
>
> - err = import_ubuf(ITER_DEST, ent->payload, ring->max_payload_sz, &iter);
> - if (err) {
> - pr_info_ratelimited("fuse: Import of user buffer failed\n");
> - return err;
> + if (ent->zero_copy) {
> + iter = ent->payload_iter;
> + } else {
> + err = import_ubuf(ITER_DEST, ent->payload, ring->max_payload_sz, &iter);
> + if (err) {
> + pr_info_ratelimited("fuse: Import of user buffer failed\n");
> + return err;
> + }
> }
>
> fuse_copy_init(&cs, true, &iter);
> @@ -1068,6 +1077,17 @@ fuse_uring_create_ring_ent(struct io_uring_cmd *cmd,
> ent->headers = iov[0].iov_base;
> ent->payload = iov[1].iov_base;
>
> + if (READ_ONCE(cmd->sqe->uring_cmd_flags) & IORING_URING_CMD_FIXED) {
> + ent->zero_copy = true;
> + err = io_uring_cmd_import_fixed((u64)ent->payload, payload_size, ITER_DEST,
> + &ent->payload_iter, cmd, 0);
> +
> + if (err) {
> + kfree(ent);
> + return ERR_PTR(err);
> + }
> + }
> +
> atomic_inc(&ring->queue_refs);
> return ent;
> }
> diff --git a/fs/fuse/dev_uring_i.h b/fs/fuse/dev_uring_i.h
> index 51a563922ce1..cb5de6e7a262 100644
> --- a/fs/fuse/dev_uring_i.h
> +++ b/fs/fuse/dev_uring_i.h
> @@ -38,6 +38,10 @@ enum fuse_ring_req_state {
>
> /** A fuse ring entry, part of the ring queue */
> struct fuse_ring_ent {
> + bool zero_copy;
> +
> + struct iov_iter payload_iter;
> +
> /* userspace buffer */
> struct fuse_uring_req_header __user *headers;
> void __user *payload;
> --
> 2.34.1
>
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: [PATCH] fuse: add zero-copy to fuse-over-io_uring
2025-12-04 8:25 ` [PATCH] fuse: add zero-copy to fuse-over-io_uring Xiaobing Li
2025-12-04 22:37 ` kernel test robot
2025-12-05 0:26 ` Joanne Koong
@ 2025-12-12 16:50 ` kernel test robot
2 siblings, 0 replies; 4+ messages in thread
From: kernel test robot @ 2025-12-12 16:50 UTC (permalink / raw)
To: Xiaobing Li, miklos, axboe
Cc: llvm, oe-kbuild-all, io-uring, linux-fsdevel, bschubert,
asml.silence, joannelkoong, dw, josef, kbusch, peiwei.li, joshi.k,
Xiaobing Li
Hi Xiaobing,
kernel test robot noticed the following build errors:
[auto build test ERROR on mszeredi-fuse/for-next]
[also build test ERROR on linus/master v6.18 next-20251212]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Xiaobing-Li/fuse-add-zero-copy-to-fuse-over-io_uring/20251204-165924
base: https://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse.git for-next
patch link: https://lore.kernel.org/r/20251204082536.17349-1-xiaobing.li%40samsung.com
patch subject: [PATCH] fuse: add zero-copy to fuse-over-io_uring
config: hexagon-randconfig-002-20251212 (https://download.01.org/0day-ci/archive/20251213/202512130032.tLxDKB6B-lkp@intel.com/config)
compiler: clang version 22.0.0git (https://github.com/llvm/llvm-project 1335a05ab8bc8339ce24be3a9da89d8c3f4e0571)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20251213/202512130032.tLxDKB6B-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202512130032.tLxDKB6B-lkp@intel.com/
All errors (new ones prefixed by >>):
In file included from fs/fuse/trace.c:6:
>> fs/fuse/dev_uring_i.h:43:18: error: field has incomplete type 'struct iov_iter'
43 | struct iov_iter payload_iter;
| ^
include/linux/fs.h:74:8: note: forward declaration of 'struct iov_iter'
74 | struct iov_iter;
| ^
1 error generated.
vim +43 fs/fuse/dev_uring_i.h
38
39 /** A fuse ring entry, part of the ring queue */
40 struct fuse_ring_ent {
41 bool zero_copy;
42
> 43 struct iov_iter payload_iter;
44
45 /* userspace buffer */
46 struct fuse_uring_req_header __user *headers;
47 void __user *payload;
48
49 /* the ring queue that owns the request */
50 struct fuse_ring_queue *queue;
51
52 /* fields below are protected by queue->lock */
53
54 struct io_uring_cmd *cmd;
55
56 struct list_head list;
57
58 enum fuse_ring_req_state state;
59
60 struct fuse_req *fuse_req;
61 };
62
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 4+ messages in thread