Re: [RFC 7/7] io_uring,fs: introduce IORING_OP_GET_BUF

[Ming Lei <ming.lei@redhat.com>]

On Tue, May 02, 2023 at 10:57:30PM +0800, Ming Lei wrote:
> On Sun, Apr 30, 2023 at 10:35:29AM +0100, Pavel Begunkov wrote:
> > There are several problems with splice requests, aka IORING_OP_SPLICE:
> > 1) They are always executed by a worker thread, which is a slow path,
> >    as we don't have any reliable way to execute it NOWAIT.
> > 2) It can't easily poll for data, as there are 2 files it operates on.
> >    It would either need to track what file to poll or poll both of them,
> >    in both cases it'll be a mess and add lot of overhead.
> > 3) It has to have pipes in the middle, which adds overhead and is not
> >    great from the uapi design perspective when it goes for io_uring
> >    requests.
> > 4) We want to operate with spliced data as with a normal buffer, i.e.
> >    write / send / etc. data as normally while it's zerocopy.
> > 
> > It can partially be solved, but the root cause is a suboptimal for
> > io_uring design of IORING_OP_SPLICE. Introduce a new request type
> > called IORING_OP_GET_BUF, inspired by splice(2) as well as other
> > proposals like fused requests. The main idea is to use io_uring's
> > registered buffers as the middle man instead of pipes. Once a buffer
> > is fetched / spliced from a file using a new fops callback
> > ->iou_get_buf, it's installed as a registered buffers and can be used
> > by all operations supporting the feature.
> > 
> > Once the userspace releases the buffer, io_uring will wait for all
> > requests using the buffer to complete and then use a file provided
> > callback ->release() to return the buffer back. It operates on the
> 
> In the commit of "io_uring: add an example for buf-get op", I don't see
> any code to release the buffer, can you explain it in details about how
> to release the buffer in userspace? And add it in your example?
> 
> Here I guess the ->release() is called in the following code path:
> 
> io_buffer_unmap
>     io_rsrc_buf_put
>         io_rsrc_put_work
>             io_rsrc_node_ref_zero
>                 io_put_rsrc_node
> 
> If it is true, what is counter-pair code for io_put_rsrc_node()?

It is io_req_set_rsrc_node() called from io_prep_rw() and
io_send_zc_prep().

So only fs IO and net zc supports it, however the fixed buffer
is used in ->prep(), that means this way can't work when the
rw/net_send_zc OP follows IORING_OP_GET_BUF by IO_LINK.

This way is one big defect because IOSQE_IO_LINK does simplify
application a lot.

Thanks,
Ming