Re: [PATCH V6 0/8] io_uring: support sqe group and provide group kbuf

[Pavel Begunkov <asml.silence@gmail.com>]

On 9/26/24 13:18, Jens Axboe wrote:
> On 9/26/24 4:27 AM, Ming Lei wrote:
>> Hello Pavel, Jens and Guys,
>>
>> On Thu, Sep 12, 2024 at 06:49:20PM +0800, Ming Lei wrote:
>>> Hello,
>>>
>>> The 1st 3 patches are cleanup, and prepare for adding sqe group.
>>>
>>> The 4th patch supports generic sqe group which is like link chain, but
>>> allows each sqe in group to be issued in parallel and the group shares
>>> same IO_LINK & IO_DRAIN boundary, so N:M dependency can be supported with
>>> sqe group & io link together. sqe group changes nothing on
>>> IOSQE_IO_LINK.
>>>
>>> The 5th patch supports one variant of sqe group: allow members to depend
>>> on group leader, so that kernel resource lifetime can be aligned with
>>> group leader or group, then any kernel resource can be shared in this
>>> sqe group, and can be used in generic device zero copy.
>>>
>>> The 6th & 7th patches supports providing sqe group buffer via the sqe
>>> group variant.
>>>
>>> The 8th patch supports ublk zero copy based on io_uring providing sqe
>>> group buffer.
>>>
>>> Tests:
>>>
>>> 1) pass liburing test
>>> - make runtests
>>>
>>> 2) write/pass sqe group test case and sqe provide buffer case:
>>>
>>> https://github.com/axboe/liburing/compare/master...ming1:liburing:sqe_group_v3
>>>
>>> https://github.com/ming1/liburing/tree/sqe_group_v3
>>>
>>> - covers related sqe flags combination and linking groups, both nop and
>>> one multi-destination file copy.
>>>
>>> - cover failure handling test: fail leader IO or member IO in both single
>>>    group and linked groups, which is done in each sqe flags combination
>>>    test
>>>
>>> - covers IORING_PROVIDE_GROUP_KBUF by adding ublk-loop-zc
>>>
>>> 3) ublksrv zero copy:
>>>
>>> ublksrv userspace implements zero copy by sqe group & provide group
>>> kbuf:
>>>
>>> 	git clone https://github.com/ublk-org/ublksrv.git -b group-provide-buf_v3
>>> 	make test T=loop/009:nbd/061	#ublk zc tests
>>>
>>> When running 64KB/512KB block size test on ublk-loop('ublk add -t loop --buffered_io -f $backing'),
>>> it is observed that perf is doubled.
>>>
>>>
>>> V6:
>>> 	- follow Pavel's suggestion to disallow IOSQE_CQE_SKIP_SUCCESS &
>>> 	  LINK_TIMEOUT
>>> 	- kill __io_complete_group_member() (Pavel)
>>> 	- simplify link failure handling (Pavel)
>>> 	- move members' queuing out of completion lock (Pavel)
>>> 	- cleanup group io complete handler
>>> 	- add more comment
>>> 	- add ublk zc into liburing test for covering
>>> 	  IOSQE_SQE_GROUP & IORING_PROVIDE_GROUP_KBUF
>>
>> Any comments on V6? So that I may address them in next version since
>> v6 has small conflict with mainline.
> 
> It looks fine to me, don't know if Pavel has any comments. Maybe just
> toss out a v7 so it applies cleanly? I'll kick off the 6.13 branch
> pretty soon.

Impl is not that straightforwardand warrants some prudence in
reviewing. I was visiting conferences, but going to take a look
next week or earlier.

-- 
Pavel Begunkov