Re: [PATCH V6 0/8] io_uring: support sqe group and provide group kbuf

[Jens Axboe <axboe@kernel.dk>]

On 9/26/24 4:27 AM, Ming Lei wrote:
> Hello Pavel, Jens and Guys,
> 
> On Thu, Sep 12, 2024 at 06:49:20PM +0800, Ming Lei wrote:
>> Hello,
>>
>> The 1st 3 patches are cleanup, and prepare for adding sqe group.
>>
>> The 4th patch supports generic sqe group which is like link chain, but
>> allows each sqe in group to be issued in parallel and the group shares
>> same IO_LINK & IO_DRAIN boundary, so N:M dependency can be supported with
>> sqe group & io link together. sqe group changes nothing on
>> IOSQE_IO_LINK.
>>
>> The 5th patch supports one variant of sqe group: allow members to depend
>> on group leader, so that kernel resource lifetime can be aligned with
>> group leader or group, then any kernel resource can be shared in this
>> sqe group, and can be used in generic device zero copy.
>>
>> The 6th & 7th patches supports providing sqe group buffer via the sqe
>> group variant.
>>
>> The 8th patch supports ublk zero copy based on io_uring providing sqe
>> group buffer.
>>
>> Tests:
>>
>> 1) pass liburing test
>> - make runtests
>>
>> 2) write/pass sqe group test case and sqe provide buffer case:
>>
>> https://github.com/axboe/liburing/compare/master...ming1:liburing:sqe_group_v3
>>
>> https://github.com/ming1/liburing/tree/sqe_group_v3
>>
>> - covers related sqe flags combination and linking groups, both nop and
>> one multi-destination file copy.
>>
>> - cover failure handling test: fail leader IO or member IO in both single
>>   group and linked groups, which is done in each sqe flags combination
>>   test
>>
>> - covers IORING_PROVIDE_GROUP_KBUF by adding ublk-loop-zc
>>
>> 3) ublksrv zero copy:
>>
>> ublksrv userspace implements zero copy by sqe group & provide group
>> kbuf:
>>
>> 	git clone https://github.com/ublk-org/ublksrv.git -b group-provide-buf_v3
>> 	make test T=loop/009:nbd/061	#ublk zc tests
>>
>> When running 64KB/512KB block size test on ublk-loop('ublk add -t loop --buffered_io -f $backing'),
>> it is observed that perf is doubled.
>>
>>
>> V6:
>> 	- follow Pavel's suggestion to disallow IOSQE_CQE_SKIP_SUCCESS &
>> 	  LINK_TIMEOUT
>> 	- kill __io_complete_group_member() (Pavel)
>> 	- simplify link failure handling (Pavel)
>> 	- move members' queuing out of completion lock (Pavel)
>> 	- cleanup group io complete handler
>> 	- add more comment
>> 	- add ublk zc into liburing test for covering
>> 	  IOSQE_SQE_GROUP & IORING_PROVIDE_GROUP_KBUF 
> 
> Any comments on V6? So that I may address them in next version since
> v6 has small conflict with mainline.

It looks fine to me, don't know if Pavel has any comments. Maybe just
toss out a v7 so it applies cleanly? I'll kick off the 6.13 branch
pretty soon.

-- 
Jens Axboe