[RFC PATCH 0/1] io_uring: honor I/O nowait flag for read/write

[Chaitanya Kulkarni <kch@nvidia.com>]

Hi,

When IO_URING_F_NONBLOCK is set on io_kiocb req->flag in io_write() or
io_read() IOCB_NOWAIT is set for kiocb when passed it to the respective
rw_iter callback. This sets REQ_NOWAIT for underlaying I/O. The result
is low level driver always sees block layer request as REQ_NOWAIT even
if user has submitted request with nowait = 0 e.g. fio nowait=0.

That is not consistent behaviour with other fio ioengine such as
libaio as it will issue non REQ_NOWAIT request with REQ_NOWAIT:-

libaio nowait = 0:-
null_blk: fio:null_handle_rq 1288 *NOWAIT=FALSE* REQ_OP_WRITE

libaio nowait = 1:-
null_blk: fio:null_handle_rq 1288 *NOWAIT=TRUE* REQ_OP_WRITE

* Without this patch with fio ioengine io_uring :-
---------------------------------------------------

iouring nowait = 0:-
null_blk: fio:null_handle_rq 1288 *NOWAIT=TRUE* REQ_OP_WRITE

iouring nowait = 1:-
null_blk: fio:null_handle_rq 1288 *NOWAIT=TRUE* REQ_OP_WRITE

* With this patch with fio ioengine io_uring :-
---------------------------------------------------

iouring nowait = 0:-
null_blk: fio:null_handle_rq 1307 *REQ_NOWAIT=FALSE* WRITE

iouring nowait = 1:
null_blk: fio:null_handle_rq 1307 *REQ_NOWAIT=TRUE* WRITE

Instead of only relying on IO_URING_F_NONBLOCK blindly in io_read() and
io_write(), also make sure io_kiocb->io_rw->flags is set to RWF_NOWAIT
before we mark kiocb->ki_flags = IOCB_NOWAIT.

Below is the deatailed testing log.

-ck

Chaitanya Kulkarni (1):
  io_uring: honor I/O nowait flag for read/write

 io_uring/rw.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

$ sh test-iouring-no-wait
* Without this patch :-
#######################
-------------------libaio nowait = 0--------------------------

++ grep -e ioengine -e rw -e nowait fio/randwrite-libaio-nowait-0.fio
ioengine=libaio
rw=randwrite
nowait=0   <-----------------------------
overwrite=0
++ fio fio/randwrite-libaio-nowait-0.fio --filename=/dev/nullb0
++ grep err
RANDWRITE: (groupid=0, jobs=1): err= 0: pid=40835: Thu Apr 20 21:31:09 2023
++ dmesg -c
[17978.613789] null_blk: disk nullb0 created
[17978.613814] null_blk: module loaded
__________________________________________________________________________
[17978.796560] null_blk: fio:null_handle_rq 1288 *NOWAIT=FALSE* REQ_OP_WRITE

-------------------libaio nowait = 1--------------------------

++ blkdiscard /dev/nullb0
++ grep -e ioengine -e rw -e nowait fio/randwrite-libaio-nowait-1.fio
ioengine=libaio
rw=randwrite
nowait=1   <-----------------------------
overwrite=0
++ fio fio/randwrite-libaio-nowait-1.fio --filename=/dev/nullb0
++ grep err
RANDWRITE: (groupid=0, jobs=1): err= 0: pid=40842: Thu Apr 20 21:31:09 2023
++ dmesg -c
__________________________________________________________________________
[17979.019595] null_blk: fio:null_handle_rq 1288 *NOWAIT=TRUE* REQ_OP_WRITE

-------------------iouring nowait = 0--------------------------

++ blkdiscard /dev/nullb0
++ grep -e ioengine -e rw -e nowait fio/randwrite-iouring-nowait-0.fio
ioengine=io_uring
rw=randwrite
nowait=0   <-----------------------------
overwrite=0
++ fio fio/randwrite-iouring-nowait-0.fio --filename=/dev/nullb0
++ grep err
RANDWRITE: (groupid=0, jobs=1): err= 0: pid=40849: Thu Apr 20 21:31:10 2023
++ dmesg -c
__________________________________________________________________________
[17979.242849] null_blk: fio:null_handle_rq 1288 *NOWAIT=TRUE* REQ_OP_WRITE

-------------------iouring nowait = 1--------------------------

++ blkdiscard /dev/nullb0
++ grep -e ioengine -e rw -e nowait fio/randwrite-iouring-nowait-1.fio
ioengine=io_uring
rw=randwrite
nowait=1   <-----------------------------
overwrite=0
++ fio fio/randwrite-iouring-nowait-1.fio --filename=/dev/nullb0
++ grep err
RANDWRITE: (groupid=0, jobs=1): err= 0: pid=40856: Thu Apr 20 21:31:10 2023
++ dmesg -c
__________________________________________________________________________
[17979.454102] null_blk: fio:null_handle_rq 1288 *NOWAIT=TRUE* REQ_OP_WRITE

* With this patch :-
####################
-------------------iouring nowait = 0--------------------------
+ blkdiscard /dev/nullb0
+ grep -e ioengine -e rw -e nowait fio/randwrite-iouring-nowait-0.fio
ioengine=io_uring
rw=randwrite
nowait=0   <-----------------------------
overwrite=0
+ fio fio/randwrite-iouring-nowait-0.fio --filename=/dev/nullb0
+ grep err
RANDWRITE: (groupid=0, jobs=1): err= 0: pid=2788: Thu Apr 20 23:35:40 2023
+ dmesg -c
__________________________________________________________________________
[  164.255136] null_blk: fio:null_handle_rq 1307 *REQ_NOWAIT=FALSE* WRITE
-------------------iouring nowait = 1--------------------------
+ blkdiscard /dev/nullb0
+ grep -e ioengine -e rw -e nowait fio/randwrite-iouring-nowait-1.fio
ioengine=io_uring
rw=randwrite
nowait=1   <-----------------------------
overwrite=0
+ fio fio/randwrite-iouring-nowait-1.fio --filename=/dev/nullb0
+ grep err
RANDWRITE: (groupid=0, jobs=1): err= 0: pid=2795: Thu Apr 20 23:35:41 2023
+ dmesg -c
__________________________________________________________________________
[  164.467420] null_blk: fio:null_handle_rq 1307 *REQ_NOWAIT=TRUE* WRITE


-- 
2.40.0