public inbox for [email protected]
 help / color / mirror / Atom feed
* [PATCHSET 0/3] Ensure file refs are dropped on io_uring fd release
@ 2023-08-11 17:12 Jens Axboe
  2023-08-11 17:12 ` [PATCH 1/3] io_uring: move to using private ring references Jens Axboe
                   ` (2 more replies)
  0 siblings, 3 replies; 7+ messages in thread
From: Jens Axboe @ 2023-08-11 17:12 UTC (permalink / raw)
  To: io-uring

Hi,

When the io_uring fd is closed, we ensure that any pending or future
request is canceled when run. But we don't wait for that to happen,
it happens out-of-line in a workqueue. This means that if you kill a
task that has pending IO with io_uring, when the process has exited,
there's still an amount of time until all file references has gone away.
This makes a test case like:

#!/bin/bash

DEV=/dev/nvme0n1
MNT=/data
ITER=0

while true; do
	echo loop $ITER
	sudo mount $DEV $MNT
	fio --name=test --ioengine=io_uring --iodepth=2 --filename=$MNT/foo --size=1g --buffered=1 --overwrite=0 --numjobs=12 --minimal --rw=randread --thread=1 --output=/dev/null &
	Y=$(($RANDOM % 3))
	X=$(($RANDOM % 10))
	VAL="$Y.$X"
	sleep $VAL
	ps -e | grep fio > /dev/null 2>&1
	while [ $? -eq 0 ]; do
		killall -9 fio > /dev/null 2>&1
		wait > /dev/null 2>&1
		ps -e | grep "fio " > /dev/null 2>&1
	done
	sudo umount /data
	if [ $? -ne 0 ]; then
		break
	fi
	((ITER++))
done

fail with -EBUSY for the umount, even though the task that had them
open is gone and reaped.

This patchset attempts to rectify that. Patch 1 switches us to a
simpler private percpu reference count, which means we don't need to
sync RCU when exiting. A RCU grace period can take a long time, and now
the task will be waiting for that when exiting the ring fd. Patch 2
tweaks when we consider things cancelable, moving away from using
PF_EXITING and just looking at the ring ref state instead. Patch 3
finally does the trivial "wait for cancelations to happen before
considering the fd closed" trick which fixes the above test case.

-- 
Jens Axboe



^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2023-08-15 22:53 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-08-11 17:12 [PATCHSET 0/3] Ensure file refs are dropped on io_uring fd release Jens Axboe
2023-08-11 17:12 ` [PATCH 1/3] io_uring: move to using private ring references Jens Axboe
2023-08-15 17:45   ` Pavel Begunkov
2023-08-15 21:45     ` Jens Axboe
2023-08-15 22:50       ` Pavel Begunkov
2023-08-11 17:12 ` [PATCH 2/3] io_uring: consider ring dead once the ref is marked dying Jens Axboe
2023-08-11 17:12 ` [PATCH 3/3] io_uring: wait for cancelations on final ring put Jens Axboe

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox