From: Ming Lei <[email protected]>
To: Ziyang Zhang <[email protected]>
Cc: [email protected], [email protected],
Harris James R <[email protected]>,
[email protected],
Gabriel Krisman Bertazi <[email protected]>,
Xiaoguang Wang <[email protected]>,
Stefan Hajnoczi <[email protected]>,
Jens Axboe <[email protected]>
Subject: Re: [PATCH V2 1/1] ubd: add io_uring based userspace block driver
Date: Tue, 17 May 2022 20:55:20 +0800 [thread overview]
Message-ID: <YoObOMur7x/u0w1C@T590> (raw)
In-Reply-To: <[email protected]>
On Tue, May 17, 2022 at 06:00:57PM +0800, Ziyang Zhang wrote:
> On 2022/5/17 13:53, Ming Lei wrote:
>
> > +
> > +static void ubd_cancel_queue(struct ubd_queue *ubq)
> > +{
> > + int i;
> > +
> > + for (i = 0; i < ubq->q_depth; i++) {
> > + struct ubd_io *io = &ubq->ios[i];
> > +
> > + if (io->flags & UBD_IO_FLAG_ACTIVE) {
> > + io->flags &= ~UBD_IO_FLAG_ACTIVE;
> > + io_uring_cmd_done(io->cmd, UBD_IO_RES_ABORT, 0);
> > + }
> > + }
> > +}
>
> Hi Ming,
>
> When ubdsrv sends STOP_DEV and all active IOs in ubd_drv are done(UBD_IO_RES_ABORT),
> there may be still some IOs handled by ubdsrv(UBD_IO_FLAG_ACTIVE not set).
> When these IOs complete and return to ubd_drv, how to handle them?
Either UBD_IO_COMMIT_AND_FETCH_REQ or UBD_IO_COMMIT_REQ will be sent to ubd_drv
for completing these IOs. And finally ubd_cancel_dev() in ubd driver will
cancel all pending io commands, so io_uring can be exited. I guess
UBD_IO_COMMIT_REQ can be removed too.
> I find that UBD_IO_FETCH_REQ are still set,
> so will these IOs be issued to ubdsrv again or canceled?
> (I see ubd_drv fails IOs when the daemon is dying
> but maybe here the daemon is still alive)
If daemon is alive, ubd_drv will rely on ubq_daemon for completing
all inflight IOs. Otherwise, the monitor work will be triggered for
completing/failing inflight IOs. The mechanism is actually very simple:
static void ubd_stop_dev(struct ubd_device *ub)
{
mutex_lock(&ub->mutex);
if (!disk_live(ub->ub_disk))
goto unlock;
del_gendisk(ub->ub_disk); // drain & wait in-flight IOs
ub->dev_info.state = UBD_S_DEV_DEAD;
ub->dev_info.ubdsrv_pid = -1;
ubd_cancel_dev(ub); //No IO is possible now, so cancel pending io commands
unlock:
mutex_unlock(&ub->mutex);
cancel_delayed_work_sync(&ub->monitor_work);
}
When waiting for IO completion in del_gendisk(), in case that ubq_daemon
is exiting/dying, monitor work will be triggered to call ubd_abort_queue() to
fail in-flight requests for making forward progress. ubd_abort_queue() may
looks a bit tricky to try using task work for aborting request, that
is just for sync with ubd_rq_task_work_fn().
Thanks,
Ming
next prev parent reply other threads:[~2022-05-17 12:55 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-05-17 5:53 [PATCH V2 0/1] ubd: add io_uring based userspace block driver Ming Lei
2022-05-17 5:53 ` [PATCH V2 1/1] " Ming Lei
2022-05-17 10:00 ` Ziyang Zhang
2022-05-17 12:55 ` Ming Lei [this message]
2022-05-18 5:53 ` Ziyang Zhang
2022-05-17 8:01 ` [PATCH V2 0/1] " Christoph Hellwig
2022-05-17 14:06 ` Stefan Hajnoczi
2022-05-18 7:09 ` Ming Lei
2022-05-18 10:45 ` Stefan Hajnoczi
2022-05-18 12:53 ` Ming Lei
2022-05-18 15:49 ` Stefan Hajnoczi
2022-05-19 2:42 ` Ming Lei
2022-05-19 9:46 ` Stefan Hajnoczi
2022-05-18 6:38 ` Liu Xiaodong
2022-05-18 13:18 ` Ming Lei
2022-05-23 14:56 ` Liu Xiaodong
2022-05-24 2:59 ` Ming Lei
2022-05-18 9:26 ` Stefan Hajnoczi
2022-05-19 13:33 ` Ming Lei
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YoObOMur7x/u0w1C@T590 \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox