public inbox for [email protected]
 help / color / mirror / Atom feed
From: Ming Lei <[email protected]>
To: Pavel Begunkov <[email protected]>
Cc: [email protected], [email protected],
	Jens Axboe <[email protected]>, Kanchan Joshi <[email protected]>
Subject: Re: [PATCH v2 05/14] ublk: don't hard code IO_URING_F_UNLOCKED
Date: Mon, 18 Mar 2024 22:39:30 +0800	[thread overview]
Message-ID: <ZfhSIt/4w+z6/5U2@fedora> (raw)
In-Reply-To: <[email protected]>

On Mon, Mar 18, 2024 at 02:32:16PM +0000, Pavel Begunkov wrote:
> On 3/18/24 13:37, Pavel Begunkov wrote:
> > On 3/18/24 12:52, Pavel Begunkov wrote:
> > > On 3/18/24 08:16, Ming Lei wrote:
> > > > On Mon, Mar 18, 2024 at 12:41:50AM +0000, Pavel Begunkov wrote:
> > > > > uring_cmd implementations should not try to guess issue_flags, just use
> > > > > a newly added io_uring_cmd_complete(). We're loosing an optimisation in
> > > > > the cancellation path in ublk_uring_cmd_cancel_fn(), but the assumption
> > > > > is that we don't care that much about it.
> > > > > 
> > > > > Signed-off-by: Pavel Begunkov <[email protected]>
> > > > > Link: https://lore.kernel.org/r/2f7bc9fbc98b11412d10b8fd88e58e35614e3147.1710514702.git.asml.silence@gmail.com
> > > > > Signed-off-by: Jens Axboe <[email protected]>
> > > > > ---
> > > > >   drivers/block/ublk_drv.c | 18 ++++++++----------
> > > > >   1 file changed, 8 insertions(+), 10 deletions(-)
> > > > > 
> > > > > diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c
> > > > > index bea3d5cf8a83..97dceecadab2 100644
> > > > > --- a/drivers/block/ublk_drv.c
> > > > > +++ b/drivers/block/ublk_drv.c
> > > > > @@ -1417,8 +1417,7 @@ static bool ublk_abort_requests(struct ublk_device *ub, struct ublk_queue *ubq)
> > > > >       return true;
> > > > >   }
> > > > > -static void ublk_cancel_cmd(struct ublk_queue *ubq, struct ublk_io *io,
> > > > > -        unsigned int issue_flags)
> > > > > +static void ublk_cancel_cmd(struct ublk_queue *ubq, struct ublk_io *io)
> > > > >   {
> > > > >       bool done;
> > > > > @@ -1432,15 +1431,14 @@ static void ublk_cancel_cmd(struct ublk_queue *ubq, struct ublk_io *io,
> > > > >       spin_unlock(&ubq->cancel_lock);
> > > > >       if (!done)
> > > > > -        io_uring_cmd_done(io->cmd, UBLK_IO_RES_ABORT, 0, issue_flags);
> > > > > +        io_uring_cmd_complete(io->cmd, UBLK_IO_RES_ABORT, 0);
> > > > >   }
> > > > >   /*
> > > > >    * The ublk char device won't be closed when calling cancel fn, so both
> > > > >    * ublk device and queue are guaranteed to be live
> > > > >    */
> > > > > -static void ublk_uring_cmd_cancel_fn(struct io_uring_cmd *cmd,
> > > > > -        unsigned int issue_flags)
> > > > > +static void ublk_uring_cmd_cancel_fn(struct io_uring_cmd *cmd)
> > > > >   {
> > > > >       struct ublk_uring_cmd_pdu *pdu = ublk_get_uring_cmd_pdu(cmd);
> > > > >       struct ublk_queue *ubq = pdu->ubq;
> > > > > @@ -1464,7 +1462,7 @@ static void ublk_uring_cmd_cancel_fn(struct io_uring_cmd *cmd,
> > > > >       io = &ubq->ios[pdu->tag];
> > > > >       WARN_ON_ONCE(io->cmd != cmd);
> > > > > -    ublk_cancel_cmd(ubq, io, issue_flags);
> > > > > +    ublk_cancel_cmd(ubq, io);
> > > > 
> > > > .cancel_fn is always called with .uring_lock held, so this 'issue_flags' can't
> > > > be removed, otherwise double task run is caused because .cancel_fn
> > > > can be called multiple times if the request stays in ctx->cancelable_uring_cmd.
> > > 
> > > I see, that's exactly why I was asking whether it can be deferred
> > > to tw. Let me see if I can get by without that patch, but honestly
> > > it's a horrible abuse of the ring state. Any ideas how that can be
> > > cleaned up?
> > 
> > I assume io_uring_try_cancel_uring_cmd() can run in parallel with
> > completions, so there can be two parallel calls calls to ->uring_cmd
> > (e.g. io-wq + cancel), which gives me shivers. Also, I'd rather
> > no cancel in place requests of another task, io_submit_flush_completions()
> > but it complicates things.
> 
> I'm wrong though on flush_completions, the task there cancels only
> its own requests
> 
> io_uring_try_cancel_uring_cmd() {
> 	...
> 	if (!cancel_all && req->task != task)
> 		continue;
> }
> 
> 
> > Is there any argument against removing requests from the cancellation
> > list in io_uring_try_cancel_uring_cmd() before calling ->uring_cmd?
> > 
> > io_uring_try_cancel_uring_cmd() {
> >      lock();
> >      for_each_req() {
> >          remove_req_from_cancel_list(req);
> >          req->file->uring_cmd();
> >      }
> >      unlock();

Also the req may not be ready to cancel in ->uring_cmd(), so it
should be allowed to retry in future if it isn't canceled this time.


Thanks,
Ming


  reply	other threads:[~2024-03-18 14:40 UTC|newest]

Thread overview: 41+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-03-18  0:41 [PATCH v2 00/14] remove aux CQE caches Pavel Begunkov
2024-03-18  0:41 ` [PATCH v2 01/14] io_uring/cmd: kill one issue_flags to tw conversion Pavel Begunkov
2024-03-18  0:41 ` [PATCH v2 02/14] io_uring/cmd: fix tw <-> issue_flags conversion Pavel Begunkov
2024-03-18  2:23   ` Ming Lei
2024-03-18  2:25     ` Jens Axboe
2024-03-18  2:32       ` Pavel Begunkov
2024-03-18  2:40         ` Jens Axboe
2024-03-18  2:43           ` Pavel Begunkov
2024-03-18  2:46             ` Jens Axboe
2024-03-18  2:47           ` Ming Lei
2024-03-18  3:11             ` Jens Axboe
2024-03-18  3:24               ` Pavel Begunkov
2024-03-18  6:59               ` Ming Lei
2024-03-18 11:45                 ` Pavel Begunkov
2024-03-18  0:41 ` [PATCH v2 03/14] io_uring/cmd: make io_uring_cmd_done irq safe Pavel Begunkov
2024-03-18  8:10   ` Ming Lei
2024-03-18 11:50     ` Pavel Begunkov
2024-03-18 11:59       ` Ming Lei
2024-03-18 12:46         ` Pavel Begunkov
2024-03-18 13:09           ` Ming Lei
2024-03-18  0:41 ` [PATCH v2 04/14] io_uring/cmd: introduce io_uring_cmd_complete Pavel Begunkov
2024-03-18  0:41 ` [PATCH v2 05/14] ublk: don't hard code IO_URING_F_UNLOCKED Pavel Begunkov
2024-03-18  8:16   ` Ming Lei
2024-03-18 12:52     ` Pavel Begunkov
2024-03-18 13:37       ` Pavel Begunkov
2024-03-18 14:32         ` Pavel Begunkov
2024-03-18 14:39           ` Ming Lei [this message]
2024-03-18 14:34       ` Ming Lei
2024-03-18 15:08         ` Pavel Begunkov
2024-03-18 15:16           ` Ming Lei
2024-03-18  0:41 ` [PATCH v2 06/14] nvme/io_uring: " Pavel Begunkov
2024-03-18 13:26   ` Kanchan Joshi
2024-03-18 13:38     ` Pavel Begunkov
2024-03-18  0:41 ` [PATCH v2 07/14] io_uring/rw: avoid punting to io-wq directly Pavel Begunkov
2024-03-18  0:41 ` [PATCH v2 08/14] io_uring: force tw ctx locking Pavel Begunkov
2024-03-18  0:41 ` [PATCH v2 09/14] io_uring: remove struct io_tw_state::locked Pavel Begunkov
2024-03-18  0:41 ` [PATCH v2 10/14] io_uring: refactor io_fill_cqe_req_aux Pavel Begunkov
2024-03-18  0:41 ` [PATCH v2 11/14] io_uring: get rid of intermediate aux cqe caches Pavel Begunkov
2024-03-18  0:41 ` [PATCH v2 12/14] io_uring: remove current check from complete_post Pavel Begunkov
2024-03-18  0:41 ` [PATCH v2 13/14] io_uring: refactor io_req_complete_post() Pavel Begunkov
2024-03-18  0:41 ` [PATCH v2 14/14] io_uring: clean up io_lockdep_assert_cq_locked Pavel Begunkov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=ZfhSIt/4w+z6/5U2@fedora \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox