public inbox for [email protected]
 help / color / mirror / Atom feed
From: Jens Axboe <[email protected]>
To: Jeff Moyer <[email protected]>,
	Bijan Mottahedeh <[email protected]>
Cc: [email protected]
Subject: Re: [RFC 2/2] io_uring: mark REQ_NOWAIT for a non-mq queue as unspported
Date: Thu, 28 May 2020 13:01:59 -0600	[thread overview]
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>

On 5/28/20 12:35 PM, Jeff Moyer wrote:
> Bijan Mottahedeh <[email protected]> writes:
> 
>> Mark a REQ_NOWAIT request for a non-mq queue as unspported instead of
>> retryable since otherwise the io_uring layer will keep resubmitting
>> the request.
> 
> Getting back to this...
> 
> Jens, right now (using your io_uring-5.7 or linus' tree) fio's
> t/io_uring will never get io completions when run against a file on a
> file system that is backed by lvm.  The system will have one workqueue
> per sqe submitted, all spinning, eating up CPU time.
> 
> # ./t/io_uring /mnt/test/poo 
> Added file /mnt/test/poo
> sq_ring ptr = 0x0x7fbed40ae000
> sqes ptr    = 0x0x7fbed40ac000
> cq_ring ptr = 0x0x7fbed40aa000
> polled=1, fixedbufs=1, buffered=0 QD=128, sq_ring=128, cq_ring=256
> submitter=3851
> IOPS=128, IOS/call=6/0, inflight=128 (128)
> IOPS=0, IOS/call=0/0, inflight=128 (128)
> IOPS=0, IOS/call=0/0, inflight=128 (128)
> IOPS=0, IOS/call=0/0, inflight=128 (128)
> IOPS=0, IOS/call=0/0, inflight=128 (128)
> IOPS=0, IOS/call=0/0, inflight=128 (128)
> ...
> 
> # ps auxw | grep io_wqe
> root      3849 80.1  0.0      0     0 ?        R    14:32   0:40 [io_wqe_worker-0]
> root      3850  0.0  0.0      0     0 ?        S    14:32   0:00 [io_wqe_worker-0]
> root      3853 72.8  0.0      0     0 ?        R    14:32   0:36 [io_wqe_worker-0]
> root      3854 81.4  0.0      0     0 ?        R    14:32   0:40 [io_wqe_worker-1]
> root      3855 74.8  0.0      0     0 ?        R    14:32   0:37 [io_wqe_worker-0]
> root      3856 74.8  0.0      0     0 ?        R    14:32   0:37 [io_wqe_worker-1]
> ...
> 
> # ps auxw | grep io_wqe | grep -v grep | wc -l
> 129
> 
> With this patch applied, the test program will exit without doing I/O
> (which I don't think is the right behavior either, right?):
> 
> # t/io_uring /mnt/test/poo
> Added file /mnt/test/poo
> sq_ring ptr = 0x0x7fdb98f00000
> sqes ptr    = 0x0x7fdb98efe000
> cq_ring ptr = 0x0x7fdb98efc000
> polled=1, fixedbufs=1, buffered=0 QD=128, sq_ring=128, cq_ring=256
> submitter=33233
> io: unexpected ret=-95
> Your filesystem/driver/kernel doesn't support polled IO
> IOPS=128, IOS/call=32/0, inflight=128 (127)
> 
> /mnt/test is an xfs file system on top of a linear LVM volume on an nvme
> device (with 8 poll queues configured).

poll won't work over dm, so that looks correct. What happens if you edit
it and disable poll? Would be curious to see both buffered = 0 and
buffered = 1 runs with that.

I'll try this here too.

-- 
Jens Axboe


  reply	other threads:[~2020-05-28 19:02 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-05-19 21:52 [RFC 0/2] io_uring: don't use kiocb.private to store buf_index Bijan Mottahedeh
2020-05-19 21:52 ` [RFC 1/2] " Bijan Mottahedeh
2020-05-19 22:07   ` Jens Axboe
2020-05-19 22:20     ` Jens Axboe
2020-05-19 22:48       ` Bijan Mottahedeh
2020-05-19 21:52 ` [RFC 2/2] io_uring: mark REQ_NOWAIT for a non-mq queue as unspported Bijan Mottahedeh
2020-05-28 18:35   ` Jeff Moyer
2020-05-28 19:01     ` Jens Axboe [this message]
2020-05-28 19:22       ` Jens Axboe
2020-05-28 19:31         ` Jeff Moyer
2020-05-28 22:12         ` Jeff Moyer
2020-05-28 23:03           ` Jens Axboe
2020-05-29 15:02             ` Jeff Moyer
2020-05-29 16:20               ` Jens Axboe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox