public inbox for [email protected]
 help / color / mirror / Atom feed
From: Pavel Begunkov <[email protected]>
To: Ricardo Ribalda <[email protected]>
Cc: [email protected]
Subject: Re: Zero-copy irq-driven data
Date: Fri, 4 Dec 2020 16:06:24 +0000	[thread overview]
Message-ID: <[email protected]> (raw)
In-Reply-To: <CANiDSCsXd1BLUJwgdET5XBF8wQEpbape6BoCPpG9cTGAkUJOBA@mail.gmail.com>

On 03/12/2020 15:26, Ricardo Ribalda wrote:
> Hello
> 
> I have just started using io_uring so please bear with me.
> 
> I have a device that produces data at random time and I want to read
> it with the lowest latency possible and hopefully zero copy.
> 
> In userspace:
> 
> I have a sqe with a bunch of io_uring_prep_read_fixed and when they
> are ready I process them and push them again to the sqe, so it always
> has operations.

SQ - submission queue, SQE - SQ entry.
To clarify misunderstanding I guess you wanted to say that you have
an SQ filled with fixed read requests (i.e. SQEs prep'ed with
io_uring_prep_read_fixed()), and so on.

> 
> In kernelspace:
> 
> I have implemented the read() file operation in my driver. The data

I'd advise you to implement read_iter() instead, otherwise io_uring
won't be able to get all performance out of it, especially for fixed
reqs.

> handling follows this loop:
> 
> loop():
>  1) read() gets called by io_uring
>  2) save the userpointer and the length into a structure
>  3) go to sleep
>  4) get an IRQ from the device, with new data
>  5) dma/copy the data to the user
>  6) wake up read() and return
> 
> I guess at this point you see my problem.... What happens if I get an
> IRQ between 6 and 1?
> Even if there are plenty of read_operations waiting in the sqe, that
> data will be lost. :(

Frankly, that's not related to io_uring and more rather a device driver
writing question. That's not the right list to ask these questions.
Though I don't know which would suit your case...

> So I guess what I am asking is:
> 
> A) Am I doing something stupid?

In essence, since you're writing up your own driver from scratch
(not on top of some framework), all that stuff is to you to handle.
E.g. you may create a list and adding a short entry with an address
to dma on each IRQ. And then dma and serve them only when you've got
a request. Or any other design. But for sure there will be enough
of pitfalls on your way.

Also, I'd recommend first to make it work with old good read(2) first.

> 
> B) Is there a way for a driver to call a callback when it receives
> data and push it to a read operation on the cqe?

In short: No

After you fill an SQE (which is also just a chunk of memory), io_uring
gets it and creates a request, which in your case will call ->read*().
So you'd get a driver-visible read request (not necessarily issued by
io_uring)

> 
> C) Can I ask the io_uring to call read() more than once if there are
> more read_operations in the sqe?

"read_operations in the sqe" what it means?

> 
> D) Can the driver inspect what is in the sqe, to make an educated

No, and shouldn't be needed.

> decision of delaying the irq handling for some cycles if there are
> more reads pending?

-- 
Pavel Begunkov

  reply	other threads:[~2020-12-04 16:10 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-12-03 15:26 Zero-copy irq-driven data Ricardo Ribalda
2020-12-04 16:06 ` Pavel Begunkov [this message]
2020-12-07 15:07   ` Ricardo Ribalda
2020-12-07 23:55     ` Jens Axboe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox