Re: Zero-copy irq-driven data

[Ricardo Ribalda <ribalda@chromium.org>]

Hi Pavel

Thanks for your response

On Fri, Dec 4, 2020 at 5:09 PM Pavel Begunkov <asml.silence@gmail.com> wrote:
>
> On 03/12/2020 15:26, Ricardo Ribalda wrote:
> > Hello
> >
> > I have just started using io_uring so please bear with me.
> >
> > I have a device that produces data at random time and I want to read
> > it with the lowest latency possible and hopefully zero copy.
> >
> > In userspace:
> >
> > I have a sqe with a bunch of io_uring_prep_read_fixed and when they
> > are ready I process them and push them again to the sqe, so it always
> > has operations.
>
> SQ - submission queue, SQE - SQ entry.
> To clarify misunderstanding I guess you wanted to say that you have
> an SQ filled with fixed read requests (i.e. SQEs prep'ed with
> io_uring_prep_read_fixed()), and so on.


Sorry, I am a mess with acronyms.

>
>
> >
> > In kernelspace:
> >
> > I have implemented the read() file operation in my driver. The data
>
> I'd advise you to implement read_iter() instead, otherwise io_uring
> won't be able to get all performance out of it, especially for fixed
> reqs.
>
> > handling follows this loop:
> >
> > loop():
> >  1) read() gets called by io_uring
> >  2) save the userpointer and the length into a structure
> >  3) go to sleep
> >  4) get an IRQ from the device, with new data
> >  5) dma/copy the data to the user
> >  6) wake up read() and return
> >
> > I guess at this point you see my problem.... What happens if I get an
> > IRQ between 6 and 1?
> > Even if there are plenty of read_operations waiting in the sqe, that
> > data will be lost. :(
>
> Frankly, that's not related to io_uring and more rather a device driver
> writing question. That's not the right list to ask these questions.
> Though I don't know which would suit your case...
>
> > So I guess what I am asking is:
> >
> > A) Am I doing something stupid?
>
> In essence, since you're writing up your own driver from scratch
> (not on top of some framework), all that stuff is to you to handle.
> E.g. you may create a list and adding a short entry with an address
> to dma on each IRQ. And then dma and serve them only when you've got
> a request. Or any other design. But for sure there will be enough
> of pitfalls on your way.
>
> Also, I'd recommend first to make it work with old good read(2) first.
>
> >
> > B) Is there a way for a driver to call a callback when it receives
> > data and push it to a read operation on the cqe?
>
> In short: No
>
> After you fill an SQE (which is also just a chunk of memory), io_uring
> gets it and creates a request, which in your case will call ->read*().
> So you'd get a driver-visible read request (not necessarily issued by
> io_uring)
>
> >
> > C) Can I ask the io_uring to call read() more than once if there are
> > more read_operations in the sqe?
>
> "read_operations in the sqe" what it means?

Lets say I have 3 read_operations in the sq. A standard trace from the
driver will look like

read()
 return
read()
 return
read ()
 return

If I could get

read()
read()
read()
 return
 return
 return

Then I would not lose any data during " read() reloading"

>
> >
> > D) Can the driver inspect what is in the sqe, to make an educated
>
> No, and shouldn't be needed.
>
> > decision of delaying the irq handling for some cycles if there are
> > more reads pending?
>
> --
> Pavel Begunkov



-- 
Ricardo Ribalda