Re: memory access op ideas

[Avi Kivity <avi@scylladb.com>]

On 23/04/2022 20.30, Jens Axboe wrote:
> On 4/23/22 10:23 AM, Avi Kivity wrote:
>> Perhaps the interface should be kept separate from io_uring. e.g. use
>> a pidfd to represent the address space, and then issue
>> IORING_OP_PREADV/IORING_OP_PWRITEV to initiate dma. Then one can copy
>> across process boundaries.
> Then you just made it a ton less efficient, particularly if you used the
> vectored read/write. For this to make sense, I think it has to be a
> separate op. At least that's the only implementation I'd be willing to
> entertain for the immediate copy.


Sorry, I caused a lot of confusion by bundling immediate copy and a DMA 
engine interface. For sure the immediate copy should be a direct 
implementation like you posted!


User-to-user copies are another matter. I feel like that should be a 
stand-alone driver, and that io_uring should be an io_uring-y way to 
access it. Just like io_uring isn't an NVMe driver.


>> A different angle is to use expose the dma device as a separate fd.
>> This can be useful as dma engine can often do other operations, like
>> xor or crc or encryption or compression. In any case I'd argue for the
>> interface to be useful outside io_uring, although that considerably
>> increases the scope. I also don't have a direct use case for it,
>> though I'm sure others will.
> I'd say that whoever does it get to at least dictate the initial
> implementation.


Of course, but bikeshedding from the sidelines never hurt anyone.


>   For outside of io_uring, you're looking at a sync
> interface, which I think already exists for this (ioctls?).


Yes, it would be a asynchronous interface. I don't know if one exists, 
but I can't claim to have kept track.


>
>> The kernel itself should find the DMA engine useful for things like
>> memory compaction.
> That's a very different use case though and just deals with wiring it up
> internally.
>
> Let's try and keep the scope here reasonable, imho nothing good comes
> out of attempting to do all the things at once.
>

For sure, I'm just noting that the DMA engine has many different uses 
and so deserves an interface that is untied to io_uring.