dma_buf support with io_uring

["Fang, Wilson" <wilson.fang@intel.com>]

Hi Jens,

We are exploring a kernel native mechanism to support peer to peer data transfer between a NVMe SSD and another device supporting dma_buf, connected on the same PCIe root complex.
NVMe SSD DMA engine requires physical memory address and there is no easy way to pass non system memory address through VFS to the block device driver.
One of the ideas is to use the io_uring and dma_buf mechanism which is supported by the peer device of the SSD.

The flow is as below:
1. Application passes the dma_buf fd to the kernel through liburing.
2. Io_uring adds two new options IORING_OP_READ_DMA and IORING_OP_WRITE_DMA to support read write operations that DMA to/from the peer device memory.
3. If the dma_buf fd is valid, io_uring attaches dma_buf and get sgl which contains physical memory addresses to be passed down to the block device driver.
4. NVMe SSD DMA engine DMA the data to/from the physical memory address.

The road blocker we are facing is that dma_buf_attach() and dma_buf_map_attachment() APIs expects the caller to provide the struct device *dev as input parameter pointing to the device which does the DMA (in this case the block/NVMe device that holds the source data). 
But since io_uring operates at the VFS layer there is no straight forward way of finding the block/NVMe device object (struct device*) from the source file descriptor.

Do you have any recommendations? Much appreciated!

Thanks,
Wilson