Re: [PATCHv10 0/9] write hints with nvme fdp, scsi streams

["Martin K. Petersen" <martin.petersen@oracle.com>]

Bart,

> Submitting a copy operation as two bios or two requests means that
> there is a risk that one of the two operations never reaches the block
> driver at the bottom of the storage stack and hence that a deadlock
> occurs. I prefer not to introduce any mechanisms that can cause a
> deadlock.

How do you copy a block range without offload? You perform a READ to
read the data into memory. And once the READ completes, you do a WRITE
of the data to the new location.

Token-based copy offload works exactly the same way. You do a POPULATE
TOKEN which is identical to a READ except you get a cookie instead of
the actual data. And then once you have the cookie, you perform a WRITE
USING TOKEN to perform the write operation. Semantically, it's exactly
the same as a normal copy except for the lack of data movement. That's
the whole point!

Once I had support for token-based copy offload working, it became clear
to me that this approach is much simpler than pointer matching, bio
pairs, etc. The REQ_OP_COPY_IN operation and the REQ_OP_COPY_OUT
operation are never in flight at the same time. There are no
synchronization hassles, no lifetimes, no lookup tables in the sd
driver, no nonsense. Semantically, it's a read followed by a write.

For devices that implement single-command copy offload, the
REQ_OP_COPY_IN operation only serves as a validation that no splitting
took place. Once the bio reaches the ULD, the I/O is completed without
ever sending a command to the device. blk-lib then issues a
REQ_OP_COPY_OUT which gets turned into EXTENDED COPY or NVMe Copy and
sent to the destination device.

Aside from making things trivially simple, the COPY_IN/COPY_OUT semantic
is a *requirement* for token-based offload devices. Why would we even
consider having two incompatible sets of copy offload semantics coexist
in the block layer?

-- 
Martin K. Petersen	Oracle Linux Engineering