Re: [LSF/MM/BPF ATTEND][LSF/MM/BPF Topic] Non-block IO

[Bart Van Assche <bvanassche@acm.org>]

On 2/10/23 11:34, Kanchan Joshi wrote:
> On Fri, Feb 10, 2023 at 10:18:08AM -0800, Bart Van Assche wrote:
>> On 2/10/23 10:00, Kanchan Joshi wrote:
>>> 3. DMA cost: is high in presence of IOMMU. Keith posted the work[1],
>>> with block IO path, last year. I imagine plumbing to get a bit simpler
>>> with passthrough-only support. But what are the other things that must
>>> be sorted out to have progress on moving DMA cost out of the fast path?
>>
>> Are performance numbers available?
> 
> Around 55% decline when I checked last (6.1-rcX kernel).
> 512b randread IOPS with optane, on AMD ryzen 9 box -
> when iommu is set to lazy (default config)= 3.1M
> when iommmu is disabled or in passthrough mode = 4.9M

Hi Kanchan,

Thank you for having shared these numbers. More information would be 
welcome, e.g. the latency impact on a QD=1 test of the IOMMU, the queue 
depth of the test results mentioned above and also how much additional 
CPU time is needed with the IOMMU enabled. I'm wondering whether the 
IOMMU cost is dominated by the IOMMU hardware or by software bottlenecks 
(e.g. spinlocks).

Thanks,

Bart.