Re: io_uring zero-copy send test results

[Pavel Begunkov <asml.silence@gmail.com>]

On 4/6/25 22:08, vitalif@yourcmc.ru wrote:
> Hi again!
> 
> More interesting data for you. :-)
> 
> We tried iommu=pt.

What kernel version you use? I'm specifically interested whether it has:

6fe4220912d19 ("io_uring/notif: implement notification stacking")

That would explain why it's slow even with huge pages.

...
> Xeon Gold 6342 + Mellanox ConnectX-6 Dx
> 
>             4096  8192   10000  12000  16384  32768  65435
> zc MB/s    2060  2950   2927   2934   2945   2945   2947
> zc CPU     99%   62%    59%    29%    22%    23%    11%
> send MB/s  2950  2949   2950   2950   2949   2949   2949
> send CPU   64%   44%    50%    46%    51%    49%    45%
> 
> Xeon Gold 6342 + Mellanox ConnectX-6 Dx + iommu=pt
> 
>             4096  8192   10000  12000  16384  32768  65435
> zc MB/s    2165  2277   2790   2802   2871   2945   2944
> zc CPU     99%   89%    75%    65%    53%    34%    36%
> send MB/s  2902  2912   2945   2943   2927   2935   2941
> send CPU   80%   63%    55%    64%    78%    68%    65%
> 
> Here, disabling iommu actually makes things worse - CPU usage increases in all tests. The default mode is optimal.

That doesn't make sense. Do you see anything odd in the profile?

-- 
Pavel Begunkov