Re: [PATCH v15 0/7] io_uring: add napi busy polling support

[Jens Axboe <axboe@kernel.dk>]

On 2/2/24 1:20 PM, Olivier Langlois wrote:
> On Wed, 2024-01-31 at 13:52 -0700, Jens Axboe wrote:
>> On 1/31/24 12:56 PM, Olivier Langlois wrote:
>>> On Wed, 2024-01-31 at 12:59 -0500, Olivier Langlois wrote:
>>>> On Wed, 2024-01-31 at 10:32 -0700, Jens Axboe wrote:
>>>>>
>>>>> Thanks for testing!
>>>>>
>>>>> Any chance that you could run some tests with and without NAPI
>>>>> that
>>>>> help
>>>>> validate that it actually works? That part is what I'm most
>>>>> interested
>>>>> in, not too worried about the stability of it as I have
>>>>> scrutinized
>>>>> it
>>>>> pretty close already.
>>>>>
>>>>
>>>> There is maybe a test that I can perform. The data that I receive
>>>> is
>>>> timestamped. I have a small test program that checks the age of
>>>> the
>>>> updates on their reception...
>>>>
>>>> I would expect that it should be possible to perceive the busy
>>>> polling
>>>> effect by comparing the average update age with and without the
>>>> feature
>>>> enabled...
>>>>
>>>> A word of warning... The service that my client is connecting to
>>>> has
>>>> relocated recently. I used to have an RTT of about 8mSec with it
>>>> to
>>>> about 400-500 mSec today...
>>>>
>>>> because of the huge RTT, I am unsure that the test is going to be
>>>> conclusive at all...
>>>>
>>>> However, I am also in the process of relocating my client closer
>>>> to
>>>> the
>>>> service. If you can wait a week or so, I should able to do that
>>>> test
>>>> with a RTT < 1 mSec...
>>>>
>>>> Beside that, I could redo the same test that Stefan did with the
>>>> ping
>>>> client/server setup but would that test add any value to the
>>>> current
>>>> collective knowledge?
>>>>
>>>> I'll do the update age test when I restart my client and I'll
>>>> report
>>>> back the result but my expectations aren't very high that it is
>>>> going
>>>> to be conclusive due to the huge RTT.
>>>>
>>>>
>>> As I expected, the busy polling difference in the update age test
>>> is so
>>> small compared to the RTT that the result is inconclusive, IMHO...
>>>
>>> The number of collected updates to build the stats is 500.
>>>
>>> System clocks are assumed to be synchronized and the RTT is the
>>> difference between the local time and the update timestamp.
>>> Actually, it may be more accurate to say that the displayed RTT
>>> values
>>> are in fact TT...
>>>
>>> latency NO napi busy poll:
>>> [2024-01-31 11:28:34] INFO Main/processCollectedData rtt
>>> min/avg/max/mdev = 74.509/76.752/115.969/3.110 ms
>>>
>>> latency napi busy poll:
>>> [2024-01-31 11:33:05] INFO Main/processCollectedData rtt
>>> min/avg/max/mdev = 75.347/76.740/134.588/1.648 ms
>>>
>>> I'll redo the test once my RTT is closer to 1mSec. The relative
>>> gain
>>> should be more impressive...
>>
>> Also happy to try and run it here, if you can share it? If not I have
>> some other stuff I can try as well, with netbench.
>>
> I have redone my test with a fixed liburing lib that actually enable
> io_uring NAPI busy polling correctly and I have slightly more
> convincing result:
> 
> latency NO napi busy poll (kernel v7.2.3):
> [2024-02-02 11:42:41] INFO Main/processCollectedData rtt
> min/avg/max/mdev = 73.089/75.142/107.169/2.954 ms
> 
> latency napi busy poll (kernel v7.2.3):
> [2024-02-02 11:48:18] INFO Main/processCollectedData rtt
> min/avg/max/mdev = 72.862/73.878/124.536/1.288 ms
> 
> FYI, I said that I could redo the test once I relocate my client to
> have a RTT < 1ms...
> 
> I might not be able to do that. I might settle for an AWS VPS instead
> of a bare metal setup and when you are running the kernel on a VPS,
> AFAIK, the virtual Ethernet driver does not have NAPI...

I'm going to try some local 10g testing here, I don't think the above
says a whole lot as we're dealing with tens of msecs here. But if
significant and stable, does look like an improvement. If NAPI works how
it should, then sub msec ping/pong replies should show a significant
improvement. I'll report back once I get to it...

-- 
Jens Axboe