Re: [PATCH v15 0/7] io_uring: add napi busy polling support

[Olivier Langlois <olivier@trillion01.com>]

On Wed, 2024-01-31 at 13:52 -0700, Jens Axboe wrote:
> On 1/31/24 12:56 PM, Olivier Langlois wrote:
> > On Wed, 2024-01-31 at 12:59 -0500, Olivier Langlois wrote:
> > > On Wed, 2024-01-31 at 10:32 -0700, Jens Axboe wrote:
> > > > 
> > > > Thanks for testing!
> > > > 
> > > > Any chance that you could run some tests with and without NAPI
> > > > that
> > > > help
> > > > validate that it actually works? That part is what I'm most
> > > > interested
> > > > in, not too worried about the stability of it as I have
> > > > scrutinized
> > > > it
> > > > pretty close already.
> > > > 
> > > 
> > > There is maybe a test that I can perform. The data that I receive
> > > is
> > > timestamped. I have a small test program that checks the age of
> > > the
> > > updates on their reception...
> > > 
> > > I would expect that it should be possible to perceive the busy
> > > polling
> > > effect by comparing the average update age with and without the
> > > feature
> > > enabled...
> > > 
> > > A word of warning... The service that my client is connecting to
> > > has
> > > relocated recently. I used to have an RTT of about 8mSec with it
> > > to
> > > about 400-500 mSec today...
> > > 
> > > because of the huge RTT, I am unsure that the test is going to be
> > > conclusive at all...
> > > 
> > > However, I am also in the process of relocating my client closer
> > > to
> > > the
> > > service. If you can wait a week or so, I should able to do that
> > > test
> > > with a RTT < 1 mSec...
> > > 
> > > Beside that, I could redo the same test that Stefan did with the
> > > ping
> > > client/server setup but would that test add any value to the
> > > current
> > > collective knowledge?
> > > 
> > > I'll do the update age test when I restart my client and I'll
> > > report
> > > back the result but my expectations aren't very high that it is
> > > going
> > > to be conclusive due to the huge RTT.
> > > 
> > > 
> > As I expected, the busy polling difference in the update age test
> > is so
> > small compared to the RTT that the result is inconclusive, IMHO...
> > 
> > The number of collected updates to build the stats is 500.
> > 
> > System clocks are assumed to be synchronized and the RTT is the
> > difference between the local time and the update timestamp.
> > Actually, it may be more accurate to say that the displayed RTT
> > values
> > are in fact TT...
> > 
> > latency NO napi busy poll:
> > [2024-01-31 11:28:34] INFO Main/processCollectedData rtt
> > min/avg/max/mdev = 74.509/76.752/115.969/3.110 ms
> > 
> > latency napi busy poll:
> > [2024-01-31 11:33:05] INFO Main/processCollectedData rtt
> > min/avg/max/mdev = 75.347/76.740/134.588/1.648 ms
> > 
> > I'll redo the test once my RTT is closer to 1mSec. The relative
> > gain
> > should be more impressive...
> 
> Also happy to try and run it here, if you can share it? If not I have
> some other stuff I can try as well, with netbench.
> 
I have redone my test with a fixed liburing lib that actually enable
io_uring NAPI busy polling correctly and I have slightly more
convincing result:

latency NO napi busy poll (kernel v7.2.3):
[2024-02-02 11:42:41] INFO Main/processCollectedData rtt
min/avg/max/mdev = 73.089/75.142/107.169/2.954 ms

latency napi busy poll (kernel v7.2.3):
[2024-02-02 11:48:18] INFO Main/processCollectedData rtt
min/avg/max/mdev = 72.862/73.878/124.536/1.288 ms

FYI, I said that I could redo the test once I relocate my client to
have a RTT < 1ms...

I might not be able to do that. I might settle for an AWS VPS instead
of a bare metal setup and when you are running the kernel on a VPS,
AFAIK, the virtual Ethernet driver does not have NAPI...