Re: [PATCH v15 0/7] io_uring: add napi busy polling support

[Olivier Langlois <olivier@trillion01.com>]

On Tue, 2024-01-30 at 15:59 -0700, Jens Axboe wrote:
> On 1/30/24 2:20 PM, Olivier Langlois wrote:
> > Hi,
> > 
> > I was wondering what did happen to this patch submission...
> > 
> > It seems like Stefan did put a lot of effort in addressing every
> > reported issue for several weeks/months...
> > 
> > and then nothing... as if this patch has never been reviewed by
> > anyone...
> > 
> > has it been decided to not integrate NAPI busy looping in io_uring
> > privately finally?
> 
> It's really just waiting for testing, I want to ensure it's working
> as
> we want it to before committing. But the production bits I wanted to
> test on have been dragging out, hence I have not made any moves
> towards
> merging this for upstream just yet.
> 
> FWIW, I have been maintaining the patchset, you can find the current
> series here:
> 
> https://git.kernel.dk/cgit/linux/log/?h=io_uring-napi
> 

test setup:
-----------
- kernel 6.7.2 with Jens patchset applied (It did almost work as-is
except for modifs in io_uring/register.c that was in
io_uring/io_uring.c in 6.7.2)
- liburing 2.5 patched with Stefan patch after having carefully make
sure that IORING_REGISTER_NAPI,IORING_UNREGISTER_NAPI values match the
ones found in the kernel. (It was originally 26,27 and it is now 27,28)
- 3 threads each having their own private io_uring ring.

thread 1:
- use SQ_POLL kernel thread
- reads data stream from 15-20 TCP connections
- enable NAPI busy polling by calling io_uring_register_napi()

[2024-01-31 08:59:55] INFO WSBASE/client_established 1010
LWS_CALLBACK_CLIENT_ESTABLISHED client 3(fd 43), napi_id:31
[2024-01-31 08:59:55] INFO WSBASE/client_established 1010
LWS_CALLBACK_CLIENT_ESTABLISHED client 8(fd 38), napi_id:30
[2024-01-31 08:59:56] INFO WSBASE/client_established 1010
LWS_CALLBACK_CLIENT_ESTABLISHED client 10(fd 36), napi_id:25
[2024-01-31 08:59:56] INFO WSBASE/client_established 1010
LWS_CALLBACK_CLIENT_ESTABLISHED client 14(fd 32), napi_id:25
[2024-01-31 08:59:56] INFO WSBASE/client_established 1010
LWS_CALLBACK_CLIENT_ESTABLISHED client 12(fd 34), napi_id:28
[2024-01-31 08:59:56] INFO WSBASE/client_established 1010
LWS_CALLBACK_CLIENT_ESTABLISHED client 2(fd 44), napi_id:31
[2024-01-31 08:59:56] INFO WSBASE/client_established 1010
LWS_CALLBACK_CLIENT_ESTABLISHED client 16(fd 30), napi_id:31
[2024-01-31 08:59:56] INFO WSBASE/client_established 1010
LWS_CALLBACK_CLIENT_ESTABLISHED client 9(fd 37), napi_id:31
[2024-01-31 08:59:56] INFO WSBASE/client_established 1010
LWS_CALLBACK_CLIENT_ESTABLISHED client 20(fd 26), napi_id:31
[2024-01-31 08:59:56] INFO WSBASE/client_established 1010
LWS_CALLBACK_CLIENT_ESTABLISHED client 1(fd 45), napi_id:30
[2024-01-31 08:59:56] INFO WSBASE/client_established 1010
LWS_CALLBACK_CLIENT_ESTABLISHED client 6(fd 40), napi_id:28
[2024-01-31 08:59:56] INFO WSBASE/client_established 1010
LWS_CALLBACK_CLIENT_ESTABLISHED client 13(fd 33), napi_id:25
[2024-01-31 08:59:56] INFO WSBASE/client_established 1010
LWS_CALLBACK_CLIENT_ESTABLISHED client 22(fd 22), napi_id:25
[2024-01-31 08:59:56] INFO WSBASE/client_established 1010
LWS_CALLBACK_CLIENT_ESTABLISHED client 7(fd 39), napi_id:30
[2024-01-31 08:59:56] INFO WSBASE/client_established 1010
LWS_CALLBACK_CLIENT_ESTABLISHED client 18(fd 28), napi_id:28
[2024-01-31 08:59:56] INFO WSBASE/client_established 1010
LWS_CALLBACK_CLIENT_ESTABLISHED client 19(fd 27), napi_id:25
[2024-01-31 08:59:56] INFO WSBASE/client_established 1028
LWS_CALLBACK_CLIENT_ESTABLISHED client 23(fd 21), napi_id:31
[2024-01-31 08:59:56] INFO WSBASE/client_established 1010
LWS_CALLBACK_CLIENT_ESTABLISHED client 4(fd 42), napi_id:31
[2024-01-31 08:59:56] INFO WSBASE/client_established 1010
LWS_CALLBACK_CLIENT_ESTABLISHED client 5(fd 41), napi_id:25
[2024-01-31 08:59:56] INFO WSBASE/client_established 1010
LWS_CALLBACK_CLIENT_ESTABLISHED client 21(fd 24), napi_id:31
[2024-01-31 08:59:56] INFO WSBASE/client_established 1010
LWS_CALLBACK_CLIENT_ESTABLISHED client 17(fd 29), napi_id:30
[2024-01-31 08:59:56] INFO WSBASE/client_established 1010
LWS_CALLBACK_CLIENT_ESTABLISHED client 15(fd 31), napi_id:28
[2024-01-31 08:59:57] INFO WSBASE/client_established 1010
LWS_CALLBACK_CLIENT_ESTABLISHED client 11(fd 35), napi_id:30
[2024-01-31 09:00:14] INFO WSBASE/client_established 1031
LWS_CALLBACK_CLIENT_ESTABLISHED client 24(fd 25), napi_id:30

thread 2:
- No SQ_POLL
- reads data stream from 1 TCP socket
- enable NAPI busy polling by calling io_uring_register_napi()

[2024-01-31 09:01:45] INFO WSBASE/client_established 1031
LWS_CALLBACK_CLIENT_ESTABLISHED client 25(fd 23), napi_id:31

thread 3:
- No SQ_POLL
- No NAPI busy polling
- read data stream from 1 TCP socket

Outcome:
--------

I did not measure latency to make sure that NAPI polling was effective
but I did ensure the stability of running the patchset by letting the
program run for 5+ hours non stop without experiencing any glitches

Tested-by: Olivier Langlois <olivier@trillion01.com>