Questions about usage of io-uring in a network application

Questions about usage of io-uring in a network application

[Richard W.M. Jones]

Hopefully these questions are not too stupid, but I didn't find any
answers looking over the archives of this list or in the io_uring.pdf
document.

I have an existing C library which is the client part of a
client/server network service.  It's not exactly the same as HTTP, but
it could be thought of as like an HTTP network client where we may
hold a few separate TCP/IP connections open to the same server, and we
issue multiple requests in flight per TCP connection.  There is one
pthread per TCP connection usually.

I want to try to see if io_uring gives us any performance benefit by
seeing if we can use IORING_OP_SENDMSG/IORING_OP_RECVMSG +
IOSQE_IO_LINK, as an experiment.


(1) How many io_urings should we create?

One ring per process?  All of the connections share the same ring.
Presumably there would be a lot of contention because I suppose we'd
need to lock the ring while submitting requests from multiple threads.
If there are multiple independent libraries or separate of the program
all trying to use io_uring, should they try to share a single ring?

One ring per pthread?  It seems we could implement this without locks
using thread-local storage to hold the the io_uring fd.

One ring per physical CPU?  (Not sure how to implement this race-free
in userspace).

One ring per TCP connection?


(2) The existing API (which we cannot change) takes user-allocated
buffers for the data to read/write.  We don't know if these were
allocated using malloc, they might be statically allocated or even
come from something exotic like mmap of a file.  I understand that we
cannot register these buffers using IORING_REGISTER_BUFFERS.  But can
these be passed in the io_uring_sqe->addr field?  ie. Do the same
restrictions in IORING_REGISTER_BUFFERS also apply to the addr field?


Thanks,
Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v

Re: Questions about usage of io-uring in a network application

[Pavel Begunkov]

On 10/05/2020 11:00, Richard W.M. Jones wrote:
> Hopefully these questions are not too stupid, but I didn't find any
> answers looking over the archives of this list or in the io_uring.pdf
> document.
> 
> I have an existing C library which is the client part of a
> client/server network service.  It's not exactly the same as HTTP, but
> it could be thought of as like an HTTP network client where we may
> hold a few separate TCP/IP connections open to the same server, and we
> issue multiple requests in flight per TCP connection.  There is one
> pthread per TCP connection usually.
> 
> I want to try to see if io_uring gives us any performance benefit by
> seeing if we can use IORING_OP_SENDMSG/IORING_OP_RECVMSG +
> IOSQE_IO_LINK, as an experiment.
> 
> 
> (1) How many io_urings should we create?
> 
> One ring per process?  All of the connections share the same ring.
> Presumably there would be a lot of contention because I suppose we'd
> need to lock the ring while submitting requests from multiple threads.
> If there are multiple independent libraries or separate of the program
> all trying to use io_uring, should they try to share a single ring?
> 
> One ring per pthread?  It seems we could implement this without locks
> using thread-local storage to hold the the io_uring fd.
> 
> One ring per physical CPU?  (Not sure how to implement this race-free
> in userspace).
> 
> One ring per TCP connection?

It's totally up to a user. Threads can try to share io_uring, but after some
number of IOPS you indeed may find it to be a bottleneck. It rather depends on
your system/workload/etc.

There are 2 more things you may want to consider:
1. only 1 thread can do in-kernel submission (i.e. inside io_uring_enter()) at a
time. And it can take some time, because it can to execute and complete some
requests inline (e.g. doing copying buffers, etc.).

2. there is an option to share internal thread pools between io_uring instances.
If you have multiple io_uring instances, it's a good start point to share a
single thread pool between them. see IORING_SETUP_ATTACH_WQ.

To summarise, for a really high-throughput system, I'd incline using the
per-process scheme with a single shared thread pool, and adjust then. As for
clients, and it's depends on a kind of a client you have, but an average one
would probably not bottleneck there.

I would love to see more comments from guys really using it. E.g. see
https://lore.kernel.org/io-uring/7692E70C-A0EA-423B-883F-6BF91B0DB359@icloud.com/


> 
> (2) The existing API (which we cannot change) takes user-allocated
> buffers for the data to read/write.  We don't know if these were
> allocated using malloc, they might be statically allocated or even
> come from something exotic like mmap of a file.  I understand that we
> cannot register these buffers using IORING_REGISTER_BUFFERS.  But can
> these be passed in the io_uring_sqe->addr field?  ie. Do the same
> restrictions in IORING_REGISTER_BUFFERS also apply to the addr field?

It should work as any other read(2)/write(2)/etc. A simple test with
file-backed buffers works for me.

-- 
Pavel Begunkov