public inbox for [email protected]
 help / color / mirror / Atom feed
From: Jens Axboe <[email protected]>
To: Pavel Begunkov <[email protected]>,
	[email protected], [email protected]
Cc: "David S . Miller" <[email protected]>,
	Jakub Kicinski <[email protected]>,
	David Ahern <[email protected]>,
	Eric Dumazet <[email protected]>,
	Willem de Bruijn <[email protected]>
Subject: Re: [RFC 0/6] implement io_uring notification (ubuf_info) stacking
Date: Fri, 12 Apr 2024 07:44:36 -0600	[thread overview]
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>

On 4/12/24 6:55 AM, Pavel Begunkov wrote:
> io_uring allocates a ubuf_info per zerocopy send request, it's convenient
> for the userspace but with how things are it means that every time the 
> TCP stack has to allocate a new skb instead of amending into a previous
> one. Unless sends are big enough, there will be lots of small skbs
> straining the stack and dipping performance.
> 
> The patchset implements notification, i.e. an io_uring's ubuf_info
> extension, stacking. It tries to link ubuf_info's into a list, and
> the entire link will be put down together once all references are
> gone.

Excellent! I'll take a closer look, but I ran a quick test with my test
tool just to see the difference. This is on a 100G link.

Packet size	Before (Mbit)    After (Mbit)   Diff
====================================================
100		290		  1250		4.3x
200		560		  2460		4.4x
400		1190		  4900		4.1x
800		2300		  9700		4.2x
1600		4500		 19100		4.2x
3200		8900		 35000		3.9x

which are just rough numbers and the tool isn't that great, but
definitely encouraging. And it does have parity with sync MSG_ZEROPCY,
which is what I was really bugged about before.

-- 
Jens Axboe


  parent reply	other threads:[~2024-04-12 13:44 UTC|newest]

Thread overview: 25+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-04-12 12:55 [RFC 0/6] implement io_uring notification (ubuf_info) stacking Pavel Begunkov
2024-04-12 12:55 ` [RFC 1/6] net: extend ubuf_info callback to ops structure Pavel Begunkov
2024-04-13 17:17   ` David Ahern
2024-04-14 17:07   ` Willem de Bruijn
2024-04-15  0:07     ` Pavel Begunkov
2024-04-15 15:06       ` Willem de Bruijn
2024-04-15 18:55         ` Pavel Begunkov
2024-04-15 19:01           ` Willem de Bruijn
2024-04-16 14:50       ` David Ahern
2024-04-16 15:31         ` Pavel Begunkov
2024-04-12 12:55 ` [RFC 2/6] net: add callback for setting a ubuf_info to skb Pavel Begunkov
2024-04-13 17:18   ` David Ahern
2024-04-12 12:55 ` [RFC 3/6] io_uring/notif: refactor io_tx_ubuf_complete() Pavel Begunkov
2024-04-12 12:55 ` [RFC 4/6] io_uring/notif: remove ctx var from io_notif_tw_complete Pavel Begunkov
2024-04-12 12:55 ` [RFC 5/6] io_uring/notif: simplify io_notif_flush() Pavel Begunkov
2024-04-12 12:55 ` [RFC 6/6] io_uring/notif: implement notification stacking Pavel Begunkov
2024-04-14 17:10   ` Willem de Bruijn
2024-04-14 23:55     ` Pavel Begunkov
2024-04-15 15:15       ` Willem de Bruijn
2024-04-15 18:51         ` Pavel Begunkov
2024-04-15 19:02           ` Willem de Bruijn
2024-04-12 13:44 ` Jens Axboe [this message]
2024-04-12 14:52 ` [RFC 0/6] implement io_uring notification (ubuf_info) stacking Jens Axboe
2024-04-13 17:17 ` David Ahern
2024-04-15  0:08   ` Pavel Begunkov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox