public inbox for [email protected]
 help / color / mirror / Atom feed
From: Jonathan Lemon <[email protected]>
To: <[email protected]>
Cc: <[email protected]>
Subject: [RFC PATCH v2 00/13] zero-copy RX for io_uring
Date: Tue, 18 Oct 2022 12:15:49 -0700	[thread overview]
Message-ID: <[email protected]> (raw)

This series is a RFC for io_uring/zctap.  This is an evolution of
the earlier zctap work, re-targeted to use io_uring as the userspace
API.  The current code is intended to provide a zero-copy RX path for
upper-level networking protocols (aka TCP and UDP).  The current draft
focuses on host-provided memory (not GPU memory).

This RFC contains the upper-level core code required for operation,
with the intent of soliciting feedback on the general API.  This does
not contain the network driver side changes required for complete
operation.  Also please note that as an RFC, there are some things
which are incomplete or in need of rework.

The intent is to use a network driver which provides header/data
splitting, so the frame header (which is processed by the networking
stack) does not reside in user memory.

The code is successfully receiving a zero-copy TCP stream from a
remote sender.  An RFC, the intent is to solicit feedback on the
API and overall design.  The current code will also work with
system pages, copying the data out to the application - this is
intended as a fallback/testing path.

Performance numbers are coming soon!

High level description:

The application allocates a frame backing store, and provides this
to the kernel for use.  An interface queue is requested from the
networking device, and incoming frames are deposited into the provided
memory region.

Responsibility for correctly steering incoming frames to the queue
is outside the scope of this work - it is assumed that the user 
has set steering rules up separately.

Incoming frames are sent up the stack as skb's and eventually
land in the application's socket receive queue.  This differs
from AF_XDP, which receives raw frames directly to userspace,
without protocol processing.

The RECV_ZC opcode then returns an iov[] style vector which points
to the data in userspace memory.  When the application has completed
processing of the data, the buffer is returned back to the kernel
through a fill ring for reuse.

Changelog:
 v1: initial version
 v2: Remove separate PROVIDE_REGION opcode, fold this functionality
     into REGISTER_IFQ.  Remove page_pool hooks, as it appears the 
     page pool is currently incompatible with user-mapped memory.
     Add io_zctap_buffers and network driver API.
.

Jonathan Lemon (13):
  io_uring: add zctap ifq definition
  netdevice: add SETUP_ZCTAP to the netdev_bpf structure
  io_uring: add register ifq opcode
  io_uring: create a zctap region for a mapped buffer
  io_uring: create page freelist for the ifq region
  io_uring: Provide driver API for zctap packet buffers.
  io_uring: Allocate the zctap buffers for the device
  io_uring: Add zctap buffer get/put functions and refcounting.
  skbuff: Introduce SKBFL_FIXED_FRAG and skb_fixed()
  io_uring: Allocate a uarg for use by the ifq RX
  io_uring: Define the zctap iov[] returned to the user.
  io_uring: add OP_RECV_ZC command.
  io_uring: Make remove_ifq_region a delayed work call

 include/linux/io_uring.h       |  35 ++
 include/linux/io_uring_types.h |  11 +
 include/linux/netdevice.h      |   6 +
 include/linux/skbuff.h         |  10 +-
 include/uapi/linux/io_uring.h  |  24 +
 io_uring/Makefile              |   3 +-
 io_uring/io_uring.c            |   8 +
 io_uring/kbuf.c                |  13 +
 io_uring/kbuf.h                |   2 +
 io_uring/net.c                 | 123 +++++
 io_uring/opdef.c               |  15 +
 io_uring/zctap.c               | 842 +++++++++++++++++++++++++++++++++
 io_uring/zctap.h               |  16 +
 13 files changed, 1106 insertions(+), 2 deletions(-)
 create mode 100644 io_uring/zctap.c
 create mode 100644 io_uring/zctap.h

-- 
2.30.2


             reply	other threads:[~2022-10-18 19:16 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-10-18 19:15 Jonathan Lemon [this message]
2022-10-18 19:15 ` [RFC PATCH v2 01/13] io_uring: add zctap ifq definition Jonathan Lemon
2022-10-18 19:15 ` [RFC PATCH v2 02/13] netdevice: add SETUP_ZCTAP to the netdev_bpf structure Jonathan Lemon
2022-10-18 19:15 ` [RFC PATCH v2 03/13] io_uring: add register ifq opcode Jonathan Lemon
2022-10-18 19:15 ` [RFC PATCH v2 04/13] io_uring: create a zctap region for a mapped buffer Jonathan Lemon
2022-10-18 19:15 ` [RFC PATCH v2 05/13] io_uring: create page freelist for the ifq region Jonathan Lemon
2022-10-18 19:15 ` [RFC PATCH v2 06/13] io_uring: Provide driver API for zctap packet buffers Jonathan Lemon
2022-10-18 19:15 ` [RFC PATCH v2 07/13] io_uring: Allocate the zctap buffers for the device Jonathan Lemon
2022-10-18 19:15 ` [RFC PATCH v2 08/13] io_uring: Add zctap buffer get/put functions and refcounting Jonathan Lemon
2022-10-18 19:15 ` [RFC PATCH v2 09/13] skbuff: Introduce SKBFL_FIXED_FRAG and skb_fixed() Jonathan Lemon
2022-10-18 19:15 ` [RFC PATCH v2 10/13] io_uring: Allocate a uarg for use by the ifq RX Jonathan Lemon
2022-10-18 19:16 ` [RFC PATCH v2 11/13] io_uring: Define the zctap iov[] returned to the user Jonathan Lemon
2022-10-18 19:16 ` [RFC PATCH v2 12/13] io_uring: add OP_RECV_ZC command Jonathan Lemon
2022-10-18 19:16 ` [RFC PATCH v2 13/13] io_uring: Make remove_ifq_region a delayed work call Jonathan Lemon
2022-10-20  3:35 ` [RFC PATCH v2 00/13] zero-copy RX for io_uring Ziyang Zhang
2022-11-02 23:33   ` Jonathan Lemon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox