From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wr1-f46.google.com (mail-wr1-f46.google.com [209.85.221.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D6FCB288C81 for ; Thu, 26 Feb 2026 12:48:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.46 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772110132; cv=none; b=gZ31uS4CfNBZbbwBdIQJXHL4X6mqeSTh02zUt+NicM18nMGq0tCgz3qix1ZHLutGVJrycfYyBcoILFMAbCLo2mjWHFTLL1yusasZo7NnAYwqNW5XsuuW51S+SljteOVQACT7wpHrFpmqLAfnaYEdqsiGi25E60/ho7q5hUa3EYE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772110132; c=relaxed/simple; bh=zRoD5mOyMdod4JrnF+t4+ynfDUgxUc2HrKn2wC8CieE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=STVIK54zq5s1MxdsStUOuNKVu9uYCKetHDwq/xHRLshODgGdy55RUeevHHbB1DKtYAZFFwPAyjTxEAGN+5WZT4II7xxU2y2z5hLGP34ci/IjsX5cjpbvGjpWtWYvb+QDaEQr29eORdE5+tHfICX4L9UA7gPAbYF7WHtVYOAxqQ4= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=l8pB0wE3; arc=none smtp.client-ip=209.85.221.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="l8pB0wE3" Received: by mail-wr1-f46.google.com with SMTP id ffacd0b85a97d-4398d9a12c6so636849f8f.2 for ; Thu, 26 Feb 2026 04:48:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1772110129; x=1772714929; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=C6cyZmo5lFFgJUyRDm0eIF4T/QbcI6+i0uyHVRh10LE=; b=l8pB0wE3W2E/PJFGNVZGqEcovB9MYA8om+o5KhylcuWU6D/efcPXdq64CQCbKMhCOD vJDrCvn6egdqNGIrrwwtuwOTwS06qpJZONX9CeQzp8MjzXcRUciYxrNPtDRPldltns8Q yBHiLM5PySB6/2g07iW9HAIooSl8m0TuYHaTYgazdTnz4+3xdcgm37mWKKdINCBwf2Zj lDIbHle3kEHkLsDagwaVeiFI2sKD7WZgoRqp/R+HV9m+SZA+wtiTV+wlL3PRMXTP2D6n 5QpOashAFJ1v+TReHzT9/q2W83qeR7xLVmrXV9OG7Pk3G80TwXxL7Yx3JoJAE1j6ZVKn 3ycA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772110129; x=1772714929; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=C6cyZmo5lFFgJUyRDm0eIF4T/QbcI6+i0uyHVRh10LE=; b=ClWJ0SzskE1+pbpUm5Ml4/t4u9tYV09AShMIXpcMkR9n5tIowGXrSm+vUowetd18bc ZjmD51pmLsB1BFPEPXasu+WFG2X4g0S86rCMc7aQmtckJlu+FleweY8GzQV0LHjm934v 4WZ+EWEMdllAiV51Xg5UzFh2rfOLfTc3w7BKT1Clue6EWvBlwCpY67APgkDHfIhlKYTo lgWD0msVPoM52C7zlL9fJAR/8cOF5591EUqmccV25csgtyg1ORhIj9AL8NIjNjw+18xI a1+oKJrh1iuKnda4xYOxtNApgWC+4wIJSBj6eRQitRiyvHOKiphLdx1wx347kCaBdZA/ ui+w== X-Gm-Message-State: AOJu0YyJisOqNWO0AwQ5Di9Us92FxbfjzDLJvuAlZuffwEyZIaYuNMLG wihU85bd8Qnb0ujuTxyOG6GRS3W8bcjXppmQq1NU33IyBULc2NOErsERMMetlw== X-Gm-Gg: ATEYQzwkWdhrHnGYpruXHApGO25HEK5xaqpciqHd/rd1aFzXVbFvz0G5qWcF17kphdy pNL+LhaQqW6x130hKkOVLMxuQMhkMpCQLxPapjYrbgpROg/txQJss/q3Nx5sy2z/Za/5NjkZGn3 yKDYUKt1hwKSPZuSD8TTs5m1Y7OPSL68vdjB+P/KIPYz4QCmNaNuJyRTb+4zeHtMLil4mzKQD8t ySSkgTw2V/FjRm1ySEo+uuRQFzXMR1E5AeDd/trsMD97G94/rSBw5b9y55k3EA40bkO3oIylhcX oM+8gMMneKUeNqSqyi+fAH19XE3n5G7N9/gs4ChFMb36uIo61IyY5uBZJUp77QggA/v8snho4QT krvEB8MvejX14qcj6g+R7MhSsL9alTHCu7piqqzLplbtIRttn/LXmTTYEniX5gbIFcYpBeIqDnM Zx976hIb6bPfWkoU07MPdvK7ezZjNLZO8CGkbpAXb6w5DJLjCE8aY2f3HTA2aVOK7s5mB1WzUq/ KRKbUaCTxSApg== X-Received: by 2002:adf:f64a:0:b0:439:858e:6e34 with SMTP id ffacd0b85a97d-439858e7101mr13498593f8f.20.1772110128753; Thu, 26 Feb 2026 04:48:48 -0800 (PST) Received: from 127.com ([2620:10d:c092:600::1:2ab0]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-43970d4c977sm43734576f8f.32.2026.02.26.04.48.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Feb 2026 04:48:48 -0800 (PST) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: asml.silence@gmail.com, bpf@vger.kernel.org, axboe@kernel.dk, Alexei Starovoitov Subject: [PATCH v10 2/4] io_uring/bpf-ops: implement loop_step with BPF struct_ops Date: Thu, 26 Feb 2026 12:48:39 +0000 Message-ID: <98db437651ce64e9cbeb611c60bf5887259db09f.1772109579.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Introduce io_uring BPF struct ops implementing the loop_step callback, which will allow BPF to overwrite the default io_uring event loop logic. The callback takes an io_uring context, the main role of which is to be passed to io_uring kfuncs. The other argument is a struct iou_loop_params, which BPF can use to request CQ waiting and communicate other parameters. See the event loop description in the previous patch for more details. Signed-off-by: Pavel Begunkov --- io_uring/Kconfig | 5 ++ io_uring/Makefile | 1 + io_uring/bpf-ops.c | 127 ++++++++++++++++++++++++++++++++++++++++++++ io_uring/bpf-ops.h | 14 +++++ io_uring/io_uring.c | 1 + 5 files changed, 148 insertions(+) create mode 100644 io_uring/bpf-ops.c create mode 100644 io_uring/bpf-ops.h diff --git a/io_uring/Kconfig b/io_uring/Kconfig index a7ae23cf1035..a283d9e53787 100644 --- a/io_uring/Kconfig +++ b/io_uring/Kconfig @@ -14,3 +14,8 @@ config IO_URING_BPF def_bool y depends on BPF depends on NET + +config IO_URING_BPF_OPS + def_bool y + depends on IO_URING + depends on BPF_SYSCALL && BPF_JIT && DEBUG_INFO_BTF diff --git a/io_uring/Makefile b/io_uring/Makefile index 1c1f47de32a4..c54e328d1410 100644 --- a/io_uring/Makefile +++ b/io_uring/Makefile @@ -25,3 +25,4 @@ obj-$(CONFIG_NET) += net.o cmd_net.o obj-$(CONFIG_PROC_FS) += fdinfo.o obj-$(CONFIG_IO_URING_MOCK_FILE) += mock_file.o obj-$(CONFIG_IO_URING_BPF) += bpf_filter.o +obj-$(CONFIG_IO_URING_BPF_OPS) += bpf-ops.o diff --git a/io_uring/bpf-ops.c b/io_uring/bpf-ops.c new file mode 100644 index 000000000000..975db5a78188 --- /dev/null +++ b/io_uring/bpf-ops.c @@ -0,0 +1,127 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#include +#include +#include + +#include "io_uring.h" +#include "register.h" +#include "bpf-ops.h" +#include "loop.h" + +static const struct btf_type *loop_params_type; + +static int io_bpf_ops__loop_step(struct io_ring_ctx *ctx, + struct iou_loop_params *lp) +{ + return IOU_LOOP_STOP; +} + +static struct io_uring_bpf_ops io_bpf_ops_stubs = { + .loop_step = io_bpf_ops__loop_step, +}; + +static bool bpf_io_is_valid_access(int off, int size, + enum bpf_access_type type, + const struct bpf_prog *prog, + struct bpf_insn_access_aux *info) +{ + if (type != BPF_READ) + return false; + if (off < 0 || off >= sizeof(__u64) * MAX_BPF_FUNC_ARGS) + return false; + if (off % size != 0) + return false; + + return btf_ctx_access(off, size, type, prog, info); +} + +static int bpf_io_btf_struct_access(struct bpf_verifier_log *log, + const struct bpf_reg_state *reg, int off, + int size) +{ + const struct btf_type *t = btf_type_by_id(reg->btf, reg->btf_id); + + if (t == loop_params_type) { + if (off + size <= offsetofend(struct iou_loop_params, cq_wait_idx)) + return SCALAR_VALUE; + } + + return -EACCES; +} + +static const struct bpf_verifier_ops bpf_io_verifier_ops = { + .get_func_proto = bpf_base_func_proto, + .is_valid_access = bpf_io_is_valid_access, + .btf_struct_access = bpf_io_btf_struct_access, +}; + +static const struct btf_type * +io_lookup_struct_type(struct btf *btf, const char *name) +{ + s32 type_id; + + type_id = btf_find_by_name_kind(btf, name, BTF_KIND_STRUCT); + if (type_id < 0) + return NULL; + return btf_type_by_id(btf, type_id); +} + +static int bpf_io_init(struct btf *btf) +{ + loop_params_type = io_lookup_struct_type(btf, "iou_loop_params"); + if (!loop_params_type) { + pr_err("io_uring: Failed to locate iou_loop_params\n"); + return -EINVAL; + } + + return 0; +} + +static int bpf_io_check_member(const struct btf_type *t, + const struct btf_member *member, + const struct bpf_prog *prog) +{ + return 0; +} + +static int bpf_io_init_member(const struct btf_type *t, + const struct btf_member *member, + void *kdata, const void *udata) +{ + return 0; +} + +static int bpf_io_reg(void *kdata, struct bpf_link *link) +{ + return -EOPNOTSUPP; +} + +static void bpf_io_unreg(void *kdata, struct bpf_link *link) +{ +} + +static struct bpf_struct_ops bpf_ring_ops = { + .verifier_ops = &bpf_io_verifier_ops, + .reg = bpf_io_reg, + .unreg = bpf_io_unreg, + .check_member = bpf_io_check_member, + .init_member = bpf_io_init_member, + .init = bpf_io_init, + .cfi_stubs = &io_bpf_ops_stubs, + .name = "io_uring_bpf_ops", + .owner = THIS_MODULE, +}; + +static int __init io_uring_bpf_init(void) +{ + int ret; + + ret = register_bpf_struct_ops(&bpf_ring_ops, io_uring_bpf_ops); + if (ret) { + pr_err("io_uring: Failed to register struct_ops (%d)\n", ret); + return ret; + } + + return 0; +} +__initcall(io_uring_bpf_init); diff --git a/io_uring/bpf-ops.h b/io_uring/bpf-ops.h new file mode 100644 index 000000000000..e8a08ae2df0a --- /dev/null +++ b/io_uring/bpf-ops.h @@ -0,0 +1,14 @@ +// SPDX-License-Identifier: GPL-2.0 +#ifndef IOU_BPF_OPS_H +#define IOU_BPF_OPS_H + +#include + +struct io_uring_bpf_ops { + int (*loop_step)(struct io_ring_ctx *ctx, struct iou_loop_params *lp); + + __u32 ring_fd; + void *priv; +}; + +#endif /* IOU_BPF_OPS_H */ diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index 0c8bb4e8480a..548ea5a080a0 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -87,6 +87,7 @@ #include "msg_ring.h" #include "memmap.h" #include "zcrx.h" +#include "bpf-ops.h" #include "timeout.h" #include "poll.h" -- 2.53.0