[REGRESSION] io_uring/futex: scalar wait/wake slowdown after 079afb081c42

[REGRESSION] io_uring/futex: scalar wait/wake slowdown after 079afb081c42

[Chengfeng Lin]

Hi Jens,

I tested 079afb081c42 against its direct parent on bare metal. In a narrow
scalar io_uring futex wait/wake workload, the child was 9.27% slower. A
separate 338-line standalone reproduced the result at 10.02%. All compared
kernels actually ran with preempt=full.

#regzbot introduced: 079afb081c4288e94d5e4223d3eb6306d853c68b
#regzbot title: io_uring scalar futex wait/wake slowdown

This is a focused synthetic microbenchmark, not an application benchmark. It
uses one raw-UAPI ring and 32 cacheline-separated private futex words on one
pinned P-core. Each timed cycle submits 32 scalar IORING_OP_FUTEX_WAIT
requests, then 32 scalar IORING_OP_FUTEX_WAKE requests, and drains exactly 64
CQEs. Every wait must return 0 and every wake must return 1.

I used a fresh boot for each point:

  6a8118a77eec parent A -> 079afb081c42 child -> 6a8118a77eec parent B

Each point had 3 warm-up rounds and 15 measured rounds. Every measured round
ran 512 cycles, or 16,384 wait/wake pairs. The results in ns/pair were:

  implementation   parent A    child    parent B    child vs midpoint
  formal              180.079  196.647     179.856        +9.268%
  standalone          180.171  197.856     179.502       +10.020%

For the formal source, dropping the first measured round gave +9.269%. Parent
drift was -0.124%, and the maximum CV was 0.169%. The standalone drop-first
result was +9.996%, with -0.371% parent drift. All 90 scalar timing rows
passed the CQE, result, timeout, overflow, outstanding-request, and CPU
checks.

An untimed child trace also hit io_futex_prep(), io_futex_wait(),
io_futex_wake(), and io_futex_complete() with the expected request counts.

A matched WAITV -> WAKE profile changed by only +1.385%, below my preregistered
5% signal gate, so my claim is limited to scalar wait/wake.

I understand that 079afb fixes the exit-time use-after-free by keeping pending
private futex waits visible to cancellation before their mm state disappears.
Scalar WAIT and WAKE both use io_futex_prep(), so in the child both sides of
each measured pair execute the added tracking call. I am not suggesting a
revert.

Is this per-request cost an expected trade-off for the lifetime fix, or could
the same exit/mm-lifetime guarantee be retained with cheaper tracking?

Evidence bundle:

  https://github.com/lcf0399/linux-regression-evidence/tree/65c8cbf86f40cbe759e3f7db1d29d152ba03a8f2/io-uring-futex-inflight-wait-wake

Standalone reproducer:

  https://github.com/lcf0399/linux-regression-evidence/tree/65c8cbf86f40cbe759e3f7db1d29d152ba03a8f2/io-uring-futex-inflight-wait-wake/reproducer

Thanks,
Chengfeng

Re: [REGRESSION] io_uring/futex: scalar wait/wake slowdown after 079afb081c42

[Jens Axboe]

On 7/30/26 4:42 AM, Chengfeng Lin wrote:
> Hi Jens,
> 
> I tested 079afb081c42 against its direct parent on bare metal. In a narrow
> scalar io_uring futex wait/wake workload, the child was 9.27% slower. A
> separate 338-line standalone reproduced the result at 10.02%. All compared
> kernels actually ran with preempt=full.
> 
> #regzbot introduced: 079afb081c4288e94d5e4223d3eb6306d853c68b
> #regzbot title: io_uring scalar futex wait/wake slowdown
> 
> This is a focused synthetic microbenchmark, not an application benchmark. It
> uses one raw-UAPI ring and 32 cacheline-separated private futex words on one
> pinned P-core. Each timed cycle submits 32 scalar IORING_OP_FUTEX_WAIT
> requests, then 32 scalar IORING_OP_FUTEX_WAKE requests, and drains exactly 64
> CQEs. Every wait must return 0 and every wake must return 1.
> 
> I used a fresh boot for each point:
> 
>   6a8118a77eec parent A -> 079afb081c42 child -> 6a8118a77eec parent B
> 
> Each point had 3 warm-up rounds and 15 measured rounds. Every measured round
> ran 512 cycles, or 16,384 wait/wake pairs. The results in ns/pair were:
> 
>   implementation   parent A    child    parent B    child vs midpoint
>   formal              180.079  196.647     179.856        +9.268%
>   standalone          180.171  197.856     179.502       +10.020%
> 
> For the formal source, dropping the first measured round gave +9.269%. Parent
> drift was -0.124%, and the maximum CV was 0.169%. The standalone drop-first
> result was +9.996%, with -0.371% parent drift. All 90 scalar timing rows
> passed the CQE, result, timeout, overflow, outstanding-request, and CPU
> checks.
> 
> An untimed child trace also hit io_futex_prep(), io_futex_wait(),
> io_futex_wake(), and io_futex_complete() with the expected request counts.
> 
> A matched WAITV -> WAKE profile changed by only +1.385%, below my preregistered
> 5% signal gate, so my claim is limited to scalar wait/wake.
> 
> I understand that 079afb fixes the exit-time use-after-free by keeping pending
> private futex waits visible to cancellation before their mm state disappears.
> Scalar WAIT and WAKE both use io_futex_prep(), so in the child both sides of
> each measured pair execute the added tracking call. I am not suggesting a
> revert.
> 
> Is this per-request cost an expected trade-off for the lifetime fix, or could
> the same exit/mm-lifetime guarantee be retained with cheaper tracking?
> 
> Evidence bundle:
> 
>   https://github.com/lcf0399/linux-regression-evidence/tree/65c8cbf86f40cbe759e3f7db1d29d152ba03a8f2/io-uring-futex-inflight-wait-wake
> 
> Standalone reproducer:
> 
>   https://github.com/lcf0399/linux-regression-evidence/tree/65c8cbf86f40cbe759e3f7db1d29d152ba03a8f2/io-uring-futex-inflight-wait-wake/reproducer

Great report, thanks for that! I'll take a look at this. The inflight
tracking is a bit of a big hammer for sure for this, and it isn't even
needed on the wake side. Can you tell me what parameters you're using
for the reproducers?

-- 
Jens Axboe

Re: [REGRESSION] io_uring/futex: scalar wait/wake slowdown after 079afb081c42

[Jens Axboe]

[-- Attachment #1: Type: text/plain, Size: 3120 bytes --]

On 7/30/26 8:34 AM, Jens Axboe wrote:
> On 7/30/26 4:42 AM, Chengfeng Lin wrote:
>> Hi Jens,
>>
>> I tested 079afb081c42 against its direct parent on bare metal. In a narrow
>> scalar io_uring futex wait/wake workload, the child was 9.27% slower. A
>> separate 338-line standalone reproduced the result at 10.02%. All compared
>> kernels actually ran with preempt=full.
>>
>> #regzbot introduced: 079afb081c4288e94d5e4223d3eb6306d853c68b
>> #regzbot title: io_uring scalar futex wait/wake slowdown
>>
>> This is a focused synthetic microbenchmark, not an application benchmark. It
>> uses one raw-UAPI ring and 32 cacheline-separated private futex words on one
>> pinned P-core. Each timed cycle submits 32 scalar IORING_OP_FUTEX_WAIT
>> requests, then 32 scalar IORING_OP_FUTEX_WAKE requests, and drains exactly 64
>> CQEs. Every wait must return 0 and every wake must return 1.
>>
>> I used a fresh boot for each point:
>>
>>   6a8118a77eec parent A -> 079afb081c42 child -> 6a8118a77eec parent B
>>
>> Each point had 3 warm-up rounds and 15 measured rounds. Every measured round
>> ran 512 cycles, or 16,384 wait/wake pairs. The results in ns/pair were:
>>
>>   implementation   parent A    child    parent B    child vs midpoint
>>   formal              180.079  196.647     179.856        +9.268%
>>   standalone          180.171  197.856     179.502       +10.020%
>>
>> For the formal source, dropping the first measured round gave +9.269%. Parent
>> drift was -0.124%, and the maximum CV was 0.169%. The standalone drop-first
>> result was +9.996%, with -0.371% parent drift. All 90 scalar timing rows
>> passed the CQE, result, timeout, overflow, outstanding-request, and CPU
>> checks.
>>
>> An untimed child trace also hit io_futex_prep(), io_futex_wait(),
>> io_futex_wake(), and io_futex_complete() with the expected request counts.
>>
>> A matched WAITV -> WAKE profile changed by only +1.385%, below my preregistered
>> 5% signal gate, so my claim is limited to scalar wait/wake.
>>
>> I understand that 079afb fixes the exit-time use-after-free by keeping pending
>> private futex waits visible to cancellation before their mm state disappears.
>> Scalar WAIT and WAKE both use io_futex_prep(), so in the child both sides of
>> each measured pair execute the added tracking call. I am not suggesting a
>> revert.
>>
>> Is this per-request cost an expected trade-off for the lifetime fix, or could
>> the same exit/mm-lifetime guarantee be retained with cheaper tracking?
>>
>> Evidence bundle:
>>
>>   https://github.com/lcf0399/linux-regression-evidence/tree/65c8cbf86f40cbe759e3f7db1d29d152ba03a8f2/io-uring-futex-inflight-wait-wake
>>
>> Standalone reproducer:
>>
>>   https://github.com/lcf0399/linux-regression-evidence/tree/65c8cbf86f40cbe759e3f7db1d29d152ba03a8f2/io-uring-futex-inflight-wait-wake/reproducer
> 
> Great report, thanks for that! I'll take a look at this. The inflight
> tracking is a bit of a big hammer for sure for this, and it isn't even
> needed on the wake side. Can you tell me what parameters you're using
> for the reproducers?

Try with these two patches.

-- 
Jens Axboe

[-- Attachment #2: 0002-io_uring-futex-only-mark-private-futex-waits-as-infl.patch --]
[-- Type: text/x-patch, Size: 2623 bytes --]

From b47f14f3025f908dc71932a2ae4b82ffcad2cd37 Mon Sep 17 00:00:00 2001
From: Jens Axboe <axboe@kernel.dk>
Date: Thu, 30 Jul 2026 06:50:47 -0600
Subject: [PATCH 2/2] io_uring/futex: only mark private futex waits as inflight

Inflight tracking of futex wait requests exists to ensure that do_exit()
-> io_uring_files_cancel() cancels them before the mm goes away, as a
private futex wait depends on the mm private futex hash staying alive
for the duration of the request.

Shared futexes have no such dependency. A FLAGS_SHARED request always
resolves to either an inode based key or an mm-shared key, both of which
fail futex_key_is_private() and hence always hash into the global futex
hash, whose lifetime isn't tied to the mm.

Only mark vectored futex waits as inflight if the futex is private.

Cc: stable@vger.kernel.org
Fixes: 079afb081c42 ("io_uring/futex: mark wait requests as inflight")
Link: https://lore.kernel.org/io-uring/CANGjgdn=R_qyUdE=j9za+vkmqcxacbP-84OHXF4nZ4ho9qRyVg@mail.gmail.com/
Signed-off-by: Jens Axboe <axboe@kernel.dk>
---
 io_uring/futex.c | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/io_uring/futex.c b/io_uring/futex.c
index eea0425f2bcb..f0d80a444f45 100644
--- a/io_uring/futex.c
+++ b/io_uring/futex.c
@@ -154,14 +154,16 @@ int io_futex_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
 
 int io_futex_wait_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
 {
+	struct io_futex *iof = io_kiocb_to_cmd(req, struct io_futex);
 	int ret;
 
 	ret = io_futex_prep(req, sqe);
 	if (unlikely(ret))
 		return ret;
 
-	/* Mark as inflight, so file exit cancelation will find it */
-	io_req_track_inflight(req);
+	/* inflight tracking only needed for mm private hash */
+	if (!(iof->futex_flags & FLAGS_SHARED))
+		io_req_track_inflight(req);
 	return 0;
 }
 
@@ -186,6 +188,7 @@ int io_futexv_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
 {
 	struct io_futex *iof = io_kiocb_to_cmd(req, struct io_futex);
 	struct io_futexv_data *ifd;
+	unsigned int i;
 	int ret;
 
 	/* No flags or mask supported for waitv */
@@ -210,8 +213,14 @@ int io_futexv_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
 		return ret;
 	}
 
-	/* Mark as inflight, so file exit cancelation will find it */
-	io_req_track_inflight(req);
+	/* inflight tracking only needed for mm private hash */
+	for (i = 0; i < iof->futex_nr; i++) {
+		if (!(ifd->futexv[i].w.flags & FLAGS_SHARED)) {
+			io_req_track_inflight(req);
+			break;
+		}
+	}
+
 	iof->futexv_unqueued = 0;
 	req->flags |= REQ_F_ASYNC_DATA;
 	req->async_data = ifd;
-- 
2.53.0


[-- Attachment #3: 0001-io_uring-futex-don-t-mark-futex-wake-requests-as-inf.patch --]
[-- Type: text/x-patch, Size: 2805 bytes --]

From abd086a38ea8442d87bbedc0ed944fecba964be8 Mon Sep 17 00:00:00 2001
From: Jens Axboe <axboe@kernel.dk>
Date: Thu, 30 Jul 2026 06:50:13 -0600
Subject: [PATCH 1/2] io_uring/futex: don't mark futex wake requests as
 inflight

Commit 079afb081c42 ("io_uring/futex: mark wait requests as inflight")
added inflight tracking to ensure that do_exit() ->
io_uring_files_cancel() finds and cancels pending futex waits before the
mm goes away, as a private futex wait depends on the mm private futex
hash staying alive for the duration of the request. However, as
io_futex_prep() is shared between FUTEX_WAIT and FUTEX_WAKE, wake
requests got marked as inflight as well.

A futex wake executes fully inline at issue time and never depends on
the mm staying alive after completion, hence there's no need to track
it. Kill it.

Cc: stable@vger.kernel.org
Fixes: 079afb081c42 ("io_uring/futex: mark wait requests as inflight")
Reported-by: Chengfeng Lin <lin2530632123@gmail.com>
Link: https://lore.kernel.org/io-uring/CANGjgdn=R_qyUdE=j9za+vkmqcxacbP-84OHXF4nZ4ho9qRyVg@mail.gmail.com/
Signed-off-by: Jens Axboe <axboe@kernel.dk>
---
 io_uring/futex.c | 11 +++++++++++
 io_uring/futex.h |  1 +
 io_uring/opdef.c |  2 +-
 3 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/io_uring/futex.c b/io_uring/futex.c
index 906701b3c5c6..eea0425f2bcb 100644
--- a/io_uring/futex.c
+++ b/io_uring/futex.c
@@ -149,6 +149,17 @@ int io_futex_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
 	    !futex_validate_input(iof->futex_flags, iof->futex_mask))
 		return -EINVAL;
 
+	return 0;
+}
+
+int io_futex_wait_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
+{
+	int ret;
+
+	ret = io_futex_prep(req, sqe);
+	if (unlikely(ret))
+		return ret;
+
 	/* Mark as inflight, so file exit cancelation will find it */
 	io_req_track_inflight(req);
 	return 0;
diff --git a/io_uring/futex.h b/io_uring/futex.h
index d789fcf715e3..987db3f2c6d9 100644
--- a/io_uring/futex.h
+++ b/io_uring/futex.h
@@ -3,6 +3,7 @@
 #include "cancel.h"
 
 int io_futex_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe);
+int io_futex_wait_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe);
 int io_futexv_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe);
 int io_futex_wait(struct io_kiocb *req, unsigned int issue_flags);
 int io_futexv_wait(struct io_kiocb *req, unsigned int issue_flags);
diff --git a/io_uring/opdef.c b/io_uring/opdef.c
index 4e58eb1344ea..cf3aa2242cd7 100644
--- a/io_uring/opdef.c
+++ b/io_uring/opdef.c
@@ -467,7 +467,7 @@ const struct io_issue_def io_issue_defs[] = {
 	},
 	[IORING_OP_FUTEX_WAIT] = {
 #if defined(CONFIG_FUTEX)
-		.prep			= io_futex_prep,
+		.prep			= io_futex_wait_prep,
 		.issue			= io_futex_wait,
 #else
 		.prep			= io_eopnotsupp_prep,
-- 
2.53.0

Re: [REGRESSION] io_uring/futex: scalar wait/wake slowdown after 079afb081c42

[Chengfeng Lin]

Hi Jens,

Thanks for your reply, and sorry that I forgot to include the reproducer
parameters in my previous message. The exact standalone parameters are:

  ring entries: 64
  futex words: 32 private u32 words, one per cache line
  futex flags: FUTEX2_SIZE_U32 | FUTEX2_PRIVATE
  per cycle: submit 32 scalar WAITs for value 0, then 32 scalar WAKEs
             with nr_wake=1, then drain exactly 64 CQEs
  per round: 512 cycles, or 16,384 WAIT/WAKE pairs
  run: 3 untimed warm-up rounds followed by 15 measured rounds
  machine: Intel Core i7-12700KF, 20 logical CPUs, 32 GiB RAM
  CPU: one process pinned to P-core CPU 2

Each WAIT CQE must return 0, each WAKE CQE must return 1, and the program
fails on a missing or duplicate CQE, an unexpected flag or result, timeout,
CQ overflow, outstanding request, or CPU migration. The normal command has
no arguments:

  ./io_uring_futex_wait_wake_standalone

I also tested the exact two patches you attached on frozen Linus master
48a5a7ab8d6a. I used the same normalized config, GCC 15.2.0 toolchain, and
Kbuild metadata, with a fresh boot for every measured point:

  baseline A -> patch 1 A -> full series -> patch 1 B -> baseline B

The means were:

  point              ns/pair
  baseline A          196.271
  patch 1 A           191.071
  full series         189.900
  patch 1 B           188.824
  baseline B          196.962

Using the surrounding-control midpoints, patch 1 was 3.392% faster than the
baseline and the full series was 3.416% faster. Full versus the patch-1
midpoint changed by only -0.025%. This is consistent with patch 2 retaining
inflight tracking for private waits.

Baseline drift was 0.352% and patch-1 drift was -1.176%. Dropping the first
measured round gave -3.323%, -3.345%, and -0.022% for the same three
comparisons.

I then added a matched --shared mode. It keeps the workload and timing the
same. The only changes are that the words use a shared anonymous mapping and
the SQEs use FUTEX2_SIZE_U32 without FUTEX2_PRIVATE. This directly exercises
patch 2's scalar shared-WAIT path. I compared:

  patch 1 A -> full series -> patch 1 B

The means were 271.107, 267.254, and 271.971 ns/pair. The full series was
1.578% faster than the patch-1 midpoint. Patch-1 drift was 0.319%, the
drop-first result was also -1.578%, and the maximum CV was 0.200%.

All 120 measured rows across the private and shared tests passed their
checks. All eight measured boots actually ran with preempt=full; the process
used the performance governor and EPP, with Turbo disabled.

On this frozen-master baseline, patch 1 reduced latency by 3.392% in the
private-futex workload. Adding patch 2 did not materially change the private
result. In the matched scalar shared-futex test, patch 2 reduced latency by
1.578%. I have not tested shared WAITV.

This is a different source baseline from my earlier direct-parent test of
079afb081c42, so I am not subtracting the 3.4% here from the earlier 9.27%
result.

The compact results and exact patch identities are available here:

  https://github.com/lcf0399/linux-regression-evidence/tree/87da380476eb56cb3642d6ed15baf22c59496094/io-uring-futex-inflight-wait-wake

Thanks,
Chengfeng

Jens Axboe <axboe@kernel.dk> 于2026年7月30日周四 23:49写道：
>
> On 7/30/26 8:34 AM, Jens Axboe wrote:
> > On 7/30/26 4:42 AM, Chengfeng Lin wrote:
> >> Hi Jens,
> >>
> >> I tested 079afb081c42 against its direct parent on bare metal. In a narrow
> >> scalar io_uring futex wait/wake workload, the child was 9.27% slower. A
> >> separate 338-line standalone reproduced the result at 10.02%. All compared
> >> kernels actually ran with preempt=full.
> >>
> >> #regzbot introduced: 079afb081c4288e94d5e4223d3eb6306d853c68b
> >> #regzbot title: io_uring scalar futex wait/wake slowdown
> >>
> >> This is a focused synthetic microbenchmark, not an application benchmark. It
> >> uses one raw-UAPI ring and 32 cacheline-separated private futex words on one
> >> pinned P-core. Each timed cycle submits 32 scalar IORING_OP_FUTEX_WAIT
> >> requests, then 32 scalar IORING_OP_FUTEX_WAKE requests, and drains exactly 64
> >> CQEs. Every wait must return 0 and every wake must return 1.
> >>
> >> I used a fresh boot for each point:
> >>
> >>   6a8118a77eec parent A -> 079afb081c42 child -> 6a8118a77eec parent B
> >>
> >> Each point had 3 warm-up rounds and 15 measured rounds. Every measured round
> >> ran 512 cycles, or 16,384 wait/wake pairs. The results in ns/pair were:
> >>
> >>   implementation   parent A    child    parent B    child vs midpoint
> >>   formal              180.079  196.647     179.856        +9.268%
> >>   standalone          180.171  197.856     179.502       +10.020%
> >>
> >> For the formal source, dropping the first measured round gave +9.269%. Parent
> >> drift was -0.124%, and the maximum CV was 0.169%. The standalone drop-first
> >> result was +9.996%, with -0.371% parent drift. All 90 scalar timing rows
> >> passed the CQE, result, timeout, overflow, outstanding-request, and CPU
> >> checks.
> >>
> >> An untimed child trace also hit io_futex_prep(), io_futex_wait(),
> >> io_futex_wake(), and io_futex_complete() with the expected request counts.
> >>
> >> A matched WAITV -> WAKE profile changed by only +1.385%, below my preregistered
> >> 5% signal gate, so my claim is limited to scalar wait/wake.
> >>
> >> I understand that 079afb fixes the exit-time use-after-free by keeping pending
> >> private futex waits visible to cancellation before their mm state disappears.
> >> Scalar WAIT and WAKE both use io_futex_prep(), so in the child both sides of
> >> each measured pair execute the added tracking call. I am not suggesting a
> >> revert.
> >>
> >> Is this per-request cost an expected trade-off for the lifetime fix, or could
> >> the same exit/mm-lifetime guarantee be retained with cheaper tracking?
> >>
> >> Evidence bundle:
> >>
> >>   https://github.com/lcf0399/linux-regression-evidence/tree/65c8cbf86f40cbe759e3f7db1d29d152ba03a8f2/io-uring-futex-inflight-wait-wake
> >>
> >> Standalone reproducer:
> >>
> >>   https://github.com/lcf0399/linux-regression-evidence/tree/65c8cbf86f40cbe759e3f7db1d29d152ba03a8f2/io-uring-futex-inflight-wait-wake/reproducer
> >
> > Great report, thanks for that! I'll take a look at this. The inflight
> > tracking is a bit of a big hammer for sure for this, and it isn't even
> > needed on the wake side. Can you tell me what parameters you're using
> > for the reproducers?
>
> Try with these two patches.
>
> --
> Jens Axboe