Re: [RFC PATCH 2/2] io_uring: fix io may accumulation in poll mode

["Diangang Li" <lidiangang@bytedance.com>]

On 2025/12/18 00:25, Jens Axboe wrote:
> On 12/17/25 5:34 AM, Diangang Li wrote:
>> Hi Jens,
>>
>> We?ve identified one critical panic issue here.
>>
>> [ 4504.422964] [  T63683] list_del corruption, ff2adc9b51d19a90->next is
>> LIST_POISON1 (dead000000000100)
>> [ 4504.422994] [  T63683] ------------[ cut here ]------------
>> [ 4504.422995] [  T63683] kernel BUG at lib/list_debug.c:56!
>> [ 4504.423006] [  T63683] Oops: invalid opcode: 0000 [#1] SMP NOPTI
>> [ 4504.423017] [  T63683] CPU: 38 UID: 0 PID: 63683 Comm: io_uring
>> Kdump: loaded Tainted: G S          E       6.19.0-rc1+ #1
>> PREEMPT(voluntary)
>> [ 4504.423032] [  T63683] Tainted: [S]=CPU_OUT_OF_SPEC, [E]=UNSIGNED_MODULE
>> [ 4504.423040] [  T63683] Hardware name: Inventec S520-A6/Nanping MLB,
>> BIOS 01.01.01.06.03 03/03/2023
>> [ 4504.423050] [  T63683] RIP:
>> 0010:__list_del_entry_valid_or_report+0x94/0x100
>> [ 4504.423064] [  T63683] Code: 89 fe 48 c7 c7 f0 78 87 b5 e8 38 07 ae
>> ff 0f 0b 48 89 ef e8 6e 40 cd ff 48 89 ea 48 89 de 48 c7 c7 20 79 87 b5
>> e8 1c 07 ae ff <0f> 0b 4c 89 e7 e8 52 40 cd ff 4c 89 e2 48 89 de 48 c7
>> c7 58 79 87
>> [ 4504.423085] [  T63683] RSP: 0018:ff4efd9f3838fdb0 EFLAGS: 00010246
>> [ 4504.423093] [  T63683] RAX: 000000000000004e RBX: ff2adc9b51d19a90
>> RCX: 0000000000000027
>> [ 4504.423103] [  T63683] RDX: 0000000000000000 RSI: 0000000000000001
>> RDI: ff2add151cf99580
>> [ 4504.423112] [  T63683] RBP: dead000000000100 R08: 0000000000000000
>> R09: 0000000000000003
>> [ 4504.423120] [  T63683] R10: ff4efd9f3838fc60 R11: ff2add151cdfffe8
>> R12: dead000000000122
>> [ 4504.423130] [  T63683] R13: ff2adc9b51d19a00 R14: 0000000000000000
>> R15: 0000000000000000
>> [ 4504.423139] [  T63683] FS:  00007fae4f7ff6c0(0000)
>> GS:ff2add15665f5000(0000) knlGS:0000000000000000
>> [ 4504.423148] [  T63683] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> [ 4504.423157] [  T63683] CR2: 000055aa8afe5000 CR3: 00000083037ee006
>> CR4: 0000000000773ef0
>> [ 4504.423166] [  T63683] PKRU: 55555554
>> [ 4504.423171] [  T63683] Call Trace:
>> [ 4504.423178] [  T63683]  <TASK>
>> [ 4504.423184] [  T63683]  io_do_iopoll+0x298/0x330
>> [ 4504.423193] [  T63683]  ? asm_sysvec_apic_timer_interrupt+0x1a/0x20
>> [ 4504.423204] [  T63683]  __do_sys_io_uring_enter+0x421/0x770
>> [ 4504.423214] [  T63683]  do_syscall_64+0x67/0xf00
>> [ 4504.423223] [  T63683]  entry_SYSCALL_64_after_hwframe+0x76/0x7e
>> [ 4504.423232] [  T63683] RIP: 0033:0x55aa707e99c3
>>
>> It can be reproduced in three ways:
>> - Running iopoll tests while switching the block scheduler
>> - A split IO scenario in iopoll (e.g., bs=512k with max_sectors_kb=256k)
>> - Multi poll queues with multi threads
>>
>> All cases appear related to IO completions occurring outside the
>> io_do_iopoll() loop. The root cause remains unclear.
> 
> Ah I see what it is - we can get multiple completions on the iopoll
> side, if you have multiple bio's per request. This didn't matter before
> the patch that uses a lockless list to collect them, as it just marked
> the request completed by writing to ->iopoll_complete and letting the
> reaper find them. But it matters with the llist change, as then we're
> adding the request to the llist more than once.
> 
> 

 From e2f749299e3c76ef92d3edfd9f8f7fc9a029129a Mon Sep 17 00:00:00 2001
From: Diangang Li <lidiangang@bytedance.com>
Date: Fri, 19 Dec 2025 10:14:33 +0800
Subject: [PATCH] io_uring: fix race between adding to ctx->iopoll_list 
and ctx->iopoll_complete
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Since commit 316693eb8aed ("io_uring: be smarter about handling IOPOLL
completions") introduced ctx->iopoll_complete to cache polled 
completions, a request can be enqueued to ctx->iopoll_complete as part 
of a batched poll while it is still in the issuing path.

If the IO was submitted via io_wq_submit_work(), it may still be stuck 
in io_iopoll_req_issued() waiting for ctx->uring_lock, which is held by
io_do_iopoll(). In this state, io_do_iopoll() may attempt to delete the
request from ctx->iopoll_list before it has ever been linked, leading to 
a list_del() corruption.

Fix this by introducing an iopoll_state flag to mark whether the request
has been inserted into ctx->iopoll_list. When io_do_iopoll() tries to
unlink a request and the flag indicates it hasn’t been linked yet, skip
the list_del() and just requeue the completion to ctx->iopoll_complete 
for later reap.

Signed-off-by: Diangang Li <lidiangang@bytedance.com>
Signed-off-by: Fengnan Chang <changfengnan@bytedance.com>
---
  include/linux/io_uring_types.h | 1 +
  io_uring/io_uring.c            | 1 +
  io_uring/rw.c                  | 7 +++++++
  io_uring/uring_cmd.c           | 1 +
  4 files changed, 10 insertions(+)

diff --git a/include/linux/io_uring_types.h b/include/linux/io_uring_types.h
index 0f619c37dce4..aaf26911badb 100644
--- a/include/linux/io_uring_types.h
+++ b/include/linux/io_uring_types.h
@@ -677,6 +677,7 @@ struct io_kiocb {
  	};

  	u8				opcode;
+	u8				iopoll_state;

  	bool				cancel_seq_set;

diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c
index 5e503a0bfcfc..4eb206359d05 100644
--- a/io_uring/io_uring.c
+++ b/io_uring/io_uring.c
@@ -1692,6 +1692,7 @@ static void io_iopoll_req_issued(struct io_kiocb 
*req, unsigned int issue_flags)
  	}

  	list_add_tail(&req->iopoll_node, &ctx->iopoll_list);
+	smp_store_release(&req->iopoll_state, 1);

  	if (unlikely(needs_lock)) {
  		/*
diff --git a/io_uring/rw.c b/io_uring/rw.c
index ad481ca74a46..d1397739c58b 100644
--- a/io_uring/rw.c
+++ b/io_uring/rw.c
@@ -869,6 +869,7 @@ static int io_rw_init_file(struct io_kiocb *req, 
fmode_t mode, int rw_type)
  			return -EOPNOTSUPP;
  		kiocb->private = NULL;
  		kiocb->ki_flags |= IOCB_HIPRI;
+		req->iopoll_state = 0;
  		if (ctx->flags & IORING_SETUP_HYBRID_IOPOLL) {
  			/* make sure every req only blocks once*/
  			req->flags &= ~REQ_F_IOPOLL_STATE;
@@ -1355,6 +1356,12 @@ int io_do_iopoll(struct io_ring_ctx *ctx, bool 
force_nonspin)
  		struct llist_node *next = node->next;

  		req = container_of(node, struct io_kiocb, iopoll_done_list);
+		if (!READ_ONCE(req->iopoll_state)) {
+			node->next = NULL;
+			llist_add(&req->iopoll_done_list, &ctx->iopoll_complete);
+			node = next;
+			continue;
+		}
  		list_del(&req->iopoll_node);
  		wq_list_add_tail(&req->comp_list, &ctx->submit_state.compl_reqs);
  		nr_events++;
diff --git a/io_uring/uring_cmd.c b/io_uring/uring_cmd.c
index 0841fa541f5d..cf2eacea5be8 100644
--- a/io_uring/uring_cmd.c
+++ b/io_uring/uring_cmd.c
@@ -251,6 +251,7 @@ int io_uring_cmd(struct io_kiocb *req, unsigned int 
issue_flags)
  		if (!file->f_op->uring_cmd_iopoll)
  			return -EOPNOTSUPP;
  		issue_flags |= IO_URING_F_IOPOLL;
+		req->iopoll_state = 0;
  		if (ctx->flags & IORING_SETUP_HYBRID_IOPOLL) {
  			/* make sure every req only blocks once */
  			req->flags &= ~REQ_F_IOPOLL_STATE;
-- 
2.20.1