[BUG] soft lockup in seq_read while reading io_uring fdinfo

[BUG] soft lockup in seq_read while reading io_uring fdinfo

[是参差]

Hi,

I’m reporting a reproducible soft lockup observed in the seq_file read path when reading io_uring fdinfo via procfs.

The lockup is triggered by a syzkaller C reproducer that:

creates an io_uring instance with a large number of entries, and then

reads /proc/thread-self/fdinfo/<uring_fd>.

The watchdog reports a soft lockup with CPU stuck in __sanitizer_cov_trace_pc() while the task is executing seq_read() -> io_uring_show_fdinfo().

Another CPU concurrently shows an NMI backtrace stuck in KFENCE’s toggle_allocation_gate() path, going through jump_label_update() and smp_text_poke_sync_each_cpu(). This suggests a potential interaction between heavy fdinfo seq_printf output under KCOV instrumentation and KFENCE’s jump_label/text_poke synchronization.

Reproducer:

C reproducer: https://pastebin.com/raw/MxksimZh

console output: https://pastebin.com/raw/ZgwRNeTc

kernel config: https://pastebin.com/raw/qBYGyUzD

Kernel:

HEAD commit: 63804fed149a6750ffd28610c5c1c98cce6bd377

 git tree: torvalds/linux  

kernel version: 6.19.0-rc7 #2 PREEMPT(voluntary) (QEMU Ubuntu 24.10)


watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [syz.3.17:1226]
Modules linked in:
CPU: 1 UID: 0 PID: 1226 Comm: syz.3.17 Not tainted 6.19.0-rc7 #2 PREEMPT(voluntary)
Hardware name: QEMU Ubuntu 24.10 PC (i440FX + PIIX, 1996), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
RIP: 0010:__sanitizer_cov_trace_const_cmp1+0x8/0x20 kernel/kcov.c:300
Code: 00 00 f3 0f 1e fa 48 8b 0c 24 48 89 f2 48 89 fe bf 06 00 00 00 e9 18 ff ff ff 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 48 8b 0c 24 <40> 0f b6 d6 40 0f b6 f7 bf 01 00 00 00 e9 f6 fe ff ff 66 0f 1f 44
RSP: 0018:ffffc900043bf760 EFLAGS: 00000246
RAX: 000000003a736500 RBX: ffffffff850afe21 RCX: ffffffff84cd69b0
RDX: ffff88800c9a3bc0 RSI: 0000000000000073 RDI: 0000000000000025
RBP: ffffc900043bf810 R08: 000000003a736500 R09: ffffffff84cd69a3
R10: 0000000000000001 R11: ffffc90048c4d037 R12: ffffc900043bf8a0
R13: ffffffff850afe1b R14: dffffc0000000000 R15: 0000000000000073
FS: 00007f24f90fe500(0000) GS:ffff8880e5b52000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00005593e173d130 CR3: 000000000f716000 CR4: 0000000000350ef0
Call Trace:
 <TASK>
 format_decode+0x190/0xc80 lib/vsprintf.c:2692
 vsnprintf+0x186/0x1160 lib/vsprintf.c:2889
 seq_vprintf+0xe6/0x1a0 fs/seq_file.c:391
 seq_printf+0xbe/0xf0 fs/seq_file.c:406
 __io_uring_show_fdinfo io_uring/fdinfo.c:158 [inline]
 io_uring_show_fdinfo+0x9a7/0x1920 io_uring/fdinfo.c:261
 seq_show+0x469/0x730 fs/proc/fd.c:68
 seq_read_iter+0x4cc/0x11d0 fs/seq_file.c:230
 seq_read+0x391/0x570 fs/seq_file.c:162
 vfs_read fs/read_write.c:570 [inline]
 vfs_read+0x1f9/0xc90 fs/read_write.c:552
 ksys_read+0x121/0x240 fs/read_write.c:715
 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
 do_syscall_64+0xac/0x3b0 arch/x86/entry/syscall_64.c:94
 entry_SYSCALL_64_after_hwframe+0x4b/0x53
RIP: 0033:0x7f24f838ebe9
Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48

Re: [BUG] soft lockup in seq_read while reading io_uring fdinfo

[Jens Axboe]

On 2/3/26 2:39 AM, ??? wrote:
> Hi,
> 
> I?m reporting a reproducible soft lockup observed in the seq_file read path when reading io_uring fdinfo via procfs.
> 
> The lockup is triggered by a syzkaller C reproducer that:
> 
> creates an io_uring instance with a large number of entries, and then
> 
> reads /proc/thread-self/fdinfo/<uring_fd>.
> 
> The watchdog reports a soft lockup with CPU stuck in __sanitizer_cov_trace_pc() while the task is executing seq_read() -> io_uring_show_fdinfo().

It's feeding invalid cq ring head/tail entries, so it'll loop for
potentially quite a long time, particularly with debugging measures
enabled. This should sort it out:


diff --git a/io_uring/fdinfo.c b/io_uring/fdinfo.c
index 4f12e98b22c3..74ea0d965d78 100644
--- a/io_uring/fdinfo.c
+++ b/io_uring/fdinfo.c
@@ -67,7 +67,7 @@ static void __io_uring_show_fdinfo(struct io_ring_ctx *ctx, struct seq_file *m)
 	unsigned int cq_head = READ_ONCE(r->cq.head);
 	unsigned int cq_tail = READ_ONCE(r->cq.tail);
 	unsigned int sq_shift = 0;
-	unsigned int sq_entries;
+	unsigned int cq_entries, sq_entries;
 	int sq_pid = -1, sq_cpu = -1;
 	u64 sq_total_time = 0, sq_work_time = 0;
 	unsigned int i;
@@ -146,13 +146,15 @@ static void __io_uring_show_fdinfo(struct io_ring_ctx *ctx, struct seq_file *m)
 			}
 		}
 		seq_printf(m, "\n");
+		cond_resched();
 	}
 	seq_printf(m, "CQEs:\t%u\n", cq_tail - cq_head);
-	while (cq_head < cq_tail) {
+	cq_entries = min(cq_tail - cq_head, ctx->sq_entries);
+	for (i = 0; i < cq_entries; i++) {
 		struct io_uring_cqe *cqe;
 		bool cqe32 = false;
 
-		cqe = &r->cqes[(cq_head & cq_mask)];
+		cqe = &r->cqes[((cq_head + i) & cq_mask)];
 		if (cqe->flags & IORING_CQE_F_32 || ctx->flags & IORING_SETUP_CQE32)
 			cqe32 = true;
 		seq_printf(m, "%5u: user_data:%llu, res:%d, flags:%x",
@@ -165,6 +167,7 @@ static void __io_uring_show_fdinfo(struct io_ring_ctx *ctx, struct seq_file *m)
 		cq_head++;
 		if (cqe32)
 			cq_head++;
+		cond_resched();
 	}
 
 	if (ctx->flags & IORING_SETUP_SQPOLL) {

-- 
Jens Axboe