From: Jens Axboe <axboe@kernel.dk>
To: syzbot <syzbot+4eb282331cab6d5b6588@syzkaller.appspotmail.com>,
io-uring@vger.kernel.org, linux-kernel@vger.kernel.org,
syzkaller-bugs@googlegroups.com
Subject: Re: [syzbot] [io-uring?] INFO: task hung in io_wq_put_and_exit (6)
Date: Wed, 21 Jan 2026 10:39:59 -0700 [thread overview]
Message-ID: <3b3a65a9-44d7-43d3-a4ad-62557fb48758@kernel.dk> (raw)
In-Reply-To: <696fc9e7.a70a0220.111c58.0006.GAE@google.com>
On 1/20/26 11:31 AM, syzbot wrote:
> Hello,
>
> syzbot has tested the proposed patch but the reproducer is still triggering an issue:
> INFO: task hung in io_wq_put_and_exit
>
> INFO: task syz.1.135:6891 blocked for more than 143 seconds.
> Not tainted syzkaller #0
> Blocked by coredump.
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> task:syz.1.135 state:D stack:25688 pid:6891 tgid:6887 ppid:6342 task_flags:0x400548 flags:0x00080000
> Call Trace:
> <TASK>
> context_switch kernel/sched/core.c:5260 [inline]
> __schedule+0xfe4/0x5e10 kernel/sched/core.c:6867
> __schedule_loop kernel/sched/core.c:6949 [inline]
> schedule+0xdd/0x390 kernel/sched/core.c:6964
> schedule_timeout+0x1b2/0x280 kernel/time/sleep_timeout.c:75
> do_wait_for_common kernel/sched/completion.c:100 [inline]
> __wait_for_common+0x2e7/0x4c0 kernel/sched/completion.c:121
> io_wq_exit_workers io_uring/io-wq.c:1325 [inline]
Not sure how much better we can make this. syzbot is running on 2 cpus,
and spawns hundreds of "lets read 2GB from MSR", which are super slow.
So you have 2 cpus wanting to run hundreds of these. And yes that'll
mean that exiting a ring can take a loooong time, because even if it
needs to finish just a single reader, that's a lot of MSR data to read
when you have hundreds of tasks doing the same thing.
IOW, there's no bug here, other than yes if you overload the system so
substantially on a small system, then yes things will take a long time
to finish.
That said, it'd be nice to get this bug flagged as such, however I'm not
aware of any way to really do that. We can obviously work-around this in
io-wq:
diff --git a/io_uring/io-wq.c b/io_uring/io-wq.c
index 2fa7d3601edb..3c94f281ff6b 100644
--- a/io_uring/io-wq.c
+++ b/io_uring/io-wq.c
@@ -17,6 +17,7 @@
#include <linux/task_work.h>
#include <linux/audit.h>
#include <linux/mmu_context.h>
+#include <linux/sched/sysctl.h>
#include <uapi/linux/io_uring.h>
#include "io-wq.h"
@@ -1313,6 +1314,13 @@ static void io_wq_cancel_tw_create(struct io_wq *wq)
static void io_wq_exit_workers(struct io_wq *wq)
{
+ /*
+ * Shut up hung task complaint, see for example
+ *
+ * https://lore.kernel.org/all/696fc9e7.a70a0220.111c58.0006.GAE@google.com/
+ */
+ unsigned long timeout = sysctl_hung_task_timeout_secs * HZ / 2;
+
if (!wq->task)
return;
@@ -1322,7 +1330,11 @@ static void io_wq_exit_workers(struct io_wq *wq)
io_wq_for_each_worker(wq, io_wq_worker_wake, NULL);
rcu_read_unlock();
io_worker_ref_put(wq);
- wait_for_completion(&wq->worker_done);
+ do {
+ if (wait_for_completion_timeout(&wq->worker_done, timeout))
+ break;
+ printk("io-wq: taking a long time to exit\n");
+ } while (1);
spin_lock_irq(&wq->hash->wait.lock);
list_del_init(&wq->wait.entry);
which we can obviously do, but it's also really annoying imho. But I
guess that can be coupled with a dump, etc.
--
Jens Axboe
next prev parent reply other threads:[~2026-01-21 17:40 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-08-18 8:05 [syzbot] [io-uring?] INFO: task hung in io_wq_put_and_exit (6) syzbot
2026-01-20 3:48 ` Jens Axboe
2026-01-20 4:13 ` syzbot
2026-01-20 15:04 ` Jens Axboe
2026-01-20 18:05 ` Jens Axboe
2026-01-20 18:31 ` syzbot
2026-01-21 17:39 ` Jens Axboe [this message]
2026-01-21 17:41 ` Jens Axboe
2026-01-21 19:30 ` syzbot
2026-01-21 21:41 ` Jens Axboe
2026-01-21 21:43 ` syzbot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3b3a65a9-44d7-43d3-a4ad-62557fb48758@kernel.dk \
--to=axboe@kernel.dk \
--cc=io-uring@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=syzbot+4eb282331cab6d5b6588@syzkaller.appspotmail.com \
--cc=syzkaller-bugs@googlegroups.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox