public inbox for [email protected]
 help / color / mirror / Atom feed
* [syzbot] possible deadlock in io_worker_cancel_cb
@ 2021-12-13  6:08 syzbot
  2021-12-13 16:06 ` Jens Axboe
  0 siblings, 1 reply; 3+ messages in thread
From: syzbot @ 2021-12-13  6:08 UTC (permalink / raw)
  To: asml.silence, axboe, haoxu, io-uring, linux-kernel,
	syzkaller-bugs

Hello,

syzbot found the following issue on:

HEAD commit:    a763d5a5abd6 Merge tag 'scsi-fixes' of git://git.kernel.or..
git tree:       upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=12b900bab00000
kernel config:  https://syzkaller.appspot.com/x/.config?x=7d5e878e3399b6cc
dashboard link: https://syzkaller.appspot.com/bug?extid=b18b8be69df33a3918e9
compiler:       Debian clang version 11.0.1-2, GNU ld (GNU Binutils for Debian) 2.35.2
syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=143f7551b00000
C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=17f56825b00000

The issue was bisected to:

commit 71a85387546e50b1a37b0fa45dadcae3bfb35cf6
Author: Jens Axboe <[email protected]>
Date:   Fri Dec 10 15:29:30 2021 +0000

    io-wq: check for wq exit after adding new worker task_work

bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=17a634bab00000
final oops:     https://syzkaller.appspot.com/x/report.txt?x=146634bab00000
console output: https://syzkaller.appspot.com/x/log.txt?x=106634bab00000

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: [email protected]
Fixes: 71a85387546e ("io-wq: check for wq exit after adding new worker task_work")

============================================
WARNING: possible recursive locking detected
5.16.0-rc4-syzkaller #0 Not tainted
--------------------------------------------
iou-wrk-6468/6471 is trying to acquire lock:
ffff88801aa98018 (&wqe->lock){+.+.}-{2:2}, at: io_worker_cancel_cb+0xb7/0x210 fs/io-wq.c:187

but task is already holding lock:
ffff88801aa98018 (&wqe->lock){+.+.}-{2:2}, at: io_wq_worker_sleeping+0xb6/0x140 fs/io-wq.c:700

other info that might help us debug this:
 Possible unsafe locking scenario:

       CPU0
       ----
  lock(&wqe->lock);
  lock(&wqe->lock);

 *** DEADLOCK ***

 May be due to missing lock nesting notation

1 lock held by iou-wrk-6468/6471:
 #0: ffff88801aa98018 (&wqe->lock){+.+.}-{2:2}, at: io_wq_worker_sleeping+0xb6/0x140 fs/io-wq.c:700

stack backtrace:
CPU: 1 PID: 6471 Comm: iou-wrk-6468 Not tainted 5.16.0-rc4-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x1dc/0x2d8 lib/dump_stack.c:106
 print_deadlock_bug kernel/locking/lockdep.c:2956 [inline]
 check_deadlock kernel/locking/lockdep.c:2999 [inline]
 validate_chain+0x5984/0x8240 kernel/locking/lockdep.c:3788
 __lock_acquire+0x1382/0x2b00 kernel/locking/lockdep.c:5027
 lock_acquire+0x19f/0x4d0 kernel/locking/lockdep.c:5637
 __raw_spin_lock include/linux/spinlock_api_smp.h:133 [inline]
 _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:154
 io_worker_cancel_cb+0xb7/0x210 fs/io-wq.c:187
 io_wq_cancel_tw_create fs/io-wq.c:1220 [inline]
 io_queue_worker_create+0x3cf/0x4c0 fs/io-wq.c:372
 io_wq_worker_sleeping+0xbe/0x140 fs/io-wq.c:701
 sched_submit_work kernel/sched/core.c:6295 [inline]
 schedule+0x67/0x1f0 kernel/sched/core.c:6323
 schedule_timeout+0xac/0x300 kernel/time/timer.c:1857
 wait_woken+0xca/0x1b0 kernel/sched/wait.c:460
 unix_msg_wait_data net/unix/unix_bpf.c:32 [inline]
 unix_bpf_recvmsg+0x7f9/0xe20 net/unix/unix_bpf.c:77
 unix_stream_recvmsg+0x214/0x2c0 net/unix/af_unix.c:2832
 sock_recvmsg_nosec net/socket.c:944 [inline]
 sock_recvmsg net/socket.c:962 [inline]
 sock_read_iter+0x3a7/0x4d0 net/socket.c:1035
 call_read_iter include/linux/fs.h:2156 [inline]
 io_iter_do_read fs/io_uring.c:3501 [inline]
 io_read fs/io_uring.c:3558 [inline]
 io_issue_sqe+0x144c/0x9590 fs/io_uring.c:6671
 io_wq_submit_work+0x2d8/0x790 fs/io_uring.c:6836
 io_worker_handle_work+0x808/0xdd0 fs/io-wq.c:574
 io_wqe_worker+0x395/0x870 fs/io-wq.c:630
 ret_from_fork+0x1f/0x30
 </TASK>


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at [email protected].

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
For information about bisection process see: https://goo.gl/tpsmEJ#bisection
syzbot can test patches for this issue, for details see:
https://goo.gl/tpsmEJ#testing-patches

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [syzbot] possible deadlock in io_worker_cancel_cb
  2021-12-13  6:08 [syzbot] possible deadlock in io_worker_cancel_cb syzbot
@ 2021-12-13 16:06 ` Jens Axboe
  2021-12-13 16:40   ` syzbot
  0 siblings, 1 reply; 3+ messages in thread
From: Jens Axboe @ 2021-12-13 16:06 UTC (permalink / raw)
  To: syzbot, asml.silence, haoxu, io-uring, linux-kernel,
	syzkaller-bugs

On 12/12/21 11:08 PM, syzbot wrote:
> Hello,
> 
> syzbot found the following issue on:
> 
> HEAD commit:    a763d5a5abd6 Merge tag 'scsi-fixes' of git://git.kernel.or..
> git tree:       upstream
> console output: https://syzkaller.appspot.com/x/log.txt?x=12b900bab00000
> kernel config:  https://syzkaller.appspot.com/x/.config?x=7d5e878e3399b6cc
> dashboard link: https://syzkaller.appspot.com/bug?extid=b18b8be69df33a3918e9
> compiler:       Debian clang version 11.0.1-2, GNU ld (GNU Binutils for Debian) 2.35.2
> syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=143f7551b00000
> C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=17f56825b00000
> 
> The issue was bisected to:
> 
> commit 71a85387546e50b1a37b0fa45dadcae3bfb35cf6
> Author: Jens Axboe <[email protected]>
> Date:   Fri Dec 10 15:29:30 2021 +0000
> 
>     io-wq: check for wq exit after adding new worker task_work

#syz test git://git.kernel.dk/linux-block io_uring-5.16

-- 
Jens Axboe


^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [syzbot] possible deadlock in io_worker_cancel_cb
  2021-12-13 16:06 ` Jens Axboe
@ 2021-12-13 16:40   ` syzbot
  0 siblings, 0 replies; 3+ messages in thread
From: syzbot @ 2021-12-13 16:40 UTC (permalink / raw)
  To: asml.silence, axboe, haoxu, io-uring, linux-kernel,
	syzkaller-bugs

Hello,

syzbot has tested the proposed patch and the reproducer did not trigger any issue:

Reported-and-tested-by: [email protected]

Tested on:

commit:         d800c65c io-wq: drop wqe lock before creating new worker
git tree:       git://git.kernel.dk/linux-block io_uring-5.16
kernel config:  https://syzkaller.appspot.com/x/.config?x=6c3ab72998e7f1a4
dashboard link: https://syzkaller.appspot.com/bug?extid=b18b8be69df33a3918e9
compiler:       Debian clang version 11.0.1-2, GNU ld (GNU Binutils for Debian) 2.35.2

Note: testing is done by a robot and is best-effort only.

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2021-12-13 16:40 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-12-13  6:08 [syzbot] possible deadlock in io_worker_cancel_cb syzbot
2021-12-13 16:06 ` Jens Axboe
2021-12-13 16:40   ` syzbot

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox