* [PATCH] io_uring: fix NULL pointer dereference for async cancel close
@ 2021-01-18 9:50 Joseph Qi
2021-01-18 12:23 ` Pavel Begunkov
2021-01-19 18:01 ` Jens Axboe
0 siblings, 2 replies; 12+ messages in thread
From: Joseph Qi @ 2021-01-18 9:50 UTC (permalink / raw)
To: Jens Axboe; +Cc: io-uring, Xiaoguang Wang
Abaci reported the following crash:
[ 31.252589] BUG: kernel NULL pointer dereference, address: 00000000000000d8
[ 31.253942] #PF: supervisor read access in kernel mode
[ 31.254945] #PF: error_code(0x0000) - not-present page
[ 31.255964] PGD 800000010b76f067 P4D 800000010b76f067 PUD 10b462067 PMD 0
[ 31.257221] Oops: 0000 [#1] SMP PTI
[ 31.257923] CPU: 1 PID: 1788 Comm: io_uring-sq Not tainted 5.11.0-rc4 #1
[ 31.259175] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[ 31.260232] RIP: 0010:__lock_acquire+0x19d/0x18c0
[ 31.261144] Code: 00 00 8b 1d fd 56 dd 08 85 db 0f 85 43 05 00 00 48 c7 c6 98 7b 95 82 48 c7 c7 57 96 93 82 e8 9a bc f5 ff 0f 0b e9 2b 05 00 00 <48> 81 3f c0 ca 67 8a b8 00 00 00 00 41 0f 45 c0 89 04 24 e9 81 fe
[ 31.264297] RSP: 0018:ffffc90001933828 EFLAGS: 00010002
[ 31.265320] RAX: 0000000000000001 RBX: 0000000000000001 RCX: 0000000000000000
[ 31.266594] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 00000000000000d8
[ 31.267922] RBP: 0000000000000246 R08: 0000000000000001 R09: 0000000000000000
[ 31.269262] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
[ 31.270550] R13: 0000000000000000 R14: ffff888106e8a140 R15: 00000000000000d8
[ 31.271760] FS: 0000000000000000(0000) GS:ffff88813bd00000(0000) knlGS:0000000000000000
[ 31.273269] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 31.274330] CR2: 00000000000000d8 CR3: 0000000106efa004 CR4: 00000000003706e0
[ 31.275613] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 31.276855] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 31.278065] Call Trace:
[ 31.278649] lock_acquire+0x31a/0x440
[ 31.279404] ? close_fd_get_file+0x39/0x160
[ 31.280276] ? __lock_acquire+0x647/0x18c0
[ 31.281112] _raw_spin_lock+0x2c/0x40
[ 31.281821] ? close_fd_get_file+0x39/0x160
[ 31.282586] close_fd_get_file+0x39/0x160
[ 31.283338] io_issue_sqe+0x1334/0x14e0
[ 31.284053] ? lock_acquire+0x31a/0x440
[ 31.284763] ? __io_free_req+0xcf/0x2e0
[ 31.285504] ? __io_free_req+0x175/0x2e0
[ 31.286247] ? find_held_lock+0x28/0xb0
[ 31.286968] ? io_wq_submit_work+0x7f/0x240
[ 31.287733] io_wq_submit_work+0x7f/0x240
[ 31.288486] io_wq_cancel_cb+0x161/0x580
[ 31.289230] ? io_wqe_wake_worker+0x114/0x360
[ 31.290020] ? io_uring_get_socket+0x40/0x40
[ 31.290832] io_async_find_and_cancel+0x3b/0x140
[ 31.291676] io_issue_sqe+0xbe1/0x14e0
[ 31.292405] ? __lock_acquire+0x647/0x18c0
[ 31.293207] ? __io_queue_sqe+0x10b/0x5f0
[ 31.293986] __io_queue_sqe+0x10b/0x5f0
[ 31.294747] ? io_req_prep+0xdb/0x1150
[ 31.295485] ? mark_held_locks+0x6d/0xb0
[ 31.296252] ? mark_held_locks+0x6d/0xb0
[ 31.297019] ? io_queue_sqe+0x235/0x4b0
[ 31.297774] io_queue_sqe+0x235/0x4b0
[ 31.298496] io_submit_sqes+0xd7e/0x12a0
[ 31.299275] ? _raw_spin_unlock_irq+0x24/0x30
[ 31.300121] ? io_sq_thread+0x3ae/0x940
[ 31.300873] io_sq_thread+0x207/0x940
[ 31.301606] ? do_wait_intr_irq+0xc0/0xc0
[ 31.302396] ? __ia32_sys_io_uring_enter+0x650/0x650
[ 31.303321] kthread+0x134/0x180
[ 31.303982] ? kthread_create_worker_on_cpu+0x90/0x90
[ 31.304886] ret_from_fork+0x1f/0x30
This is caused by NULL files when async cancel close, which has
IO_WQ_WORK_NO_CANCEL set and continue to do work. Fix it by also setting
needs_files for IORING_OP_ASYNC_CANCEL.
Reported-by: Abaci <[email protected]>
Cc: [email protected] # 5.6+
Signed-off-by: Joseph Qi <[email protected]>
---
fs/io_uring.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/fs/io_uring.c b/fs/io_uring.c
index 985a9e3..8eb1349 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -883,7 +883,10 @@ struct io_op_def {
.pollin = 1,
.work_flags = IO_WQ_WORK_MM | IO_WQ_WORK_FILES,
},
- [IORING_OP_ASYNC_CANCEL] = {},
+ [IORING_OP_ASYNC_CANCEL] = {
+ /* for async cancel close */
+ .needs_file = 1,
+ },
[IORING_OP_LINK_TIMEOUT] = {
.needs_async_data = 1,
.async_size = sizeof(struct io_timeout_data),
--
1.8.3.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PATCH] io_uring: fix NULL pointer dereference for async cancel close
2021-01-18 9:50 [PATCH] io_uring: fix NULL pointer dereference for async cancel close Joseph Qi
@ 2021-01-18 12:23 ` Pavel Begunkov
2021-01-18 15:08 ` Pavel Begunkov
2021-01-19 2:08 ` Joseph Qi
2021-01-19 18:01 ` Jens Axboe
1 sibling, 2 replies; 12+ messages in thread
From: Pavel Begunkov @ 2021-01-18 12:23 UTC (permalink / raw)
To: Joseph Qi, Jens Axboe; +Cc: io-uring, Xiaoguang Wang
On 18/01/2021 09:50, Joseph Qi wrote:
> Abaci reported the following crash:
>
> [ 31.252589] BUG: kernel NULL pointer dereference, address: 00000000000000d8
> [ 31.253942] #PF: supervisor read access in kernel mode
> [ 31.254945] #PF: error_code(0x0000) - not-present page
> [ 31.255964] PGD 800000010b76f067 P4D 800000010b76f067 PUD 10b462067 PMD 0
> [ 31.257221] Oops: 0000 [#1] SMP PTI
> [ 31.257923] CPU: 1 PID: 1788 Comm: io_uring-sq Not tainted 5.11.0-rc4 #1
> [ 31.259175] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
> [ 31.260232] RIP: 0010:__lock_acquire+0x19d/0x18c0
> [ 31.261144] Code: 00 00 8b 1d fd 56 dd 08 85 db 0f 85 43 05 00 00 48 c7 c6 98 7b 95 82 48 c7 c7 57 96 93 82 e8 9a bc f5 ff 0f 0b e9 2b 05 00 00 <48> 81 3f c0 ca 67 8a b8 00 00 00 00 41 0f 45 c0 89 04 24 e9 81 fe
> [ 31.264297] RSP: 0018:ffffc90001933828 EFLAGS: 00010002
> [ 31.265320] RAX: 0000000000000001 RBX: 0000000000000001 RCX: 0000000000000000
> [ 31.266594] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 00000000000000d8
> [ 31.267922] RBP: 0000000000000246 R08: 0000000000000001 R09: 0000000000000000
> [ 31.269262] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
> [ 31.270550] R13: 0000000000000000 R14: ffff888106e8a140 R15: 00000000000000d8
> [ 31.271760] FS: 0000000000000000(0000) GS:ffff88813bd00000(0000) knlGS:0000000000000000
> [ 31.273269] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 31.274330] CR2: 00000000000000d8 CR3: 0000000106efa004 CR4: 00000000003706e0
> [ 31.275613] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 31.276855] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> [ 31.278065] Call Trace:
> [ 31.278649] lock_acquire+0x31a/0x440
> [ 31.279404] ? close_fd_get_file+0x39/0x160
> [ 31.280276] ? __lock_acquire+0x647/0x18c0
> [ 31.281112] _raw_spin_lock+0x2c/0x40
> [ 31.281821] ? close_fd_get_file+0x39/0x160
> [ 31.282586] close_fd_get_file+0x39/0x160
> [ 31.283338] io_issue_sqe+0x1334/0x14e0
> [ 31.284053] ? lock_acquire+0x31a/0x440
> [ 31.284763] ? __io_free_req+0xcf/0x2e0
> [ 31.285504] ? __io_free_req+0x175/0x2e0
> [ 31.286247] ? find_held_lock+0x28/0xb0
> [ 31.286968] ? io_wq_submit_work+0x7f/0x240
> [ 31.287733] io_wq_submit_work+0x7f/0x240
> [ 31.288486] io_wq_cancel_cb+0x161/0x580
> [ 31.289230] ? io_wqe_wake_worker+0x114/0x360
> [ 31.290020] ? io_uring_get_socket+0x40/0x40
> [ 31.290832] io_async_find_and_cancel+0x3b/0x140
> [ 31.291676] io_issue_sqe+0xbe1/0x14e0
> [ 31.292405] ? __lock_acquire+0x647/0x18c0
> [ 31.293207] ? __io_queue_sqe+0x10b/0x5f0
> [ 31.293986] __io_queue_sqe+0x10b/0x5f0
> [ 31.294747] ? io_req_prep+0xdb/0x1150
> [ 31.295485] ? mark_held_locks+0x6d/0xb0
> [ 31.296252] ? mark_held_locks+0x6d/0xb0
> [ 31.297019] ? io_queue_sqe+0x235/0x4b0
> [ 31.297774] io_queue_sqe+0x235/0x4b0
> [ 31.298496] io_submit_sqes+0xd7e/0x12a0
> [ 31.299275] ? _raw_spin_unlock_irq+0x24/0x30
> [ 31.300121] ? io_sq_thread+0x3ae/0x940
> [ 31.300873] io_sq_thread+0x207/0x940
> [ 31.301606] ? do_wait_intr_irq+0xc0/0xc0
> [ 31.302396] ? __ia32_sys_io_uring_enter+0x650/0x650
> [ 31.303321] kthread+0x134/0x180
> [ 31.303982] ? kthread_create_worker_on_cpu+0x90/0x90
> [ 31.304886] ret_from_fork+0x1f/0x30
>
> This is caused by NULL files when async cancel close, which has
> IO_WQ_WORK_NO_CANCEL set and continue to do work. Fix it by also setting
> needs_files for IORING_OP_ASYNC_CANCEL.
Looks good enough for a quick late-rc fix,
Reviewed-by: Pavel Begunkov <[email protected]>
But we need need to rework this NO_CANCEL case for the future. A
reproducer would help much, do you have one? Or even better a liburing
test?
>
> Reported-by: Abaci <[email protected]>
> Cc: [email protected] # 5.6+
> Signed-off-by: Joseph Qi <[email protected]>
> ---
> fs/io_uring.c | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/fs/io_uring.c b/fs/io_uring.c
> index 985a9e3..8eb1349 100644
> --- a/fs/io_uring.c
> +++ b/fs/io_uring.c
> @@ -883,7 +883,10 @@ struct io_op_def {
> .pollin = 1,
> .work_flags = IO_WQ_WORK_MM | IO_WQ_WORK_FILES,
> },
> - [IORING_OP_ASYNC_CANCEL] = {},
> + [IORING_OP_ASYNC_CANCEL] = {
> + /* for async cancel close */
> + .needs_file = 1,
> + },
> [IORING_OP_LINK_TIMEOUT] = {
> .needs_async_data = 1,
> .async_size = sizeof(struct io_timeout_data),
>
--
Pavel Begunkov
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] io_uring: fix NULL pointer dereference for async cancel close
2021-01-18 12:23 ` Pavel Begunkov
@ 2021-01-18 15:08 ` Pavel Begunkov
2021-01-19 1:58 ` Joseph Qi
2021-01-19 2:08 ` Joseph Qi
1 sibling, 1 reply; 12+ messages in thread
From: Pavel Begunkov @ 2021-01-18 15:08 UTC (permalink / raw)
To: Joseph Qi, Jens Axboe; +Cc: io-uring, Xiaoguang Wang
On 18/01/2021 12:23, Pavel Begunkov wrote:
> On 18/01/2021 09:50, Joseph Qi wrote:
>> Abaci reported the following crash:
>>
>> [ 31.252589] BUG: kernel NULL pointer dereference, address: 00000000000000d8
>> [ 31.253942] #PF: supervisor read access in kernel mode
>> [ 31.254945] #PF: error_code(0x0000) - not-present page
>> [ 31.255964] PGD 800000010b76f067 P4D 800000010b76f067 PUD 10b462067 PMD 0
>> [ 31.257221] Oops: 0000 [#1] SMP PTI
>> [ 31.257923] CPU: 1 PID: 1788 Comm: io_uring-sq Not tainted 5.11.0-rc4 #1
>> [ 31.259175] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
>> [ 31.260232] RIP: 0010:__lock_acquire+0x19d/0x18c0
>> [ 31.261144] Code: 00 00 8b 1d fd 56 dd 08 85 db 0f 85 43 05 00 00 48 c7 c6 98 7b 95 82 48 c7 c7 57 96 93 82 e8 9a bc f5 ff 0f 0b e9 2b 05 00 00 <48> 81 3f c0 ca 67 8a b8 00 00 00 00 41 0f 45 c0 89 04 24 e9 81 fe
>> [ 31.264297] RSP: 0018:ffffc90001933828 EFLAGS: 00010002
>> [ 31.265320] RAX: 0000000000000001 RBX: 0000000000000001 RCX: 0000000000000000
>> [ 31.266594] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 00000000000000d8
>> [ 31.267922] RBP: 0000000000000246 R08: 0000000000000001 R09: 0000000000000000
>> [ 31.269262] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
>> [ 31.270550] R13: 0000000000000000 R14: ffff888106e8a140 R15: 00000000000000d8
>> [ 31.271760] FS: 0000000000000000(0000) GS:ffff88813bd00000(0000) knlGS:0000000000000000
>> [ 31.273269] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> [ 31.274330] CR2: 00000000000000d8 CR3: 0000000106efa004 CR4: 00000000003706e0
>> [ 31.275613] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>> [ 31.276855] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
>> [ 31.278065] Call Trace:
>> [ 31.278649] lock_acquire+0x31a/0x440
>> [ 31.279404] ? close_fd_get_file+0x39/0x160
>> [ 31.280276] ? __lock_acquire+0x647/0x18c0
>> [ 31.281112] _raw_spin_lock+0x2c/0x40
>> [ 31.281821] ? close_fd_get_file+0x39/0x160
>> [ 31.282586] close_fd_get_file+0x39/0x160
>> [ 31.283338] io_issue_sqe+0x1334/0x14e0
>> [ 31.284053] ? lock_acquire+0x31a/0x440
>> [ 31.284763] ? __io_free_req+0xcf/0x2e0
>> [ 31.285504] ? __io_free_req+0x175/0x2e0
>> [ 31.286247] ? find_held_lock+0x28/0xb0
>> [ 31.286968] ? io_wq_submit_work+0x7f/0x240
>> [ 31.287733] io_wq_submit_work+0x7f/0x240
>> [ 31.288486] io_wq_cancel_cb+0x161/0x580
>> [ 31.289230] ? io_wqe_wake_worker+0x114/0x360
>> [ 31.290020] ? io_uring_get_socket+0x40/0x40
>> [ 31.290832] io_async_find_and_cancel+0x3b/0x140
>> [ 31.291676] io_issue_sqe+0xbe1/0x14e0
>> [ 31.292405] ? __lock_acquire+0x647/0x18c0
>> [ 31.293207] ? __io_queue_sqe+0x10b/0x5f0
>> [ 31.293986] __io_queue_sqe+0x10b/0x5f0
>> [ 31.294747] ? io_req_prep+0xdb/0x1150
>> [ 31.295485] ? mark_held_locks+0x6d/0xb0
>> [ 31.296252] ? mark_held_locks+0x6d/0xb0
>> [ 31.297019] ? io_queue_sqe+0x235/0x4b0
>> [ 31.297774] io_queue_sqe+0x235/0x4b0
>> [ 31.298496] io_submit_sqes+0xd7e/0x12a0
>> [ 31.299275] ? _raw_spin_unlock_irq+0x24/0x30
>> [ 31.300121] ? io_sq_thread+0x3ae/0x940
>> [ 31.300873] io_sq_thread+0x207/0x940
>> [ 31.301606] ? do_wait_intr_irq+0xc0/0xc0
>> [ 31.302396] ? __ia32_sys_io_uring_enter+0x650/0x650
>> [ 31.303321] kthread+0x134/0x180
>> [ 31.303982] ? kthread_create_worker_on_cpu+0x90/0x90
>> [ 31.304886] ret_from_fork+0x1f/0x30
>>
>> This is caused by NULL files when async cancel close, which has
>> IO_WQ_WORK_NO_CANCEL set and continue to do work. Fix it by also setting
>> needs_files for IORING_OP_ASYNC_CANCEL.
>
> Looks good enough for a quick late-rc fix,
> Reviewed-by: Pavel Begunkov <[email protected]>
Hmm, I hastened, for files we need IO_WQ_WORK_FILES,
+IO_WQ_WORK_BLKCG for same reasons. needs_file would make
it to grab a struct file, that is wrong.
Probably worked out because it just grabbed fd=0/stdin.
>
> But we need need to rework this NO_CANCEL case for the future. A
> reproducer would help much, do you have one? Or even better a liburing
> test?
>
>>
>> Reported-by: Abaci <[email protected]>
>> Cc: [email protected] # 5.6+
>> Signed-off-by: Joseph Qi <[email protected]>
>> ---
>> fs/io_uring.c | 5 ++++-
>> 1 file changed, 4 insertions(+), 1 deletion(-)
>>
>> diff --git a/fs/io_uring.c b/fs/io_uring.c
>> index 985a9e3..8eb1349 100644
>> --- a/fs/io_uring.c
>> +++ b/fs/io_uring.c
>> @@ -883,7 +883,10 @@ struct io_op_def {
>> .pollin = 1,
>> .work_flags = IO_WQ_WORK_MM | IO_WQ_WORK_FILES,
>> },
>> - [IORING_OP_ASYNC_CANCEL] = {},
>> + [IORING_OP_ASYNC_CANCEL] = {
>> + /* for async cancel close */
>> + .needs_file = 1,
>> + },
>> [IORING_OP_LINK_TIMEOUT] = {
>> .needs_async_data = 1,
>> .async_size = sizeof(struct io_timeout_data),
>>
>
--
Pavel Begunkov
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] io_uring: fix NULL pointer dereference for async cancel close
2021-01-18 15:08 ` Pavel Begunkov
@ 2021-01-19 1:58 ` Joseph Qi
2021-01-19 2:38 ` Pavel Begunkov
0 siblings, 1 reply; 12+ messages in thread
From: Joseph Qi @ 2021-01-19 1:58 UTC (permalink / raw)
To: Pavel Begunkov, Jens Axboe; +Cc: io-uring, Xiaoguang Wang
On 1/18/21 11:08 PM, Pavel Begunkov wrote:
> On 18/01/2021 12:23, Pavel Begunkov wrote:
>> On 18/01/2021 09:50, Joseph Qi wrote:
>>> Abaci reported the following crash:
>>>
>>> [ 31.252589] BUG: kernel NULL pointer dereference, address: 00000000000000d8
>>> [ 31.253942] #PF: supervisor read access in kernel mode
>>> [ 31.254945] #PF: error_code(0x0000) - not-present page
>>> [ 31.255964] PGD 800000010b76f067 P4D 800000010b76f067 PUD 10b462067 PMD 0
>>> [ 31.257221] Oops: 0000 [#1] SMP PTI
>>> [ 31.257923] CPU: 1 PID: 1788 Comm: io_uring-sq Not tainted 5.11.0-rc4 #1
>>> [ 31.259175] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
>>> [ 31.260232] RIP: 0010:__lock_acquire+0x19d/0x18c0
>>> [ 31.261144] Code: 00 00 8b 1d fd 56 dd 08 85 db 0f 85 43 05 00 00 48 c7 c6 98 7b 95 82 48 c7 c7 57 96 93 82 e8 9a bc f5 ff 0f 0b e9 2b 05 00 00 <48> 81 3f c0 ca 67 8a b8 00 00 00 00 41 0f 45 c0 89 04 24 e9 81 fe
>>> [ 31.264297] RSP: 0018:ffffc90001933828 EFLAGS: 00010002
>>> [ 31.265320] RAX: 0000000000000001 RBX: 0000000000000001 RCX: 0000000000000000
>>> [ 31.266594] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 00000000000000d8
>>> [ 31.267922] RBP: 0000000000000246 R08: 0000000000000001 R09: 0000000000000000
>>> [ 31.269262] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
>>> [ 31.270550] R13: 0000000000000000 R14: ffff888106e8a140 R15: 00000000000000d8
>>> [ 31.271760] FS: 0000000000000000(0000) GS:ffff88813bd00000(0000) knlGS:0000000000000000
>>> [ 31.273269] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>> [ 31.274330] CR2: 00000000000000d8 CR3: 0000000106efa004 CR4: 00000000003706e0
>>> [ 31.275613] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>>> [ 31.276855] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
>>> [ 31.278065] Call Trace:
>>> [ 31.278649] lock_acquire+0x31a/0x440
>>> [ 31.279404] ? close_fd_get_file+0x39/0x160
>>> [ 31.280276] ? __lock_acquire+0x647/0x18c0
>>> [ 31.281112] _raw_spin_lock+0x2c/0x40
>>> [ 31.281821] ? close_fd_get_file+0x39/0x160
>>> [ 31.282586] close_fd_get_file+0x39/0x160
>>> [ 31.283338] io_issue_sqe+0x1334/0x14e0
>>> [ 31.284053] ? lock_acquire+0x31a/0x440
>>> [ 31.284763] ? __io_free_req+0xcf/0x2e0
>>> [ 31.285504] ? __io_free_req+0x175/0x2e0
>>> [ 31.286247] ? find_held_lock+0x28/0xb0
>>> [ 31.286968] ? io_wq_submit_work+0x7f/0x240
>>> [ 31.287733] io_wq_submit_work+0x7f/0x240
>>> [ 31.288486] io_wq_cancel_cb+0x161/0x580
>>> [ 31.289230] ? io_wqe_wake_worker+0x114/0x360
>>> [ 31.290020] ? io_uring_get_socket+0x40/0x40
>>> [ 31.290832] io_async_find_and_cancel+0x3b/0x140
>>> [ 31.291676] io_issue_sqe+0xbe1/0x14e0
>>> [ 31.292405] ? __lock_acquire+0x647/0x18c0
>>> [ 31.293207] ? __io_queue_sqe+0x10b/0x5f0
>>> [ 31.293986] __io_queue_sqe+0x10b/0x5f0
>>> [ 31.294747] ? io_req_prep+0xdb/0x1150
>>> [ 31.295485] ? mark_held_locks+0x6d/0xb0
>>> [ 31.296252] ? mark_held_locks+0x6d/0xb0
>>> [ 31.297019] ? io_queue_sqe+0x235/0x4b0
>>> [ 31.297774] io_queue_sqe+0x235/0x4b0
>>> [ 31.298496] io_submit_sqes+0xd7e/0x12a0
>>> [ 31.299275] ? _raw_spin_unlock_irq+0x24/0x30
>>> [ 31.300121] ? io_sq_thread+0x3ae/0x940
>>> [ 31.300873] io_sq_thread+0x207/0x940
>>> [ 31.301606] ? do_wait_intr_irq+0xc0/0xc0
>>> [ 31.302396] ? __ia32_sys_io_uring_enter+0x650/0x650
>>> [ 31.303321] kthread+0x134/0x180
>>> [ 31.303982] ? kthread_create_worker_on_cpu+0x90/0x90
>>> [ 31.304886] ret_from_fork+0x1f/0x30
>>>
>>> This is caused by NULL files when async cancel close, which has
>>> IO_WQ_WORK_NO_CANCEL set and continue to do work. Fix it by also setting
>>> needs_files for IORING_OP_ASYNC_CANCEL.
>>
>> Looks good enough for a quick late-rc fix,
>> Reviewed-by: Pavel Begunkov <[email protected]>
>
> Hmm, I hastened, for files we need IO_WQ_WORK_FILES,
> +IO_WQ_WORK_BLKCG for same reasons. needs_file would make
> it to grab a struct file, that is wrong.
> Probably worked out because it just grabbed fd=0/stdin.
>
I think IO_WQ_WORK_FILES can work since it will acquire
files when initialize async cancel request.
Don't quite understand why we should have IO_WQ_WORK_BLKCG.
Thanks,
Joseph
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] io_uring: fix NULL pointer dereference for async cancel close
2021-01-18 12:23 ` Pavel Begunkov
2021-01-18 15:08 ` Pavel Begunkov
@ 2021-01-19 2:08 ` Joseph Qi
1 sibling, 0 replies; 12+ messages in thread
From: Joseph Qi @ 2021-01-19 2:08 UTC (permalink / raw)
To: Pavel Begunkov, Joseph Qi, Jens Axboe; +Cc: io-uring, Xiaoguang Wang
On 1/18/21 8:23 PM, Pavel Begunkov wrote:
> On 18/01/2021 09:50, Joseph Qi wrote:
>> Abaci reported the following crash:
>>
>> [ 31.252589] BUG: kernel NULL pointer dereference, address: 00000000000000d8
>> [ 31.253942] #PF: supervisor read access in kernel mode
>> [ 31.254945] #PF: error_code(0x0000) - not-present page
>> [ 31.255964] PGD 800000010b76f067 P4D 800000010b76f067 PUD 10b462067 PMD 0
>> [ 31.257221] Oops: 0000 [#1] SMP PTI
>> [ 31.257923] CPU: 1 PID: 1788 Comm: io_uring-sq Not tainted 5.11.0-rc4 #1
>> [ 31.259175] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
>> [ 31.260232] RIP: 0010:__lock_acquire+0x19d/0x18c0
>> [ 31.261144] Code: 00 00 8b 1d fd 56 dd 08 85 db 0f 85 43 05 00 00 48 c7 c6 98 7b 95 82 48 c7 c7 57 96 93 82 e8 9a bc f5 ff 0f 0b e9 2b 05 00 00 <48> 81 3f c0 ca 67 8a b8 00 00 00 00 41 0f 45 c0 89 04 24 e9 81 fe
>> [ 31.264297] RSP: 0018:ffffc90001933828 EFLAGS: 00010002
>> [ 31.265320] RAX: 0000000000000001 RBX: 0000000000000001 RCX: 0000000000000000
>> [ 31.266594] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 00000000000000d8
>> [ 31.267922] RBP: 0000000000000246 R08: 0000000000000001 R09: 0000000000000000
>> [ 31.269262] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
>> [ 31.270550] R13: 0000000000000000 R14: ffff888106e8a140 R15: 00000000000000d8
>> [ 31.271760] FS: 0000000000000000(0000) GS:ffff88813bd00000(0000) knlGS:0000000000000000
>> [ 31.273269] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> [ 31.274330] CR2: 00000000000000d8 CR3: 0000000106efa004 CR4: 00000000003706e0
>> [ 31.275613] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>> [ 31.276855] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
>> [ 31.278065] Call Trace:
>> [ 31.278649] lock_acquire+0x31a/0x440
>> [ 31.279404] ? close_fd_get_file+0x39/0x160
>> [ 31.280276] ? __lock_acquire+0x647/0x18c0
>> [ 31.281112] _raw_spin_lock+0x2c/0x40
>> [ 31.281821] ? close_fd_get_file+0x39/0x160
>> [ 31.282586] close_fd_get_file+0x39/0x160
>> [ 31.283338] io_issue_sqe+0x1334/0x14e0
>> [ 31.284053] ? lock_acquire+0x31a/0x440
>> [ 31.284763] ? __io_free_req+0xcf/0x2e0
>> [ 31.285504] ? __io_free_req+0x175/0x2e0
>> [ 31.286247] ? find_held_lock+0x28/0xb0
>> [ 31.286968] ? io_wq_submit_work+0x7f/0x240
>> [ 31.287733] io_wq_submit_work+0x7f/0x240
>> [ 31.288486] io_wq_cancel_cb+0x161/0x580
>> [ 31.289230] ? io_wqe_wake_worker+0x114/0x360
>> [ 31.290020] ? io_uring_get_socket+0x40/0x40
>> [ 31.290832] io_async_find_and_cancel+0x3b/0x140
>> [ 31.291676] io_issue_sqe+0xbe1/0x14e0
>> [ 31.292405] ? __lock_acquire+0x647/0x18c0
>> [ 31.293207] ? __io_queue_sqe+0x10b/0x5f0
>> [ 31.293986] __io_queue_sqe+0x10b/0x5f0
>> [ 31.294747] ? io_req_prep+0xdb/0x1150
>> [ 31.295485] ? mark_held_locks+0x6d/0xb0
>> [ 31.296252] ? mark_held_locks+0x6d/0xb0
>> [ 31.297019] ? io_queue_sqe+0x235/0x4b0
>> [ 31.297774] io_queue_sqe+0x235/0x4b0
>> [ 31.298496] io_submit_sqes+0xd7e/0x12a0
>> [ 31.299275] ? _raw_spin_unlock_irq+0x24/0x30
>> [ 31.300121] ? io_sq_thread+0x3ae/0x940
>> [ 31.300873] io_sq_thread+0x207/0x940
>> [ 31.301606] ? do_wait_intr_irq+0xc0/0xc0
>> [ 31.302396] ? __ia32_sys_io_uring_enter+0x650/0x650
>> [ 31.303321] kthread+0x134/0x180
>> [ 31.303982] ? kthread_create_worker_on_cpu+0x90/0x90
>> [ 31.304886] ret_from_fork+0x1f/0x30
>>
>> This is caused by NULL files when async cancel close, which has
>> IO_WQ_WORK_NO_CANCEL set and continue to do work. Fix it by also setting
>> needs_files for IORING_OP_ASYNC_CANCEL.
>
> Looks good enough for a quick late-rc fix,
> Reviewed-by: Pavel Begunkov <[email protected]>
>
> But we need need to rework this NO_CANCEL case for the future. A
> reproducer would help much, do you have one? Or even better a liburing
> test?
>
Yes, it's a syzkaller producer, I'll try add a liburing test case later.
Thanks,
Joseph
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] io_uring: fix NULL pointer dereference for async cancel close
2021-01-19 1:58 ` Joseph Qi
@ 2021-01-19 2:38 ` Pavel Begunkov
2021-01-19 8:00 ` Joseph Qi
0 siblings, 1 reply; 12+ messages in thread
From: Pavel Begunkov @ 2021-01-19 2:38 UTC (permalink / raw)
To: Joseph Qi, Jens Axboe; +Cc: io-uring, Xiaoguang Wang
On 19/01/2021 01:58, Joseph Qi wrote:
>> Hmm, I hastened, for files we need IO_WQ_WORK_FILES,
>> +IO_WQ_WORK_BLKCG for same reasons. needs_file would make
>> it to grab a struct file, that is wrong.
>> Probably worked out because it just grabbed fd=0/stdin.
>>
>
> I think IO_WQ_WORK_FILES can work since it will acquire
> files when initialize async cancel request.
That the one controlling files in the first place, need_file
just happened to grab them submission.
> Don't quite understand why we should have IO_WQ_WORK_BLKCG.
Because it's set for IORING_OP_CLOSE, and similar situation
may happen but with async_cancel from io-wq.
Actually, it's even nastier than that, and neither of io_op_def
flags would work because for io-wq case you can end up doing
close() with different from original files. I'll think how it
can be done tomorrow.
--
Pavel Begunkov
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] io_uring: fix NULL pointer dereference for async cancel close
2021-01-19 2:38 ` Pavel Begunkov
@ 2021-01-19 8:00 ` Joseph Qi
2021-01-19 11:45 ` Pavel Begunkov
0 siblings, 1 reply; 12+ messages in thread
From: Joseph Qi @ 2021-01-19 8:00 UTC (permalink / raw)
To: Pavel Begunkov, Joseph Qi, Jens Axboe; +Cc: io-uring, Xiaoguang Wang
On 1/19/21 10:38 AM, Pavel Begunkov wrote:
> On 19/01/2021 01:58, Joseph Qi wrote:
>>> Hmm, I hastened, for files we need IO_WQ_WORK_FILES,
>>> +IO_WQ_WORK_BLKCG for same reasons. needs_file would make
>>> it to grab a struct file, that is wrong.
>>> Probably worked out because it just grabbed fd=0/stdin.
>>>
>>
>> I think IO_WQ_WORK_FILES can work since it will acquire
>> files when initialize async cancel request.
>
> That the one controlling files in the first place, need_file
> just happened to grab them submission.
>
>> Don't quite understand why we should have IO_WQ_WORK_BLKCG.
>
> Because it's set for IORING_OP_CLOSE, and similar situation
> may happen but with async_cancel from io-wq.
>
So how about do switch and restore in io_run_cancel(), seems it can
take care of direct request, sqthread and io-wq cases.
Thanks,
Joseph
> Actually, it's even nastier than that, and neither of io_op_def
> flags would work because for io-wq case you can end up doing
> close() with different from original files. I'll think how it
> can be done tomorrow.
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] io_uring: fix NULL pointer dereference for async cancel close
2021-01-19 8:00 ` Joseph Qi
@ 2021-01-19 11:45 ` Pavel Begunkov
2021-01-19 13:12 ` Joseph Qi
0 siblings, 1 reply; 12+ messages in thread
From: Pavel Begunkov @ 2021-01-19 11:45 UTC (permalink / raw)
To: Joseph Qi, Joseph Qi, Jens Axboe; +Cc: io-uring, Xiaoguang Wang
On 19/01/2021 08:00, Joseph Qi wrote:
>
>
> On 1/19/21 10:38 AM, Pavel Begunkov wrote:
>> On 19/01/2021 01:58, Joseph Qi wrote:
>>>> Hmm, I hastened, for files we need IO_WQ_WORK_FILES,
>>>> +IO_WQ_WORK_BLKCG for same reasons. needs_file would make
>>>> it to grab a struct file, that is wrong.
>>>> Probably worked out because it just grabbed fd=0/stdin.
>>>>
>>>
>>> I think IO_WQ_WORK_FILES can work since it will acquire
>>> files when initialize async cancel request.
>>
>> That the one controlling files in the first place, need_file
>> just happened to grab them submission.
>>
>>> Don't quite understand why we should have IO_WQ_WORK_BLKCG.
>>
>> Because it's set for IORING_OP_CLOSE, and similar situation
>> may happen but with async_cancel from io-wq.
>>
> So how about do switch and restore in io_run_cancel(), seems it can
> take care of direct request, sqthread and io-wq cases.
It will get ugly pretty quickly, + this nesting of io-wq handlers
async_handler() -> io_close() is not great...
I'm more inclined to skip them in io_wqe_cancel_pending_work()
to not execute inline. That may need to do some waiting on the
async_cancel side though to not change the semantics. Can you
try out this direction?
>
>> Actually, it's even nastier than that, and neither of io_op_def
>> flags would work because for io-wq case you can end up doing
>> close() with different from original files. I'll think how it
>> can be done tomorrow.
>>
--
Pavel Begunkov
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] io_uring: fix NULL pointer dereference for async cancel close
2021-01-19 11:45 ` Pavel Begunkov
@ 2021-01-19 13:12 ` Joseph Qi
2021-01-19 13:39 ` Pavel Begunkov
0 siblings, 1 reply; 12+ messages in thread
From: Joseph Qi @ 2021-01-19 13:12 UTC (permalink / raw)
To: Pavel Begunkov, Joseph Qi, Jens Axboe; +Cc: io-uring, Xiaoguang Wang
On 1/19/21 7:45 PM, Pavel Begunkov wrote:
> On 19/01/2021 08:00, Joseph Qi wrote:
>>
>>
>> On 1/19/21 10:38 AM, Pavel Begunkov wrote:
>>> On 19/01/2021 01:58, Joseph Qi wrote:
>>>>> Hmm, I hastened, for files we need IO_WQ_WORK_FILES,
>>>>> +IO_WQ_WORK_BLKCG for same reasons. needs_file would make
>>>>> it to grab a struct file, that is wrong.
>>>>> Probably worked out because it just grabbed fd=0/stdin.
>>>>>
>>>>
>>>> I think IO_WQ_WORK_FILES can work since it will acquire
>>>> files when initialize async cancel request.
>>>
>>> That the one controlling files in the first place, need_file
>>> just happened to grab them submission.
>>>
>>>> Don't quite understand why we should have IO_WQ_WORK_BLKCG.
>>>
>>> Because it's set for IORING_OP_CLOSE, and similar situation
>>> may happen but with async_cancel from io-wq.
>>>
>> So how about do switch and restore in io_run_cancel(), seems it can
>> take care of direct request, sqthread and io-wq cases.
>
> It will get ugly pretty quickly, + this nesting of io-wq handlers
> async_handler() -> io_close() is not great...
>
> I'm more inclined to skip them in io_wqe_cancel_pending_work()
> to not execute inline. That may need to do some waiting on the
> async_cancel side though to not change the semantics. Can you
> try out this direction?
>
Sure, I'll try this way and send v2.
Thanks,
Joseph
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] io_uring: fix NULL pointer dereference for async cancel close
2021-01-19 13:12 ` Joseph Qi
@ 2021-01-19 13:39 ` Pavel Begunkov
2021-01-19 16:37 ` Pavel Begunkov
0 siblings, 1 reply; 12+ messages in thread
From: Pavel Begunkov @ 2021-01-19 13:39 UTC (permalink / raw)
To: Joseph Qi, Joseph Qi, Jens Axboe; +Cc: io-uring, Xiaoguang Wang
On 19/01/2021 13:12, Joseph Qi wrote:
> On 1/19/21 7:45 PM, Pavel Begunkov wrote:
>> On 19/01/2021 08:00, Joseph Qi wrote:
>>> On 1/19/21 10:38 AM, Pavel Begunkov wrote:
>>>> On 19/01/2021 01:58, Joseph Qi wrote:
>>>>>> Hmm, I hastened, for files we need IO_WQ_WORK_FILES,
>>>>>> +IO_WQ_WORK_BLKCG for same reasons. needs_file would make
>>>>>> it to grab a struct file, that is wrong.
>>>>>> Probably worked out because it just grabbed fd=0/stdin.
>>>>>>
>>>>>
>>>>> I think IO_WQ_WORK_FILES can work since it will acquire
>>>>> files when initialize async cancel request.
>>>>
>>>> That the one controlling files in the first place, need_file
>>>> just happened to grab them submission.
>>>>
>>>>> Don't quite understand why we should have IO_WQ_WORK_BLKCG.
>>>>
>>>> Because it's set for IORING_OP_CLOSE, and similar situation
>>>> may happen but with async_cancel from io-wq.
>>>>
>>> So how about do switch and restore in io_run_cancel(), seems it can
>>> take care of direct request, sqthread and io-wq cases.
>>
>> It will get ugly pretty quickly, + this nesting of io-wq handlers
>> async_handler() -> io_close() is not great...
>>
>> I'm more inclined to skip them in io_wqe_cancel_pending_work()
>> to not execute inline. That may need to do some waiting on the
>> async_cancel side though to not change the semantics. Can you
>> try out this direction?
>>
> Sure, I'll try this way and send v2.
There may be a much better way, that's to remove IO_WQ_WORK_NO_CANCEL
and move -EAGAIN section of io_close() before close_fd_get_file(),
so not splitting it in 2 and not keeping it half-done.
IIRC, it was done this way because of historical reasons when we
didn't put more stuff around files, but may be wrong.
Jens, do you remember what it was?
--
Pavel Begunkov
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] io_uring: fix NULL pointer dereference for async cancel close
2021-01-19 13:39 ` Pavel Begunkov
@ 2021-01-19 16:37 ` Pavel Begunkov
0 siblings, 0 replies; 12+ messages in thread
From: Pavel Begunkov @ 2021-01-19 16:37 UTC (permalink / raw)
To: Joseph Qi, Joseph Qi, Jens Axboe; +Cc: io-uring, Xiaoguang Wang
On 19/01/2021 13:39, Pavel Begunkov wrote:
> On 19/01/2021 13:12, Joseph Qi wrote:
>> On 1/19/21 7:45 PM, Pavel Begunkov wrote:
>>> On 19/01/2021 08:00, Joseph Qi wrote:
>>>> On 1/19/21 10:38 AM, Pavel Begunkov wrote:
>>>>> On 19/01/2021 01:58, Joseph Qi wrote:
>>>>>>> Hmm, I hastened, for files we need IO_WQ_WORK_FILES,
>>>>>>> +IO_WQ_WORK_BLKCG for same reasons. needs_file would make
>>>>>>> it to grab a struct file, that is wrong.
>>>>>>> Probably worked out because it just grabbed fd=0/stdin.
>>>>>>>
>>>>>>
>>>>>> I think IO_WQ_WORK_FILES can work since it will acquire
>>>>>> files when initialize async cancel request.
>>>>>
>>>>> That the one controlling files in the first place, need_file
>>>>> just happened to grab them submission.
>>>>>
>>>>>> Don't quite understand why we should have IO_WQ_WORK_BLKCG.
>>>>>
>>>>> Because it's set for IORING_OP_CLOSE, and similar situation
>>>>> may happen but with async_cancel from io-wq.
>>>>>
>>>> So how about do switch and restore in io_run_cancel(), seems it can
>>>> take care of direct request, sqthread and io-wq cases.
>>>
>>> It will get ugly pretty quickly, + this nesting of io-wq handlers
>>> async_handler() -> io_close() is not great...
>>>
>>> I'm more inclined to skip them in io_wqe_cancel_pending_work()
>>> to not execute inline. That may need to do some waiting on the
>>> async_cancel side though to not change the semantics. Can you
>>> try out this direction?
>>>
>> Sure, I'll try this way and send v2.
>
> There may be a much better way, that's to remove IO_WQ_WORK_NO_CANCEL
> and move -EAGAIN section of io_close() before close_fd_get_file(),
> so not splitting it in 2 and not keeping it half-done.
I believe it is the right way, but there are tricks to that. I hope
you don't mind me and Jens hijacking taking care of it. Enough of
non-technical hassle expected...
Thanks for reporting it!
>
> IIRC, it was done this way because of historical reasons when we
> didn't put more stuff around files, but may be wrong.
> Jens, do you remember what it was?
--
Pavel Begunkov
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] io_uring: fix NULL pointer dereference for async cancel close
2021-01-18 9:50 [PATCH] io_uring: fix NULL pointer dereference for async cancel close Joseph Qi
2021-01-18 12:23 ` Pavel Begunkov
@ 2021-01-19 18:01 ` Jens Axboe
1 sibling, 0 replies; 12+ messages in thread
From: Jens Axboe @ 2021-01-19 18:01 UTC (permalink / raw)
To: Joseph Qi; +Cc: io-uring, Xiaoguang Wang
On 1/18/21 2:50 AM, Joseph Qi wrote:
> Abaci reported the following crash:
>
> [ 31.252589] BUG: kernel NULL pointer dereference, address: 00000000000000d8
> [ 31.253942] #PF: supervisor read access in kernel mode
> [ 31.254945] #PF: error_code(0x0000) - not-present page
> [ 31.255964] PGD 800000010b76f067 P4D 800000010b76f067 PUD 10b462067 PMD 0
> [ 31.257221] Oops: 0000 [#1] SMP PTI
> [ 31.257923] CPU: 1 PID: 1788 Comm: io_uring-sq Not tainted 5.11.0-rc4 #1
> [ 31.259175] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
> [ 31.260232] RIP: 0010:__lock_acquire+0x19d/0x18c0
> [ 31.261144] Code: 00 00 8b 1d fd 56 dd 08 85 db 0f 85 43 05 00 00 48 c7 c6 98 7b 95 82 48 c7 c7 57 96 93 82 e8 9a bc f5 ff 0f 0b e9 2b 05 00 00 <48> 81 3f c0 ca 67 8a b8 00 00 00 00 41 0f 45 c0 89 04 24 e9 81 fe
> [ 31.264297] RSP: 0018:ffffc90001933828 EFLAGS: 00010002
> [ 31.265320] RAX: 0000000000000001 RBX: 0000000000000001 RCX: 0000000000000000
> [ 31.266594] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 00000000000000d8
> [ 31.267922] RBP: 0000000000000246 R08: 0000000000000001 R09: 0000000000000000
> [ 31.269262] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
> [ 31.270550] R13: 0000000000000000 R14: ffff888106e8a140 R15: 00000000000000d8
> [ 31.271760] FS: 0000000000000000(0000) GS:ffff88813bd00000(0000) knlGS:0000000000000000
> [ 31.273269] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 31.274330] CR2: 00000000000000d8 CR3: 0000000106efa004 CR4: 00000000003706e0
> [ 31.275613] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 31.276855] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> [ 31.278065] Call Trace:
> [ 31.278649] lock_acquire+0x31a/0x440
> [ 31.279404] ? close_fd_get_file+0x39/0x160
> [ 31.280276] ? __lock_acquire+0x647/0x18c0
> [ 31.281112] _raw_spin_lock+0x2c/0x40
> [ 31.281821] ? close_fd_get_file+0x39/0x160
> [ 31.282586] close_fd_get_file+0x39/0x160
> [ 31.283338] io_issue_sqe+0x1334/0x14e0
> [ 31.284053] ? lock_acquire+0x31a/0x440
> [ 31.284763] ? __io_free_req+0xcf/0x2e0
> [ 31.285504] ? __io_free_req+0x175/0x2e0
> [ 31.286247] ? find_held_lock+0x28/0xb0
> [ 31.286968] ? io_wq_submit_work+0x7f/0x240
> [ 31.287733] io_wq_submit_work+0x7f/0x240
> [ 31.288486] io_wq_cancel_cb+0x161/0x580
> [ 31.289230] ? io_wqe_wake_worker+0x114/0x360
> [ 31.290020] ? io_uring_get_socket+0x40/0x40
> [ 31.290832] io_async_find_and_cancel+0x3b/0x140
> [ 31.291676] io_issue_sqe+0xbe1/0x14e0
> [ 31.292405] ? __lock_acquire+0x647/0x18c0
> [ 31.293207] ? __io_queue_sqe+0x10b/0x5f0
> [ 31.293986] __io_queue_sqe+0x10b/0x5f0
> [ 31.294747] ? io_req_prep+0xdb/0x1150
> [ 31.295485] ? mark_held_locks+0x6d/0xb0
> [ 31.296252] ? mark_held_locks+0x6d/0xb0
> [ 31.297019] ? io_queue_sqe+0x235/0x4b0
> [ 31.297774] io_queue_sqe+0x235/0x4b0
> [ 31.298496] io_submit_sqes+0xd7e/0x12a0
> [ 31.299275] ? _raw_spin_unlock_irq+0x24/0x30
> [ 31.300121] ? io_sq_thread+0x3ae/0x940
> [ 31.300873] io_sq_thread+0x207/0x940
> [ 31.301606] ? do_wait_intr_irq+0xc0/0xc0
> [ 31.302396] ? __ia32_sys_io_uring_enter+0x650/0x650
> [ 31.303321] kthread+0x134/0x180
> [ 31.303982] ? kthread_create_worker_on_cpu+0x90/0x90
> [ 31.304886] ret_from_fork+0x1f/0x30
>
> This is caused by NULL files when async cancel close, which has
> IO_WQ_WORK_NO_CANCEL set and continue to do work. Fix it by also setting
> needs_files for IORING_OP_ASYNC_CANCEL.
I posted an alternate fix for this:
[PATCH] io_uring: fix SQPOLL IORING_OP_CLOSE cancelation state
Can you give that a spin?
--
Jens Axboe
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2021-01-20 0:23 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-01-18 9:50 [PATCH] io_uring: fix NULL pointer dereference for async cancel close Joseph Qi
2021-01-18 12:23 ` Pavel Begunkov
2021-01-18 15:08 ` Pavel Begunkov
2021-01-19 1:58 ` Joseph Qi
2021-01-19 2:38 ` Pavel Begunkov
2021-01-19 8:00 ` Joseph Qi
2021-01-19 11:45 ` Pavel Begunkov
2021-01-19 13:12 ` Joseph Qi
2021-01-19 13:39 ` Pavel Begunkov
2021-01-19 16:37 ` Pavel Begunkov
2021-01-19 2:08 ` Joseph Qi
2021-01-19 18:01 ` Jens Axboe
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox