* [bug report] BUG: kernel NULL pointer dereference, address: 0000000000000048
@ 2023-05-10 0:49 Guangwu Zhang
2023-05-10 1:29 ` Yu Kuai
0 siblings, 1 reply; 10+ messages in thread
From: Guangwu Zhang @ 2023-05-10 0:49 UTC (permalink / raw)
To: linux-block, io-uring, Jeff Moyer, Ming Lei
Hi,
We found this kernel NULL pointer issue with latest
linux-block/for-next, please check it.
Kernel repo: https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git
[ 112.483804] BUG: kernel NULL pointer dereference, address: 0000000000000048
[ 112.490809] #PF: supervisor read access in kernel mode
[ 112.495976] #PF: error_code(0x0000) - not-present page
[ 112.501141] PGD 800000044d20c067 P4D 800000044d20c067 PUD 4734d5067 PMD 0
[ 112.508057] Oops: 0000 [#1] PREEMPT SMP PTI
[ 112.512265] CPU: 24 PID: 7767 Comm: user-data Kdump: loaded Not
tainted 6.4.0-rc1+ #1
[ 112.520141] Hardware name: HPE ProLiant DL380 Gen10/ProLiant DL380
Gen10, BIOS U30 06/20/2018
[ 112.528713] RIP: 0010:bfq_bio_bfqg+0x8/0x80
[ 112.532925] Code: 6b 70 48 89 43 60 5b 5d c3 cc cc cc cc 0f 1f 44
00 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 0f 1f 44 00 00
41 54 53 <48> 8b 46 48 48 89 fb 48 89 f7 48 85 c0 74 26 48 63 15 72 40
6b 01
[ 112.551805] RSP: 0018:ffffaed687ef3b30 EFLAGS: 00010096
[ 112.557058] RAX: ffff9a90f2600000 RBX: ffff9a90f2600000 RCX: 0000000000000001
[ 112.564232] RDX: 0000000000000001 RSI: 0000000000000000 RDI: ffff9a90f2600000
[ 112.571408] RBP: ffff9a90c508d500 R08: ffff9a90e2b8a688 R09: ffff9a90e2b8a688
[ 112.578581] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
[ 112.585756] R13: ffff9a90c508d500 R14: 0000000000000000 R15: 0000000000000000
[ 112.592930] FS: 00007fe41b0f0880(0000) GS:ffff9a94afc00000(0000)
knlGS:0000000000000000
[ 112.601065] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 112.606842] CR2: 0000000000000048 CR3: 000000046346e005 CR4: 00000000007706e0
[ 112.614016] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 112.621189] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 112.628362] PKRU: 55555554
[ 112.631082] Call Trace:
[ 112.633539] <TASK>
[ 112.635650] bfq_bic_update_cgroup+0x2c/0x240
[ 112.640033] bfq_init_rq+0xdd/0x670
[ 112.643545] ? blk_rq_map_user_iov+0xc5/0x2f0
[ 112.647931] bfq_insert_request.isra.0+0x5d/0x250
[ 112.652663] bfq_insert_requests+0x59/0x80
[ 112.656782] blk_mq_flush_plug_list+0x172/0x570
[ 112.661342] blk_add_rq_to_plug+0x45/0x150
[ 112.665462] nvme_uring_cmd_io+0x242/0x390 [nvme_core]
[ 112.670652] io_uring_cmd+0x95/0x120
[ 112.674250] io_issue_sqe+0x199/0x3d0
[ 112.677932] io_submit_sqes+0x119/0x3d0
[ 112.681788] __do_sys_io_uring_enter+0x2c2/0x470
[ 112.686433] do_syscall_64+0x59/0x90
[ 112.690031] ? exc_page_fault+0x65/0x150
[ 112.693977] entry_SYSCALL_64_after_hwframe+0x72/0xdc
[ 112.699057] RIP: 0033:0x7fe41ae3ee5d
[ 112.702651] Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e
fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24
08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 93 af 1b 00 f7 d8 64 89
01 48
[ 112.721530] RSP: 002b:00007ffc6fdebc28 EFLAGS: 00000206 ORIG_RAX:
00000000000001aa
[ 112.729143] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fe41ae3ee5d
[ 112.736317] RDX: 0000000000000001 RSI: 0000000000000080 RDI: 0000000000000005
[ 112.743492] RBP: 00007ffc6fdec730 R08: 0000000000000000 R09: 0000000000000080
[ 112.750666] R10: 0000000000000001 R11: 0000000000000206 R12: 00007ffc6fdec848
[ 112.757841] R13: 0000000000401346 R14: 0000000000403de8 R15: 00007fe41b32c000
[ 112.765019] </TASK>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [bug report] BUG: kernel NULL pointer dereference, address: 0000000000000048
2023-05-10 0:49 [bug report] BUG: kernel NULL pointer dereference, address: 0000000000000048 Guangwu Zhang
@ 2023-05-10 1:29 ` Yu Kuai
2023-05-10 1:49 ` Yu Kuai
0 siblings, 1 reply; 10+ messages in thread
From: Yu Kuai @ 2023-05-10 1:29 UTC (permalink / raw)
To: Guangwu Zhang, linux-block, io-uring, Jeff Moyer, Ming Lei,
yukuai (C)
Hi,
在 2023/05/10 8:49, Guangwu Zhang 写道:
> Hi,
>
> We found this kernel NULL pointer issue with latest
> linux-block/for-next, please check it.
>
> Kernel repo: https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git
>
>
> [ 112.483804] BUG: kernel NULL pointer dereference, address: 0000000000000048
> [ 112.490809] #PF: supervisor read access in kernel mode
> [ 112.495976] #PF: error_code(0x0000) - not-present page
> [ 112.501141] PGD 800000044d20c067 P4D 800000044d20c067 PUD 4734d5067 PMD 0
> [ 112.508057] Oops: 0000 [#1] PREEMPT SMP PTI
> [ 112.512265] CPU: 24 PID: 7767 Comm: user-data Kdump: loaded Not
> tainted 6.4.0-rc1+ #1
> [ 112.520141] Hardware name: HPE ProLiant DL380 Gen10/ProLiant DL380
> Gen10, BIOS U30 06/20/2018
> [ 112.528713] RIP: 0010:bfq_bio_bfqg+0x8/0x80
Can you show more details about addr2line result? It'll be much helpful.
Thanks,
Kuai
> [ 112.532925] Code: 6b 70 48 89 43 60 5b 5d c3 cc cc cc cc 0f 1f 44
> 00 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 0f 1f 44 00 00
> 41 54 53 <48> 8b 46 48 48 89 fb 48 89 f7 48 85 c0 74 26 48 63 15 72 40
> 6b 01
> [ 112.551805] RSP: 0018:ffffaed687ef3b30 EFLAGS: 00010096
> [ 112.557058] RAX: ffff9a90f2600000 RBX: ffff9a90f2600000 RCX: 0000000000000001
> [ 112.564232] RDX: 0000000000000001 RSI: 0000000000000000 RDI: ffff9a90f2600000
> [ 112.571408] RBP: ffff9a90c508d500 R08: ffff9a90e2b8a688 R09: ffff9a90e2b8a688
> [ 112.578581] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
> [ 112.585756] R13: ffff9a90c508d500 R14: 0000000000000000 R15: 0000000000000000
> [ 112.592930] FS: 00007fe41b0f0880(0000) GS:ffff9a94afc00000(0000)
> knlGS:0000000000000000
> [ 112.601065] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 112.606842] CR2: 0000000000000048 CR3: 000000046346e005 CR4: 00000000007706e0
> [ 112.614016] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 112.621189] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> [ 112.628362] PKRU: 55555554
> [ 112.631082] Call Trace:
> [ 112.633539] <TASK>
> [ 112.635650] bfq_bic_update_cgroup+0x2c/0x240
> [ 112.640033] bfq_init_rq+0xdd/0x670
> [ 112.643545] ? blk_rq_map_user_iov+0xc5/0x2f0
> [ 112.647931] bfq_insert_request.isra.0+0x5d/0x250
> [ 112.652663] bfq_insert_requests+0x59/0x80
> [ 112.656782] blk_mq_flush_plug_list+0x172/0x570
> [ 112.661342] blk_add_rq_to_plug+0x45/0x150
> [ 112.665462] nvme_uring_cmd_io+0x242/0x390 [nvme_core]
> [ 112.670652] io_uring_cmd+0x95/0x120
> [ 112.674250] io_issue_sqe+0x199/0x3d0
> [ 112.677932] io_submit_sqes+0x119/0x3d0
> [ 112.681788] __do_sys_io_uring_enter+0x2c2/0x470
> [ 112.686433] do_syscall_64+0x59/0x90
> [ 112.690031] ? exc_page_fault+0x65/0x150
> [ 112.693977] entry_SYSCALL_64_after_hwframe+0x72/0xdc
> [ 112.699057] RIP: 0033:0x7fe41ae3ee5d
> [ 112.702651] Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e
> fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24
> 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 93 af 1b 00 f7 d8 64 89
> 01 48
> [ 112.721530] RSP: 002b:00007ffc6fdebc28 EFLAGS: 00000206 ORIG_RAX:
> 00000000000001aa
> [ 112.729143] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fe41ae3ee5d
> [ 112.736317] RDX: 0000000000000001 RSI: 0000000000000080 RDI: 0000000000000005
> [ 112.743492] RBP: 00007ffc6fdec730 R08: 0000000000000000 R09: 0000000000000080
> [ 112.750666] R10: 0000000000000001 R11: 0000000000000206 R12: 00007ffc6fdec848
> [ 112.757841] R13: 0000000000401346 R14: 0000000000403de8 R15: 00007fe41b32c000
> [ 112.765019] </TASK>
>
> .
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [bug report] BUG: kernel NULL pointer dereference, address: 0000000000000048
2023-05-10 1:29 ` Yu Kuai
@ 2023-05-10 1:49 ` Yu Kuai
2023-05-10 2:00 ` Yu Kuai
0 siblings, 1 reply; 10+ messages in thread
From: Yu Kuai @ 2023-05-10 1:49 UTC (permalink / raw)
To: Yu Kuai, Guangwu Zhang, linux-block, io-uring, Jeff Moyer,
Ming Lei, yukuai (C)
Hi,
在 2023/05/10 9:29, Yu Kuai 写道:
> Hi,
>
> 在 2023/05/10 8:49, Guangwu Zhang 写道:
>> Hi,
>>
>> We found this kernel NULL pointer issue with latest
>> linux-block/for-next, please check it.
>>
>> Kernel repo:
>> https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git
>>
>>
>> [ 112.483804] BUG: kernel NULL pointer dereference, address:
>> 0000000000000048
Base on this offset, 0x48 match bio->bi_blkg, so I guess this is because
bio is NULL, so the problem is that passthrough request insert into
elevator.
Can you try follwing patch?
diff --git a/block/blk-mq.c b/block/blk-mq.c
index f6dad0886a2f..fe3ed0a647e6 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -2731,7 +2731,7 @@ static void blk_mq_dispatch_plug_list(struct
blk_plug *plug, bool from_sched)
trace_block_unplug(this_hctx->queue, depth, !from_sched);
percpu_ref_get(&this_hctx->queue->q_usage_counter);
- if (this_hctx->queue->elevator) {
+ if (this_hctx->queue->elevator && !blk_rq_is_passthrough(rq)) {
this_hctx->queue->elevator->type->ops.insert_requests(this_hctx,
&list, 0);
blk_mq_run_hw_queue(this_hctx, from_sched);
Thanks,
Kuai
>> [ 112.490809] #PF: supervisor read access in kernel mode
>> [ 112.495976] #PF: error_code(0x0000) - not-present page
>> [ 112.501141] PGD 800000044d20c067 P4D 800000044d20c067 PUD 4734d5067
>> PMD 0
>> [ 112.508057] Oops: 0000 [#1] PREEMPT SMP PTI
>> [ 112.512265] CPU: 24 PID: 7767 Comm: user-data Kdump: loaded Not
>> tainted 6.4.0-rc1+ #1
>> [ 112.520141] Hardware name: HPE ProLiant DL380 Gen10/ProLiant DL380
>> Gen10, BIOS U30 06/20/2018
>> [ 112.528713] RIP: 0010:bfq_bio_bfqg+0x8/0x80
>
> Can you show more details about addr2line result? It'll be much helpful.
>
> Thanks,
> Kuai
>> [ 112.532925] Code: 6b 70 48 89 43 60 5b 5d c3 cc cc cc cc 0f 1f 44
>> 00 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 0f 1f 44 00 00
>> 41 54 53 <48> 8b 46 48 48 89 fb 48 89 f7 48 85 c0 74 26 48 63 15 72 40
>> 6b 01
>> [ 112.551805] RSP: 0018:ffffaed687ef3b30 EFLAGS: 00010096
>> [ 112.557058] RAX: ffff9a90f2600000 RBX: ffff9a90f2600000 RCX:
>> 0000000000000001
>> [ 112.564232] RDX: 0000000000000001 RSI: 0000000000000000 RDI:
>> ffff9a90f2600000
>> [ 112.571408] RBP: ffff9a90c508d500 R08: ffff9a90e2b8a688 R09:
>> ffff9a90e2b8a688
>> [ 112.578581] R10: 0000000000000000 R11: 0000000000000000 R12:
>> 0000000000000000
>> [ 112.585756] R13: ffff9a90c508d500 R14: 0000000000000000 R15:
>> 0000000000000000
>> [ 112.592930] FS: 00007fe41b0f0880(0000) GS:ffff9a94afc00000(0000)
>> knlGS:0000000000000000
>> [ 112.601065] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> [ 112.606842] CR2: 0000000000000048 CR3: 000000046346e005 CR4:
>> 00000000007706e0
>> [ 112.614016] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
>> 0000000000000000
>> [ 112.621189] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7:
>> 0000000000000400
>> [ 112.628362] PKRU: 55555554
>> [ 112.631082] Call Trace:
>> [ 112.633539] <TASK>
>> [ 112.635650] bfq_bic_update_cgroup+0x2c/0x240
>> [ 112.640033] bfq_init_rq+0xdd/0x670
>> [ 112.643545] ? blk_rq_map_user_iov+0xc5/0x2f0
>> [ 112.647931] bfq_insert_request.isra.0+0x5d/0x250
>> [ 112.652663] bfq_insert_requests+0x59/0x80
>> [ 112.656782] blk_mq_flush_plug_list+0x172/0x570
>> [ 112.661342] blk_add_rq_to_plug+0x45/0x150
>> [ 112.665462] nvme_uring_cmd_io+0x242/0x390 [nvme_core]
>> [ 112.670652] io_uring_cmd+0x95/0x120
>> [ 112.674250] io_issue_sqe+0x199/0x3d0
>> [ 112.677932] io_submit_sqes+0x119/0x3d0
>> [ 112.681788] __do_sys_io_uring_enter+0x2c2/0x470
>> [ 112.686433] do_syscall_64+0x59/0x90
>> [ 112.690031] ? exc_page_fault+0x65/0x150
>> [ 112.693977] entry_SYSCALL_64_after_hwframe+0x72/0xdc
>> [ 112.699057] RIP: 0033:0x7fe41ae3ee5d
>> [ 112.702651] Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e
>> fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24
>> 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 93 af 1b 00 f7 d8 64 89
>> 01 48
>> [ 112.721530] RSP: 002b:00007ffc6fdebc28 EFLAGS: 00000206 ORIG_RAX:
>> 00000000000001aa
>> [ 112.729143] RAX: ffffffffffffffda RBX: 0000000000000001 RCX:
>> 00007fe41ae3ee5d
>> [ 112.736317] RDX: 0000000000000001 RSI: 0000000000000080 RDI:
>> 0000000000000005
>> [ 112.743492] RBP: 00007ffc6fdec730 R08: 0000000000000000 R09:
>> 0000000000000080
>> [ 112.750666] R10: 0000000000000001 R11: 0000000000000206 R12:
>> 00007ffc6fdec848
>> [ 112.757841] R13: 0000000000401346 R14: 0000000000403de8 R15:
>> 00007fe41b32c000
>> [ 112.765019] </TASK>
>>
>> .
>>
>
> .
>
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [bug report] BUG: kernel NULL pointer dereference, address: 0000000000000048
2023-05-10 1:49 ` Yu Kuai
@ 2023-05-10 2:00 ` Yu Kuai
2023-05-10 2:17 ` Jens Axboe
2023-05-10 4:52 ` Guangwu Zhang
0 siblings, 2 replies; 10+ messages in thread
From: Yu Kuai @ 2023-05-10 2:00 UTC (permalink / raw)
To: Yu Kuai, Guangwu Zhang, linux-block, io-uring, Jeff Moyer,
Ming Lei, yukuai (C)
Hi,
在 2023/05/10 9:49, Yu Kuai 写道:
> Hi,
>
> 在 2023/05/10 9:29, Yu Kuai 写道:
>> Hi,
>>
>> 在 2023/05/10 8:49, Guangwu Zhang 写道:
>>> Hi,
>>>
>>> We found this kernel NULL pointer issue with latest
>>> linux-block/for-next, please check it.
>>>
>>> Kernel repo:
>>> https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git
>>>
>>>
>>> [ 112.483804] BUG: kernel NULL pointer dereference, address:
>>> 0000000000000048
>
> Base on this offset, 0x48 match bio->bi_blkg, so I guess this is because
> bio is NULL, so the problem is that passthrough request insert into
> elevator.
>
Sorry that attached patch has some problem, please try this one.
diff --git a/block/blk-mq.c b/block/blk-mq.c
index f6dad0886a2f..bd94d8a5416f 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -2712,6 +2712,7 @@ static void blk_mq_dispatch_plug_list(struct
blk_plug *plug, bool from_sched)
struct request **requeue_lastp = &requeue_list;
unsigned int depth = 0;
LIST_HEAD(list);
+ LIST_HEAD(passthrough_list);
do {
struct request *rq = rq_list_pop(&plug->mq_list);
@@ -2723,7 +2724,10 @@ static void blk_mq_dispatch_plug_list(struct
blk_plug *plug, bool from_sched)
rq_list_add_tail(&requeue_lastp, rq);
continue;
}
- list_add(&rq->queuelist, &list);
+ if (blk_rq_is_passthrough(rq))
+ list_add(&rq->queuelist, &passthrough_list);
+ else
+ list_add(&rq->queuelist, &list);
depth++;
} while (!rq_list_empty(plug->mq_list));
@@ -2731,6 +2735,9 @@ static void blk_mq_dispatch_plug_list(struct
blk_plug *plug, bool from_sched)
trace_block_unplug(this_hctx->queue, depth, !from_sched);
percpu_ref_get(&this_hctx->queue->q_usage_counter);
+ if (!list_empty(&passthrough_list))
+ blk_mq_insert_requests(this_hctx, this_ctx,
&passthrough_list,
+ from_sched);
if (this_hctx->queue->elevator) {
this_hctx->queue->elevator->type->ops.insert_requests(this_hctx,
&list, 0);
Thanks,
Kuai
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [bug report] BUG: kernel NULL pointer dereference, address: 0000000000000048
2023-05-10 2:00 ` Yu Kuai
@ 2023-05-10 2:17 ` Jens Axboe
2023-05-10 3:20 ` Yu Kuai
2023-05-10 4:52 ` Guangwu Zhang
1 sibling, 1 reply; 10+ messages in thread
From: Jens Axboe @ 2023-05-10 2:17 UTC (permalink / raw)
To: Yu Kuai, Guangwu Zhang, linux-block, io-uring, Jeff Moyer,
Ming Lei, yukuai (C)
On 5/9/23 8:00?PM, Yu Kuai wrote:
> Hi,
>
> ? 2023/05/10 9:49, Yu Kuai ??:
>> Hi,
>>
>> ? 2023/05/10 9:29, Yu Kuai ??:
>>> Hi,
>>>
>>> ? 2023/05/10 8:49, Guangwu Zhang ??:
>>>> Hi,
>>>>
>>>> We found this kernel NULL pointer issue with latest
>>>> linux-block/for-next, please check it.
>>>>
>>>> Kernel repo: https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git
>>>>
>>>>
>>>> [ 112.483804] BUG: kernel NULL pointer dereference, address: 0000000000000048
>>
>> Base on this offset, 0x48 match bio->bi_blkg, so I guess this is because
>> bio is NULL, so the problem is that passthrough request insert into
>> elevator.
>>
> Sorry that attached patch has some problem, please try this one.
Let's please fix this in bfq, this isn't a core issue and it's not a
good idea to work around it there.
--
Jens Axboe
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [bug report] BUG: kernel NULL pointer dereference, address: 0000000000000048
2023-05-10 2:17 ` Jens Axboe
@ 2023-05-10 3:20 ` Yu Kuai
[not found] ` <CAGS2=YocNy9PkgRzzRdHAK1gGdjMxzA--PYS=sPrX_nCe4U6QA@mail.gmail.com>
0 siblings, 1 reply; 10+ messages in thread
From: Yu Kuai @ 2023-05-10 3:20 UTC (permalink / raw)
To: Jens Axboe, Yu Kuai, Guangwu Zhang, linux-block, io-uring,
Jeff Moyer, Ming Lei, Jan Kara, Paolo Valente, yukuai (C)
Hi, Jens
在 2023/05/10 10:17, Jens Axboe 写道:
> On 5/9/23 8:00?PM, Yu Kuai wrote:
>> Hi,
>>
>> ? 2023/05/10 9:49, Yu Kuai ??:
>>> Hi,
>>>
>>> ? 2023/05/10 9:29, Yu Kuai ??:
>>>> Hi,
>>>>
>>>> ? 2023/05/10 8:49, Guangwu Zhang ??:
>>>>> Hi,
>>>>>
>>>>> We found this kernel NULL pointer issue with latest
>>>>> linux-block/for-next, please check it.
>>>>>
>>>>> Kernel repo: https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git
>>>>>
>>>>>
>>>>> [ 112.483804] BUG: kernel NULL pointer dereference, address: 0000000000000048
>>>
>>> Base on this offset, 0x48 match bio->bi_blkg, so I guess this is because
>>> bio is NULL, so the problem is that passthrough request insert into
>>> elevator.
>>>
>> Sorry that attached patch has some problem, please try this one.
>
> Let's please fix this in bfq, this isn't a core issue and it's not a
> good idea to work around it there.
>
I can do that, but I'm not sure because it seems passthrough rq is not
supposed to insert into elevator.
Bfq always expect that bio is not NULL, and before this commit
a327c341dc65 ("blk-mq: fix passthrough plugging"), passthrough rq can
never insert into plug, and therefor passthrough rq can never insert
into elevator.
Thanks,
Kuai
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [bug report] BUG: kernel NULL pointer dereference, address: 0000000000000048
2023-05-10 2:00 ` Yu Kuai
2023-05-10 2:17 ` Jens Axboe
@ 2023-05-10 4:52 ` Guangwu Zhang
1 sibling, 0 replies; 10+ messages in thread
From: Guangwu Zhang @ 2023-05-10 4:52 UTC (permalink / raw)
To: Yu Kuai, Jens Axboe
Cc: linux-block, io-uring, Jeff Moyer, Ming Lei, yukuai (C)
HI,
after applying your patch[1], the system will panic after reboot.
OK ] Finished Coldplug All udev Devices.
[ OK ] Reached target Network.
Starting dracut initqueue hook...
[ 4.675720] list_add double add: new=ffff90b056320a48,
prev=ffffa4f685f43a70, next=ffff90b056320a48.
[ 4.684931] ------------[ cut here ]------------
[ 4.689578] kernel BUG at lib/list_debug.c:33!
[ 4.694053] invalid opcode: 0000 [#1] PREEMPT SMP PTI
[ 4.699134] CPU: 14 PID: 706 Comm: systemd-udevd Not tainted 6.4.0-rc1+ #2
[ 4.706049] Hardware name: HPE ProLiant DL380 Gen10/ProLiant DL380
Gen10, BIOS U30 06/20/2018
[ 4.714621] RIP: 0010:__list_add_valid+0x8b/0x90
[ 4.719271] Code: d1 4c 89 c6 4c 89 ca 48 c7 c7 78 d1 bb 99 e8 cc
56 b6 ff 0f 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 d0 d1 bb 99 e8 b5
56 b6 ff <0f> 0b 0f 1f 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90
90 48
[ 4.738150] RSP: 0018:ffffa4f685f43a48 EFLAGS: 00010246
[ 4.743405] RAX: 0000000000000058 RBX: ffff90b056320a00 RCX: 0000000000000000
[ 4.750578] RDX: 0000000000000000 RSI: ffff90b76fb9f840 RDI: ffff90b76fb9f840
[ 4.757752] RBP: ffffa4f685f43a68 R08: 0000000000000000 R09: 00000000ffff7fff
[ 4.764925] R10: ffffa4f685f43900 R11: ffffffff9a1e6888 R12: ffffa4f685f43b78
[ 4.772100] R13: ffffa4f685f43a70 R14: ffff90b38a035800 R15: ffff90b056320a48
[ 4.779275] FS: 00007fea88741540(0000) GS:ffff90b76fb80000(0000)
knlGS:0000000000000000
[ 4.787411] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 4.793189] CR2: 00007fff014ba608 CR3: 0000000449ed4006 CR4: 00000000007706e0
[ 4.800362] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 4.807536] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 4.814710] PKRU: 55555554
[ 4.817429] Call Trace:
[ 4.819887] <TASK>
[ 4.821995] blk_mq_dispatch_plug_list+0x112/0x320
[ 4.826820] blk_mq_flush_plug_list+0x43/0x190
[ 4.831289] __blk_flush_plug+0x102/0x160
[ 4.835325] blk_finish_plug+0x25/0x40
[ 4.839095] read_pages+0x19a/0x220
[ 4.842606] page_cache_ra_unbounded+0x137/0x180
[ 4.847250] force_page_cache_ra+0xc5/0xf0
[ 4.851369] filemap_get_pages+0xf9/0x360
[ 4.855406] filemap_read+0xc5/0x320
[ 4.859001] ? generic_fillattr+0x45/0xf0
[ 4.863036] ? _copy_to_user+0x20/0x40
[ 4.866808] ? cp_new_stat+0x150/0x180
[ 4.870579] blkdev_read_iter+0xaf/0x170
[ 4.874524] vfs_read+0x1b5/0x2d0
[ 4.877860] ksys_read+0x5f/0xe0
[ 4.881107] do_syscall_64+0x59/0x90
[ 4.884706] ? exc_page_fault+0x65/0x150
[ 4.888653] entry_SYSCALL_64_after_hwframe+0x72/0xdc
[ 4.893737] RIP: 0033:0x7fea8934eaf2
[1] https://lore.kernel.org/linux-block/[email protected]/
> diff --git a/block/blk-mq.c b/block/blk-mq.c
> index f6dad0886a2f..bd94d8a5416f 100644
> --- a/block/blk-mq.c
> +++ b/block/blk-mq.c
> @@ -2712,6 +2712,7 @@ static void blk_mq_dispatch_plug_list(struct
> blk_plug *plug, bool from_sched)
> struct request **requeue_lastp = &requeue_list;
> unsigned int depth = 0;
> LIST_HEAD(list);
> + LIST_HEAD(passthrough_list);
>
> do {
> struct request *rq = rq_list_pop(&plug->mq_list);
> @@ -2723,7 +2724,10 @@ static void blk_mq_dispatch_plug_list(struct
> blk_plug *plug, bool from_sched)
> rq_list_add_tail(&requeue_lastp, rq);
> continue;
> }
> - list_add(&rq->queuelist, &list);
> + if (blk_rq_is_passthrough(rq))
> + list_add(&rq->queuelist, &passthrough_list);
> + else
> + list_add(&rq->queuelist, &list);
> depth++;
> } while (!rq_list_empty(plug->mq_list));
>
> @@ -2731,6 +2735,9 @@ static void blk_mq_dispatch_plug_list(struct
> blk_plug *plug, bool from_sched)
> trace_block_unplug(this_hctx->queue, depth, !from_sched);
>
> percpu_ref_get(&this_hctx->queue->q_usage_counter);
> + if (!list_empty(&passthrough_list))
> + blk_mq_insert_requests(this_hctx, this_ctx,
> &passthrough_list,
> + from_sched);
> if (this_hctx->queue->elevator) {
>
> this_hctx->queue->elevator->type->ops.insert_requests(this_hctx,
> &list, 0);
>
> Thanks,
> Kuai
>
--
Guangwu Zhang, RHCE, ISTQB, ITIL
Quality Engineer, Kernel Storage QE
Red Hat
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [bug report] BUG: kernel NULL pointer dereference, address: 0000000000000048
[not found] ` <CAGS2=YocNy9PkgRzzRdHAK1gGdjMxzA--PYS=sPrX_nCe4U6QA@mail.gmail.com>
@ 2023-05-10 6:39 ` Ming Lei
2023-05-10 6:55 ` Yu Kuai
2023-05-10 12:08 ` Guangwu Zhang
0 siblings, 2 replies; 10+ messages in thread
From: Ming Lei @ 2023-05-10 6:39 UTC (permalink / raw)
To: Guangwu Zhang
Cc: Yu Kuai, Jens Axboe, linux-block, io-uring, Jeff Moyer, Jan Kara,
Paolo Valente, yukuai (C)
On Wed, May 10, 2023 at 12:05:07PM +0800, Guangwu Zhang wrote:
> HI,
> after apply your patch[1], the system will panic after reboot.
>
Maybe you can try the following patch?
diff --git a/block/blk-mq.c b/block/blk-mq.c
index f6dad0886a2f..d84174a7e997 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -1303,7 +1303,7 @@ void blk_execute_rq_nowait(struct request *rq, bool at_head)
* device, directly accessing the plug instead of using blk_mq_plug()
* should not have any consequences.
*/
- if (current->plug && !at_head) {
+ if (current->plug && !at_head && rq->bio) {
blk_add_rq_to_plug(current->plug, rq);
return;
}
thanks,
Ming
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [bug report] BUG: kernel NULL pointer dereference, address: 0000000000000048
2023-05-10 6:39 ` Ming Lei
@ 2023-05-10 6:55 ` Yu Kuai
2023-05-10 12:08 ` Guangwu Zhang
1 sibling, 0 replies; 10+ messages in thread
From: Yu Kuai @ 2023-05-10 6:55 UTC (permalink / raw)
To: Ming Lei, Guangwu Zhang
Cc: Yu Kuai, Jens Axboe, linux-block, io-uring, Jeff Moyer, Jan Kara,
Paolo Valente, yukuai (C)
Hi,
在 2023/05/10 14:39, Ming Lei 写道:
> On Wed, May 10, 2023 at 12:05:07PM +0800, Guangwu Zhang wrote:
>> HI,
>> after apply your patch[1], the system will panic after reboot.
This is werid, I just reporduce this problem in my VM, and I verified
this patch can fix the problem.
Anyway, Ming's patch looks better, you can try it.
Thanks,
Kuai
>>
>
> Maybe you can try the following patch?
>
> diff --git a/block/blk-mq.c b/block/blk-mq.c
> index f6dad0886a2f..d84174a7e997 100644
> --- a/block/blk-mq.c
> +++ b/block/blk-mq.c
> @@ -1303,7 +1303,7 @@ void blk_execute_rq_nowait(struct request *rq, bool at_head)
> * device, directly accessing the plug instead of using blk_mq_plug()
> * should not have any consequences.
> */
> - if (current->plug && !at_head) {
> + if (current->plug && !at_head && rq->bio) {
> blk_add_rq_to_plug(current->plug, rq);
> return;
> }
>
>
> thanks,
> Ming
>
> .
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [bug report] BUG: kernel NULL pointer dereference, address: 0000000000000048
2023-05-10 6:39 ` Ming Lei
2023-05-10 6:55 ` Yu Kuai
@ 2023-05-10 12:08 ` Guangwu Zhang
1 sibling, 0 replies; 10+ messages in thread
From: Guangwu Zhang @ 2023-05-10 12:08 UTC (permalink / raw)
To: Ming Lei
Cc: Yu Kuai, Jens Axboe, linux-block, io-uring, Jeff Moyer, Jan Kara,
Paolo Valente, yukuai (C)
Hi,
Don't hit the issue after apply your patch.
thanks
Ming Lei <[email protected]> 于2023年5月10日周三 14:39写道：
>
> On Wed, May 10, 2023 at 12:05:07PM +0800, Guangwu Zhang wrote:
> > HI,
> > after apply your patch[1], the system will panic after reboot.
> >
>
> Maybe you can try the following patch?
>
> diff --git a/block/blk-mq.c b/block/blk-mq.c
> index f6dad0886a2f..d84174a7e997 100644
> --- a/block/blk-mq.c
> +++ b/block/blk-mq.c
> @@ -1303,7 +1303,7 @@ void blk_execute_rq_nowait(struct request *rq, bool at_head)
> * device, directly accessing the plug instead of using blk_mq_plug()
> * should not have any consequences.
> */
> - if (current->plug && !at_head) {
> + if (current->plug && !at_head && rq->bio) {
> blk_add_rq_to_plug(current->plug, rq);
> return;
> }
>
>
> thanks,
> Ming
>
--
Guangwu Zhang, RHCE, ISTQB, ITIL
Quality Engineer, Kernel Storage QE
Red Hat
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2023-05-10 12:08 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-05-10 0:49 [bug report] BUG: kernel NULL pointer dereference, address: 0000000000000048 Guangwu Zhang
2023-05-10 1:29 ` Yu Kuai
2023-05-10 1:49 ` Yu Kuai
2023-05-10 2:00 ` Yu Kuai
2023-05-10 2:17 ` Jens Axboe
2023-05-10 3:20 ` Yu Kuai
[not found] ` <CAGS2=YocNy9PkgRzzRdHAK1gGdjMxzA--PYS=sPrX_nCe4U6QA@mail.gmail.com>
2023-05-10 6:39 ` Ming Lei
2023-05-10 6:55 ` Yu Kuai
2023-05-10 12:08 ` Guangwu Zhang
2023-05-10 4:52 ` Guangwu Zhang
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox