Re: RCU warning off ublk_buf_cleanup() -> mas_for_each()

[Jens Axboe <axboe@kernel.dk>]

On 4/21/26 3:56 PM, Liam R. Howlett wrote:
> * Jens Axboe <axboe@kernel.dk> [260421 17:41]:
>> On 4/21/26 2:28 PM, Liam R. Howlett wrote:
>>> * Jens Axboe <axboe@kernel.dk> [260421 13:47]:
>>>> Hi Ming,
>>>>
>>>> Ran into the below running tests on the current tree:
>>>>
>>>> =============================
>>>> WARNING: suspicious RCU usage
>>>> 7.0.0+ #16 Tainted: G                 N 
>>>> -----------------------------
>>>> lib/maple_tree.c:759 suspicious rcu_dereference_check() usage!
>>>>
>>>> other info that might help us debug this:
>>>>
>>>>
>>>> rcu_scheduler_active = 2, debug_locks = 1
>>>> 1 lock held by iou-wrk-55535/55536:
>>>>  #0: ffff800085a451a0 (ublk_ctl_mutex){+.+.}-{4:4}, at: ublk_ctrl_del_dev+0xdc/0x2f8
>>>>
>>>> stack backtrace:
>>>> CPU: 4 UID: 0 PID: 55536 Comm: iou-wrk-55535 Tainted: G                 N  7.0.0+ #16 PREEMPT 
>>>> Tainted: [N]=TEST
>>>> Hardware name: linux,dummy-virt (DT)
>>>> Call trace:
>>>>  show_stack+0x1c/0x30 (C)
>>>>  dump_stack_lvl+0x68/0x90
>>>>  dump_stack+0x18/0x20
>>>>  lockdep_rcu_suspicious+0x170/0x200
>>>>  mas_walk+0x3f0/0x6a0
>>>>  mas_find+0x1b4/0x6b0
>>>>  ublk_buf_cleanup+0xe0/0x240
>>>>  ublk_cdev_rel+0x34/0x1b0
>>>>  device_release+0xa4/0x350
>>>>  kobject_put+0x138/0x250
>>>>  put_device+0x18/0x30
>>>>  ublk_put_device+0x18/0x28
>>>>  ublk_ctrl_del_dev+0x120/0x2f8
>>>>  ublk_ctrl_uring_cmd+0x598/0x29b8
>>>>  io_uring_cmd+0x1e0/0x468
>>>>  __io_issue_sqe+0xa4/0x748
>>>>  io_issue_sqe+0x80/0xf68
>>>>  io_wq_submit_work+0x26c/0xdc8
>>>>  io_worker_handle_work+0x334/0xf20
>>>>  io_wq_worker+0x278/0x9e8
>>>>  ret_from_fork+0x10/0x20
>>>> Buffer I/O error on dev ublkb0, logical block 0, async page read
>>>> Buffer I/O error on dev ublkb0, logical block 0, async page read
>>>>  ublkb0: unable to read partition table
>>>> Buffer I/O error on dev ublkb0, logical block 0, async page read
>>>> Buffer I/O error on dev ublkb0, logical block 0, async page read
>>>> Buffer I/O error on dev ublkb0, logical block 512, async page read
>>>> Buffer I/O error on dev ublkb0, logical block 512, async page read
>>>> Buffer I/O error on dev ublkb0, logical block 0, async page read
>>>> Buffer I/O error on dev ublkb0, logical block 512, async page read
>>>>
>>>> and I briefly looked at it, but then just gave up as a) the maple tree
>>>> documentation is not that detailed,
>>>
>>> Which documentation is lacking?  I will fix it.
>>>
>>> I have user documentation in the Documentation directory while
>>> technical details are in the code.
>>
>> I went into the core-api/ and leafed through that, didn't have anything
>> on mas_for_each() that pertained to locking. Was hoping I'd find a table
>> of which parts of the API requires what in terms of locking or RCU.
>> There arent a whole lot of in-kernel users of it yet, so looking at
>> other places in the kernel wasn't very useful.
> 
> There's several users and the test code.  Too bad none of them helped.

There are several, but that's not very much for a kernel API. I didn't
look at the test code. If I was writing the code I would dug deeper, but
this isn't what this is. I just found an issue and the bare basics of
poking at the API and docs to check my assumptions.

>> Since this is a merge window regression, I really just passed it to Ming
>> with you on the CC just in case, and didn't spend any more time on it.
>> I'm not the one that's supposed to be finding issues like this...
>>
> 
> I appreciate the response as I'll try to cater the doc to what users
> search for.

A great example is what the llist stuff does in terms of which parts of
the API needs what kind of exclusion against each other, if any. But
maybe not as appropriate here.

And obviously my use case may not be entirely representative in terms of
what a developer might normally want if they were writing new code and
looking to use it. Or convert existing code.

> There are two APIs outlined in the documentation normal api with mtree_
> and the advanced api with mas_.  The locking is outlined in each
> section.

If I was writing code using it, I'm sure it would've been fine. And yes
mas_find() lists it, but I'm just looking at a basic back trace and the
main user, which was mas_for_each().

> I guess a note stating to check your code with lockdep might be in order
> in the documentation.

That's always a good suggestion. And the rcu checking, as you'd probably
use both.

>>>> and b) other in-tree users also just
>>>> call mas_for_each() without either a lock held or RCU read side locked.
>>>
>>> mas_for_each() must hold a lock of some type.
>>
>> That's what I assumed. I missed that the rcu dereference check
>> checks for an external lock too, which I guess you can register
>> with the maple tree. Funky... I guess it's just for lockdep
>> purposes, makes sense then.
> 
> The external lock is so that people can use the tree (which needs to
> allocate) with sleeping locks and not just spinlocks.  Willy wants me to
> kill it one day, though.

Ah, so it's actually used. I'd have to agree with willy on that front.
Though I'm always annoyed at the forced locking that xarray imposes on
you, so...

>> Presumably the current use case is fine, as it's serialized
>> teardown. It just ends up triggering the rcu sanity checks.
> 
> On tear down, we should iterate through and free any resources then
> destroy the tree.  If you erase each element one at a time you will
> rebalance the tree - since it's a b-tree.
> 
> Thanks again for taking the time to tell me where/what you looked so I
> can better answer the questions in the docs.  At least the LLM found it
> for you.

LLM was just a way to save my time. I think I've spent longer writing
emails about it at this point ;-). Thanks for the quick replies, I'm
sure Ming will appreciate it and sort this out.

-- 
Jens Axboe