RCU warning off ublk_buf_cleanup() -> mas_for_each()

RCU warning off ublk_buf_cleanup() -> mas_for_each()

[Jens Axboe]

Hi Ming,

Ran into the below running tests on the current tree:

=============================
WARNING: suspicious RCU usage
7.0.0+ #16 Tainted: G                 N 
-----------------------------
lib/maple_tree.c:759 suspicious rcu_dereference_check() usage!

other info that might help us debug this:


rcu_scheduler_active = 2, debug_locks = 1
1 lock held by iou-wrk-55535/55536:
 #0: ffff800085a451a0 (ublk_ctl_mutex){+.+.}-{4:4}, at: ublk_ctrl_del_dev+0xdc/0x2f8

stack backtrace:
CPU: 4 UID: 0 PID: 55536 Comm: iou-wrk-55535 Tainted: G                 N  7.0.0+ #16 PREEMPT 
Tainted: [N]=TEST
Hardware name: linux,dummy-virt (DT)
Call trace:
 show_stack+0x1c/0x30 (C)
 dump_stack_lvl+0x68/0x90
 dump_stack+0x18/0x20
 lockdep_rcu_suspicious+0x170/0x200
 mas_walk+0x3f0/0x6a0
 mas_find+0x1b4/0x6b0
 ublk_buf_cleanup+0xe0/0x240
 ublk_cdev_rel+0x34/0x1b0
 device_release+0xa4/0x350
 kobject_put+0x138/0x250
 put_device+0x18/0x30
 ublk_put_device+0x18/0x28
 ublk_ctrl_del_dev+0x120/0x2f8
 ublk_ctrl_uring_cmd+0x598/0x29b8
 io_uring_cmd+0x1e0/0x468
 __io_issue_sqe+0xa4/0x748
 io_issue_sqe+0x80/0xf68
 io_wq_submit_work+0x26c/0xdc8
 io_worker_handle_work+0x334/0xf20
 io_wq_worker+0x278/0x9e8
 ret_from_fork+0x10/0x20
Buffer I/O error on dev ublkb0, logical block 0, async page read
Buffer I/O error on dev ublkb0, logical block 0, async page read
 ublkb0: unable to read partition table
Buffer I/O error on dev ublkb0, logical block 0, async page read
Buffer I/O error on dev ublkb0, logical block 0, async page read
Buffer I/O error on dev ublkb0, logical block 512, async page read
Buffer I/O error on dev ublkb0, logical block 512, async page read
Buffer I/O error on dev ublkb0, logical block 0, async page read
Buffer I/O error on dev ublkb0, logical block 512, async page read

and I briefly looked at it, but then just gave up as a) the maple tree
documentation is not that detailed, and b) other in-tree users also just
call mas_for_each() without either a lock held or RCU read side locked.

Adding Liam for shedding some light on this...

-- 
Jens Axboe

Re: RCU warning off ublk_buf_cleanup() -> mas_for_each()

[Jens Axboe]

On 4/21/26 11:47 AM, Jens Axboe wrote:
> Hi Ming,
> 
> Ran into the below running tests on the current tree:
> 
> =============================
> WARNING: suspicious RCU usage
> 7.0.0+ #16 Tainted: G                 N 
> -----------------------------
> lib/maple_tree.c:759 suspicious rcu_dereference_check() usage!

FWIW, here's what claude spits out on the maple tree usage for
ublk. Simply passing it on...

  Issue 1: ublk_buf_cleanup — missing RCU/lock for mas_for_each (line 5489)          
                                                                                     
  ublk_buf_cleanup iterates the tree using mas_for_each without holding either       
  rcu_read_lock or mas_lock. The mas_find() documentation explicitly states: "Must   
  hold rcu_read_lock or the write lock." Internally, mas_find → mas_next_slot →      
  mt_slot → rcu_dereference_check(slots[offset], mt_locked(mt)). With neither RCU nor
   the tree lock held, this will fire a lockdep splat on CONFIG_PROVE_RCU=y kernels. 

  While functionally safe (exclusive access during device release), the API contract 
  is violated. Fix: wrap the iteration in rcu_read_lock/unlock, or use
  mas_lock/unlock.                                                                   
                                                            
  Issue 2: __ublk_ctrl_unreg_buf — unpin_user_pages under spinlock (line 5431-5455)  
  
  This function holds mas_lock (a spinlock — atomic context) while unpinning         
  potentially many pages in a loop. For a large registered buffer with many disjoint
  PFN ranges, this holds the spinlock for an extended period. unpin_user_pages →     
  gup_put_folio → folio_put could also grab additional locks if the folio's refcount
  drops to zero.

  A cleaner pattern would be to collect entries, drop mas_lock, then unpin. Compare  
  with ublk_buf_cleanup which does the same unpinning work outside any lock — the
  asymmetry is notable.                                                              
                                                            
  Issue 3: ublk_buf_cleanup — kfree without mas_erase (line 5504)                    
   
  The cleanup function does kfree(range) during iteration without first calling      
  mas_erase(). This leaves dangling pointers in the tree nodes until mtree_destroy is
   called on line 5506. Not a bug (no concurrent access, mtree_destroy doesn't       
  dereference stored entries), but it's inconsistent with ublk_buf_erase_ranges and
  __ublk_ctrl_unreg_buf which both properly erase before freeing.


-- 
Jens Axboe

Re: RCU warning off ublk_buf_cleanup() -> mas_for_each()

[Liam R. Howlett]

* Jens Axboe <axboe@kernel.dk> [260421 13:47]:
> Hi Ming,
> 
> Ran into the below running tests on the current tree:
> 
> =============================
> WARNING: suspicious RCU usage
> 7.0.0+ #16 Tainted: G                 N 
> -----------------------------
> lib/maple_tree.c:759 suspicious rcu_dereference_check() usage!
> 
> other info that might help us debug this:
> 
> 
> rcu_scheduler_active = 2, debug_locks = 1
> 1 lock held by iou-wrk-55535/55536:
>  #0: ffff800085a451a0 (ublk_ctl_mutex){+.+.}-{4:4}, at: ublk_ctrl_del_dev+0xdc/0x2f8
> 
> stack backtrace:
> CPU: 4 UID: 0 PID: 55536 Comm: iou-wrk-55535 Tainted: G                 N  7.0.0+ #16 PREEMPT 
> Tainted: [N]=TEST
> Hardware name: linux,dummy-virt (DT)
> Call trace:
>  show_stack+0x1c/0x30 (C)
>  dump_stack_lvl+0x68/0x90
>  dump_stack+0x18/0x20
>  lockdep_rcu_suspicious+0x170/0x200
>  mas_walk+0x3f0/0x6a0
>  mas_find+0x1b4/0x6b0
>  ublk_buf_cleanup+0xe0/0x240
>  ublk_cdev_rel+0x34/0x1b0
>  device_release+0xa4/0x350
>  kobject_put+0x138/0x250
>  put_device+0x18/0x30
>  ublk_put_device+0x18/0x28
>  ublk_ctrl_del_dev+0x120/0x2f8
>  ublk_ctrl_uring_cmd+0x598/0x29b8
>  io_uring_cmd+0x1e0/0x468
>  __io_issue_sqe+0xa4/0x748
>  io_issue_sqe+0x80/0xf68
>  io_wq_submit_work+0x26c/0xdc8
>  io_worker_handle_work+0x334/0xf20
>  io_wq_worker+0x278/0x9e8
>  ret_from_fork+0x10/0x20
> Buffer I/O error on dev ublkb0, logical block 0, async page read
> Buffer I/O error on dev ublkb0, logical block 0, async page read
>  ublkb0: unable to read partition table
> Buffer I/O error on dev ublkb0, logical block 0, async page read
> Buffer I/O error on dev ublkb0, logical block 0, async page read
> Buffer I/O error on dev ublkb0, logical block 512, async page read
> Buffer I/O error on dev ublkb0, logical block 512, async page read
> Buffer I/O error on dev ublkb0, logical block 0, async page read
> Buffer I/O error on dev ublkb0, logical block 512, async page read
> 
> and I briefly looked at it, but then just gave up as a) the maple tree
> documentation is not that detailed,

Which documentation is lacking?  I will fix it.

I have user documentation in the Documentation directory while
technical details are in the code.


>and b) other in-tree users also just
> call mas_for_each() without either a lock held or RCU read side locked.

mas_for_each() must hold a lock of some type.

> 
> Adding Liam for shedding some light on this...
> 
> -- 
> Jens Axboe
>