Memory uninitialized after "io_uring: keep table of pointers to ubufs"

Memory uninitialized after "io_uring: keep table of pointers to ubufs"

[Andres Freund]

Hi,

I started to see buffer registration randomly failing with ENOMEM on
5.13. Registering buffer or two often succeeds, but more than that
rarely. Running the same program as root succeeds - but the user has a high
rlimit.

The issue is that io_sqe_buffer_register() doesn't initialize
imu. io_buffer_account_pin() does imu->acct_pages++, before calling
io_account_mem(ctx, imu->acct_pages);

Which means that a random amount of memory is being accounted for. On the first
few allocations this sometimes fails to fail because the memory is zero, but
after a bit of reuse...

It only doesn't fail as root because the rlimit doesn't apply.

This is caused by

commit 41edf1a5ec967bf4bddedb83c48e02dfea8315b4
Author: Pavel Begunkov <asml.silence@gmail.com>
Date:   2021-04-25 14:32:23 +0100

    io_uring: keep table of pointers to ubufs

Greetings,

Andres Freund

Re: Memory uninitialized after "io_uring: keep table of pointers to ubufs"

[Pavel Begunkov]

On 5/29/21 1:33 AM, Andres Freund wrote:
> Hi,
> 
> I started to see buffer registration randomly failing with ENOMEM on
> 5.13. Registering buffer or two often succeeds, but more than that
> rarely. Running the same program as root succeeds - but the user has a high
> rlimit.
> 
> The issue is that io_sqe_buffer_register() doesn't initialize
> imu. io_buffer_account_pin() does imu->acct_pages++, before calling
> io_account_mem(ctx, imu->acct_pages);
> 
> Which means that a random amount of memory is being accounted for. On the first
> few allocations this sometimes fails to fail because the memory is zero, but
> after a bit of reuse...

Makes sense, thanks for digging in. I've just sent a patch, would
be great if you can test it or send your own.


> It only doesn't fail as root because the rlimit doesn't apply.
> 
> This is caused by
> 
> commit 41edf1a5ec967bf4bddedb83c48e02dfea8315b4
> Author: Pavel Begunkov <asml.silence@gmail.com>
> Date:   2021-04-25 14:32:23 +0100
> 
>     io_uring: keep table of pointers to ubufs
> 
> Greetings,
> 
> Andres Freund
> 

-- 
Pavel Begunkov

Re: Memory uninitialized after "io_uring: keep table of pointers to ubufs"

[Andres Freund]

Hi,

On 2021-05-29 12:03:12 +0100, Pavel Begunkov wrote:
> On 5/29/21 1:33 AM, Andres Freund wrote:
> > Hi,
> > 
> > I started to see buffer registration randomly failing with ENOMEM on
> > 5.13. Registering buffer or two often succeeds, but more than that
> > rarely. Running the same program as root succeeds - but the user has a high
> > rlimit.
> > 
> > The issue is that io_sqe_buffer_register() doesn't initialize
> > imu. io_buffer_account_pin() does imu->acct_pages++, before calling
> > io_account_mem(ctx, imu->acct_pages);
> > 
> > Which means that a random amount of memory is being accounted for. On the first
> > few allocations this sometimes fails to fail because the memory is zero, but
> > after a bit of reuse...
> 
> Makes sense, thanks for digging in. I've just sent a patch, would
> be great if you can test it or send your own.

Sorry for the slow response, I'm off this week. I did just get around to
test and unsurprisingly: The patch does fix the issue.

Greetings,

Andres Freund

Re: Memory uninitialized after "io_uring: keep table of pointers to ubufs"

[Jens Axboe]

On 6/3/21 12:06 PM, Andres Freund wrote:
> Hi,
> 
> On 2021-05-29 12:03:12 +0100, Pavel Begunkov wrote:
>> On 5/29/21 1:33 AM, Andres Freund wrote:
>>> Hi,
>>>
>>> I started to see buffer registration randomly failing with ENOMEM on
>>> 5.13. Registering buffer or two often succeeds, but more than that
>>> rarely. Running the same program as root succeeds - but the user has a high
>>> rlimit.
>>>
>>> The issue is that io_sqe_buffer_register() doesn't initialize
>>> imu. io_buffer_account_pin() does imu->acct_pages++, before calling
>>> io_account_mem(ctx, imu->acct_pages);
>>>
>>> Which means that a random amount of memory is being accounted for. On the first
>>> few allocations this sometimes fails to fail because the memory is zero, but
>>> after a bit of reuse...
>>
>> Makes sense, thanks for digging in. I've just sent a patch, would
>> be great if you can test it or send your own.
> 
> Sorry for the slow response, I'm off this week. I did just get around to
> test and unsurprisingly: The patch does fix the issue.

OK good, thanks for confirming, I did ship it out earlier today so
should be in the next -rc.

-- 
Jens Axboe

Re: Memory uninitialized after "io_uring: keep table of pointers to ubufs"

[Pavel Begunkov]

On 6/3/21 7:06 PM, Andres Freund wrote:
> Hi,
> 
> On 2021-05-29 12:03:12 +0100, Pavel Begunkov wrote:
>> On 5/29/21 1:33 AM, Andres Freund wrote:
>>> Hi,
>>>
>>> I started to see buffer registration randomly failing with ENOMEM on
>>> 5.13. Registering buffer or two often succeeds, but more than that
>>> rarely. Running the same program as root succeeds - but the user has a high
>>> rlimit.
>>>
>>> The issue is that io_sqe_buffer_register() doesn't initialize
>>> imu. io_buffer_account_pin() does imu->acct_pages++, before calling
>>> io_account_mem(ctx, imu->acct_pages);
>>>
>>> Which means that a random amount of memory is being accounted for. On the first
>>> few allocations this sometimes fails to fail because the memory is zero, but
>>> after a bit of reuse...
>>
>> Makes sense, thanks for digging in. I've just sent a patch, would
>> be great if you can test it or send your own.
> 
> Sorry for the slow response, I'm off this week. I did just get around to
> test and unsurprisingly: The patch does fix the issue.

Yep, since you already narrowed it down. Thanks for testing

-- 
Pavel Begunkov