Re: Fwd: btrfs / io-uring corrupting reads

[Daniel Black <daniel@mariadb.org>]

Filipe,

DId you find anything? This is starting to be noticed by our mutual users.

https://jira.mariadb.org/browse/MDEV-27900
https://mariadb.zulipchat.com/#narrow/stream/118759-general/topic/Corrupt.20database.20page.20when.20updating.20from.2010.2E5

On Tue, Feb 22, 2022 at 11:55 PM Filipe Manana <fdmanana@kernel.org> wrote:
>
> On Tue, Feb 22, 2022 at 12:46 PM Filipe Manana <fdmanana@kernel.org> wrote:
> >
> > On Tue, Feb 22, 2022 at 12:22 PM Daniel Black <daniel@mariadb.org> wrote:
> > >
> > > On Tue, Feb 22, 2022 at 10:42 PM Filipe Manana <fdmanana@kernel.org> wrote:
> > >
> > > > I gave it a try, but it fails setting up io_uring:
> > > >
> > > > 2022-02-22 11:27:13 0 [Note] mysqld: O_TMPFILE is not supported on /tmp (disabling future attempts)
> > > > 2022-02-22 11:27:13 0 [Warning] mysqld: io_uring_queue_init() failed with errno 1
> > > > 2022-02-22 11:27:13 0 [Warning] InnoDB: liburing disabled: falling back to innodb_use_native_aio=OFF
> > > > 2022-02-22 11:27:13 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728
> > > > 2022-02-22 11:27:13 0 [Note] InnoDB: Completed initialization of buffer pool
> > > >
> > > > So that's why it doesn't fail here, as it fallbacks to no aio mode.
> > >
> > > error 1 is EPERM. Seems it needs --privileged on the container startup
> > > as a podman argument (before the image name). Sorry I missed that
> > >
> > > > Any idea why it's failing to setup io_uring?
> > > >
> > > > I have the liburing2 and liburing-dev packages installed on debian, and
> > > > tried with a 5.17-rc4 kernel.
> > >
> > > Taking https://packages.debian.org/bookworm/mariadb-server-core-10.6 package:
> > >
> > > mariadb-install-db --no-defaults --datadir=/empty/btrfs/path
> > > --innodb-use-native-aio=0
> > >
> > > mariadbd --no-defaults --datadir=/empty/btrfs/path --innodb-use-native-aio=1
> > >
> > > should achieve the same thing.
> >
> > Sorry, I have no experience with mariadb and podman. How am I supposed
> > to run that?
> > Is that supposed to run inside the container, on the host? Do I need
> > to change the podman command lines?
> >
> > What I did before was:
> >
> > DEV=/dev/sdh
> > MNT=/mnt/sdh
> >
> > mkfs.btrfs -f $DEV
> > mount $DEV $MNT
> >
> > mkdir $MNT/noaio
> > chown fdmanana: $MNT/noaio
> >
> > podman run --name mdbinit --rm -v $MNT/noaio/:/var/lib/mysql:Z -e
> > MARIADB_ALLOW_EMPTY_ROOT_PASSWORD=1
> > quay.io/danielgblack/mariadb-test:10.6-impish-sysbench
> > --innodb_use_native_aio=0
> >
> >
> > Then in another shell:
> >
> > podman kill --all
> >
> > podman run --rm -v $MNT/noaio/:/var/lib/mysql:Z -e
> > MARIADB_ALLOW_EMPTY_ROOT_PASSWORD=1
> > quay.io/danielgblack/mariadb-test:10.6-impish-sysbench
> > --innodb_use_native_aio=1
> >
> >
> > What should I change or add in there?
>
> Ok, just passing  --privileged to both podman commands triggered the
> bug as in your report.
> I'll see if I can figure out what's causing the read corruption.
>
>
> >
> > Thanks.