Multiple io_uring issues with sendmsg/recvmsg on loopback interfaces

[vitalif@yourcmc.ru]

Hi

I'm having problems with io_uring when several applications communicate within localhost using sockets and io_uring.

I don't have a complete testcase yet, but the problem is apparently the following:
- on kernels 5.4 and 5.5 messages sent via io_uring sendmsg() and received via io_uring recvmsg() are SOMETIMES duplicated. 1 process sends 512 bytes and the second one receives the same message twice.
- in addition to that, on kernel 5.10 I often get garbage from io_uring recvmsg().

I discovered it when running a Vitastor (https://vitastor.io/) testcase, so it's not too easy to extract a minimal example for reproduction, and I can't be 100% sure that it's not my bug, however I know for sure that:
- on kernels 5.4 and 5.5 everything is fine when I disable io_uring sendmsg OR io_uring recvmsg and replace it with a classic synchronous version
- on kernel 5.10 everything is fine when I disable BOTH io_uring sendmsg and recvmsg
- everything was fine when I was testing my project in a setup with real networking instead of loopback

If you're really curious you can try to reproduce it yourself, you need to build Vitastor for it and then run ./run_tests.sh. The first issue manifests itself as a hang during `fio` run with 'command out of sync' message in one of ./testdata/osd*.log and the second one manifests as multiple failure messages during `fio` run with 'Received garbage: ' messages on one of ./testdata/osd*.log. Both issues reproduce almost every run.

I'll try to come back with a real test-case for reproduction, but maybe even this information can be sufficient for you to look at io_uring+sendmsg+recvmsg+loopback problems?

-- 
With best regards,
  Vitaliy Filippov