recvmsg not honoring O_NONBLOCK

recvmsg not honoring O_NONBLOCK

[Hrvoje Zeba]

[-- Attachment #1: Type: text/plain, Size: 450 bytes --]

Hi folks,

I was adapting msquic library to work with iouring and found an issue
with recvmsg() where it ignores O_NONBLOCK flag set on the file
descriptor. If MSG_DONTWAIT is set in flags, it behaves as expected.
I've attached a simple test which currently just hangs on iouring's
recvmsg(). I'm guessing sendmsg() behaves the same way but I have no
idea how to fill the buffer to reliably test it.

Best,
Hrvoje

-- 
I doubt, therefore I might be.

[-- Attachment #2: recvmsg_test.c --]
[-- Type: text/x-csrc, Size: 1781 bytes --]

#include <stdio.h>
#include <assert.h>
#include <liburing.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <netinet/udp.h>

void recvmsg_syscall(int s)
{
    struct sockaddr_in addr;

    char buff[1024];
    struct iovec iov = {
        .iov_base = buff,
        .iov_len = sizeof(buff)
    };

    struct msghdr msg = {
        .msg_name = &addr,
        .msg_namelen = sizeof(addr),
        .msg_iov = &iov,
        .msg_iovlen = 1,
    };

    int res = recvmsg(s, &msg, 0/*MSG_DONTWAIT*/);

    assert(res == -1);
    assert(errno == EAGAIN);
}

void recvmsg_iouring(int s)
{
    struct sockaddr_in addr;

    char buff[1024];
    struct iovec iov = {
        .iov_base = buff,
        .iov_len = sizeof(buff)
    };

    struct msghdr msg = {
        .msg_name = &addr,
        .msg_namelen = sizeof(addr),
        .msg_iov = &iov,
        .msg_iovlen = 1,
    };

    struct io_uring ring;

    int res = io_uring_queue_init(1024, &ring, 0);
    assert(res == 0);

    struct io_uring_cqe *cqe;
    struct io_uring_sqe* sqe;

    sqe = io_uring_get_sqe(&ring);
    io_uring_prep_recvmsg(sqe, s, &msg, 0/*MSG_DONTWAIT*/);

    res = io_uring_submit(&ring);
    assert(res == 1);

    res = io_uring_wait_cqe(&ring, &cqe);
    assert(res == 0);

    assert(cqe->res == -EAGAIN);

    io_uring_cqe_seen(&ring, cqe);

    io_uring_queue_exit(&ring);
}

int main(int argc, const char* argv[])
{
    int s = socket(AF_INET, SOCK_DGRAM | SOCK_NONBLOCK, 0);
    assert(s != -1);

    struct sockaddr_in addr = {
        .sin_family = AF_INET,
        .sin_port = 0x0010,
        .sin_addr.s_addr = 0x0100007fUL
    };

    int res = bind(s, (struct sockaddr*)(&addr), sizeof(addr));
    assert(res == 0);

    recvmsg_syscall(s);
    recvmsg_iouring(s);

    return 0;
}

Re: recvmsg not honoring O_NONBLOCK

[Jens Axboe]

On 7/9/22 8:21 AM, Hrvoje Zeba wrote:
> Hi folks,
> 
> I was adapting msquic library to work with iouring and found an issue
> with recvmsg() where it ignores O_NONBLOCK flag set on the file
> descriptor. If MSG_DONTWAIT is set in flags, it behaves as expected.
> I've attached a simple test which currently just hangs on iouring's
> recvmsg(). I'm guessing sendmsg() behaves the same way but I have no
> idea how to fill the buffer to reliably test it.

I am guessing that you're waiting off waiting for the event. It's not
stalled on the issue, it'll arm poll and wait for an event. I won't go
into too much of a ran on O_NONBLOCK, but it's a giant mess, and where
possible, io_uring relies on per-request non-block flags as that is much
saner behavior.

-- 
Jens Axboe

Re: recvmsg not honoring O_NONBLOCK

[Hrvoje Zeba]

On Sat, Jul 9, 2022 at 11:26 AM Jens Axboe <axboe@kernel.dk> wrote:
>
> On 7/9/22 8:21 AM, Hrvoje Zeba wrote:
> > Hi folks,
> >
> > I was adapting msquic library to work with iouring and found an issue
> > with recvmsg() where it ignores O_NONBLOCK flag set on the file
> > descriptor. If MSG_DONTWAIT is set in flags, it behaves as expected.
> > I've attached a simple test which currently just hangs on iouring's
> > recvmsg(). I'm guessing sendmsg() behaves the same way but I have no
> > idea how to fill the buffer to reliably test it.
>
> I am guessing that you're waiting off waiting for the event. It's not
> stalled on the issue, it'll arm poll and wait for an event. I won't go
> into too much of a ran on O_NONBLOCK, but it's a giant mess, and where
> possible, io_uring relies on per-request non-block flags as that is much
> saner behavior.
>

Oh, this is intentional behaviour. Got it! Thanks for the explanation.

-- 
I doubt, therefore I might be.