* Chaining accept+read
@ 2022-09-28 9:50 Ben Noordhuis
2022-09-28 10:00 ` Pavel Begunkov
0 siblings, 1 reply; 5+ messages in thread
From: Ben Noordhuis @ 2022-09-28 9:50 UTC (permalink / raw)
To: io-uring
I'm trying to chain accept+read but it's not working.
My code looks like this:
*sqe1 = (struct io_uring_sqe){
.opcode = IORING_OP_ACCEPT,
.flags = IOSQE_IO_LINK,
.fd = listenfd,
.file_index = 42, // or 42+1
};
*sqe2 = (struct io_uring_sqe){
.opcode = IORING_OP_READ,
.flags = IOSQE_FIXED_FILE,
.addr = (u64) buf,
.len = len,
.fd = 42,
};
submit();
Both ops fail immediately; accept with -ECANCELED, read with -EBADF,
presumably because fixed fd 42 doesn't exist at the time of submission.
Would it be possible to support this pattern in io_uring or are there
reasons for why things are the way they are?
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Chaining accept+read
2022-09-28 9:50 Chaining accept+read Ben Noordhuis
@ 2022-09-28 10:00 ` Pavel Begunkov
2022-09-28 10:55 ` Ben Noordhuis
0 siblings, 1 reply; 5+ messages in thread
From: Pavel Begunkov @ 2022-09-28 10:00 UTC (permalink / raw)
To: Ben Noordhuis, io-uring
On 9/28/22 10:50, Ben Noordhuis wrote:
> I'm trying to chain accept+read but it's not working.
>
> My code looks like this:
>
> *sqe1 = (struct io_uring_sqe){
> .opcode = IORING_OP_ACCEPT,
> .flags = IOSQE_IO_LINK,
> .fd = listenfd,
> .file_index = 42, // or 42+1
> };
> *sqe2 = (struct io_uring_sqe){
> .opcode = IORING_OP_READ,
> .flags = IOSQE_FIXED_FILE,
> .addr = (u64) buf,
> .len = len,
> .fd = 42,
> };
> submit();
>
> Both ops fail immediately; accept with -ECANCELED, read with -EBADF,
> presumably because fixed fd 42 doesn't exist at the time of submission.
>
> Would it be possible to support this pattern in io_uring or are there
> reasons for why things are the way they are?
It should already be supported. And errors look a bit odd, I'd rather
expect -EBADF or some other for accept and -ECANCELED for the read.
Do you have a test program / reporoducer? Hopefully in C.
--
Pavel Begunkov
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Chaining accept+read
2022-09-28 10:00 ` Pavel Begunkov
@ 2022-09-28 10:55 ` Ben Noordhuis
2022-09-28 11:59 ` Pavel Begunkov
0 siblings, 1 reply; 5+ messages in thread
From: Ben Noordhuis @ 2022-09-28 10:55 UTC (permalink / raw)
To: Pavel Begunkov; +Cc: io-uring
On Wed, Sep 28, 2022 at 12:02 PM Pavel Begunkov <[email protected]> wrote:
>
> On 9/28/22 10:50, Ben Noordhuis wrote:
> > I'm trying to chain accept+read but it's not working.
> >
> > My code looks like this:
> >
> > *sqe1 = (struct io_uring_sqe){
> > .opcode = IORING_OP_ACCEPT,
> > .flags = IOSQE_IO_LINK,
> > .fd = listenfd,
> > .file_index = 42, // or 42+1
> > };
> > *sqe2 = (struct io_uring_sqe){
> > .opcode = IORING_OP_READ,
> > .flags = IOSQE_FIXED_FILE,
> > .addr = (u64) buf,
> > .len = len,
> > .fd = 42,
> > };
> > submit();
> >
> > Both ops fail immediately; accept with -ECANCELED, read with -EBADF,
> > presumably because fixed fd 42 doesn't exist at the time of submission.
> >
> > Would it be possible to support this pattern in io_uring or are there
> > reasons for why things are the way they are?
>
> It should already be supported. And errors look a bit odd, I'd rather
> expect -EBADF or some other for accept and -ECANCELED for the read.
> Do you have a test program / reporoducer? Hopefully in C.
Of course, please see below. Error handling elided for brevity. Hope
I'm not doing anything stupid.
For me it immediately prints this:
0 res=-125
1 res=-9
Some observations:
- it's not included in the test case but I can tell from the user_data
field the -EBADF is the read op
- replacing IORING_OP_READ with e.g. IORING_OP_NOP makes it work
(accepts a connection)
- once the fd has been installed, I can successfully chain
IOSQE_FIXED_FILE read&write ops
I'm primarily testing against a 5.15 kernel. Is this something that's
been fixed since? I went through the commit history but I didn't find
anything relevant.
---
#include <linux/io_uring.h>
#include <netinet/in.h>
#include <stdatomic.h>
#include <stdio.h>
#include <string.h>
#include <sys/mman.h>
#include <sys/syscall.h>
#include <unistd.h>
int main(void) {
struct sockaddr_in sin = {
.sin_family = AF_INET,
.sin_addr = (struct in_addr){ htonl(INADDR_ANY) },
.sin_port = htons(9000),
};
int listenfd = socket(AF_INET, SOCK_STREAM|SOCK_CLOEXEC, 0);
(void) bind(listenfd, (struct sockaddr *) &sin, sizeof(sin));
(void) listen(listenfd, 128);
struct io_uring_params p = {};
int ringfd = syscall(__NR_io_uring_setup, 32, &p);
int files[64]; memset(files, -1, sizeof(files));
syscall(__NR_io_uring_register,
ringfd, IORING_REGISTER_FILES, files, 64);
__u8 *sq = mmap(0, p.sq_off.array + p.sq_entries * sizeof(__u32),
PROT_READ|PROT_WRITE, MAP_SHARED|MAP_POPULATE,
ringfd, IORING_OFF_SQ_RING);
__u8 *cq = mmap(
0, p.cq_off.cqes + p.cq_entries * sizeof(struct io_uring_cqe),
PROT_READ|PROT_WRITE, MAP_SHARED|MAP_POPULATE,
ringfd, IORING_OFF_CQ_RING);
struct io_uring_sqe *sqe = mmap(0, p.sq_entries * sizeof(*sqe),
PROT_READ|PROT_WRITE, MAP_SHARED|MAP_POPULATE,
ringfd, IORING_OFF_SQES);
__u32 *sqtail = (__u32 *) (sq + p.sq_off.tail);
__u32 *sqarray = (__u32 *) (sq + p.sq_off.array);
struct io_uring_cqe *cqe =
(struct io_uring_cqe *) (cq + p.cq_off.cqes);
sqe[0] = (struct io_uring_sqe){
.opcode = IORING_OP_ACCEPT,
.flags = IOSQE_ASYNC|IOSQE_IO_LINK,
.fd = listenfd,
.file_index = 42,
};
char buf[256];
sqe[1] = (struct io_uring_sqe){
.opcode = IORING_OP_READ,
.flags = IOSQE_FIXED_FILE,
.fd = 42,
.len = sizeof(buf),
.addr = (__u64) buf,
};
sqarray[0] = 0; sqarray[1] = 1;
atomic_store((atomic_uint *) sqtail, 2);
int n = syscall(__NR_io_uring_enter,
ringfd, 2, 1, IORING_ENTER_GETEVENTS, 0, 0);
for (int i = 0; i < n; i++) printf("%d res=%d\n", i, cqe[i].res);
return 0;
}
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Chaining accept+read
2022-09-28 10:55 ` Ben Noordhuis
@ 2022-09-28 11:59 ` Pavel Begunkov
2022-09-28 13:49 ` Jens Axboe
0 siblings, 1 reply; 5+ messages in thread
From: Pavel Begunkov @ 2022-09-28 11:59 UTC (permalink / raw)
To: Ben Noordhuis; +Cc: io-uring
On 9/28/22 11:55, Ben Noordhuis wrote:
> On Wed, Sep 28, 2022 at 12:02 PM Pavel Begunkov <[email protected]> wrote:
>>
>> On 9/28/22 10:50, Ben Noordhuis wrote:
>>> I'm trying to chain accept+read but it's not working.
>>>
>>> My code looks like this:
>>>
>>> *sqe1 = (struct io_uring_sqe){
>>> .opcode = IORING_OP_ACCEPT,
>>> .flags = IOSQE_IO_LINK,
>>> .fd = listenfd,
>>> .file_index = 42, // or 42+1
>>> };
>>> *sqe2 = (struct io_uring_sqe){
>>> .opcode = IORING_OP_READ,
>>> .flags = IOSQE_FIXED_FILE,
>>> .addr = (u64) buf,
>>> .len = len,
>>> .fd = 42,
>>> };
>>> submit();
>>>
>>> Both ops fail immediately; accept with -ECANCELED, read with -EBADF,
>>> presumably because fixed fd 42 doesn't exist at the time of submission.
>>>
>>> Would it be possible to support this pattern in io_uring or are there
>>> reasons for why things are the way they are?
>>
>> It should already be supported. And errors look a bit odd, I'd rather
>> expect -EBADF or some other for accept and -ECANCELED for the read.
>> Do you have a test program / reporoducer? Hopefully in C.
>
> Of course, please see below. Error handling elided for brevity. Hope
> I'm not doing anything stupid.
Perfect thanks
> For me it immediately prints this:
>
> 0 res=-125
> 1 res=-9
The reason is that in older kernels we're resolving the read's
file not after accept but when assembling the link, which was
specifically fixed a bit later. Jens, are there any plans to
backport it?
--
Pavel Begunkov
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Chaining accept+read
2022-09-28 11:59 ` Pavel Begunkov
@ 2022-09-28 13:49 ` Jens Axboe
0 siblings, 0 replies; 5+ messages in thread
From: Jens Axboe @ 2022-09-28 13:49 UTC (permalink / raw)
To: Pavel Begunkov, Ben Noordhuis; +Cc: io-uring
On 9/28/22 5:59 AM, Pavel Begunkov wrote:
> On 9/28/22 11:55, Ben Noordhuis wrote:
>> On Wed, Sep 28, 2022 at 12:02 PM Pavel Begunkov <[email protected]> wrote:
>>>
>>> On 9/28/22 10:50, Ben Noordhuis wrote:
>>>> I'm trying to chain accept+read but it's not working.
>>>>
>>>> My code looks like this:
>>>>
>>>> *sqe1 = (struct io_uring_sqe){
>>>> .opcode = IORING_OP_ACCEPT,
>>>> .flags = IOSQE_IO_LINK,
>>>> .fd = listenfd,
>>>> .file_index = 42, // or 42+1
>>>> };
>>>> *sqe2 = (struct io_uring_sqe){
>>>> .opcode = IORING_OP_READ,
>>>> .flags = IOSQE_FIXED_FILE,
>>>> .addr = (u64) buf,
>>>> .len = len,
>>>> .fd = 42,
>>>> };
>>>> submit();
>>>>
>>>> Both ops fail immediately; accept with -ECANCELED, read with -EBADF,
>>>> presumably because fixed fd 42 doesn't exist at the time of submission.
>>>>
>>>> Would it be possible to support this pattern in io_uring or are there
>>>> reasons for why things are the way they are?
>>>
>>> It should already be supported. And errors look a bit odd, I'd rather
>>> expect -EBADF or some other for accept and -ECANCELED for the read.
>>> Do you have a test program / reporoducer? Hopefully in C.
>>
>> Of course, please see below. Error handling elided for brevity. Hope
>> I'm not doing anything stupid.
>
> Perfect thanks
>
>> For me it immediately prints this:
>>
>> 0 res=-125
>> 1 res=-9
>
> The reason is that in older kernels we're resolving the read's
> file not after accept but when assembling the link, which was
> specifically fixed a bit later.
Right, IORING_FEAT_LINKED_FILE can be checked to see if this is
properly supported or not on the host.
> Jens, are there any plans to backport it?
If I recall I briefly looked at it, but it was a bit more involved
that I would've liked. But then it got simplified a bit after the
fact, so should probably be doable to get into 5.15-stable at least.
Anything earlier than that stable wise is too old anyway.
--
Jens Axboe
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2022-09-28 13:50 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-09-28 9:50 Chaining accept+read Ben Noordhuis
2022-09-28 10:00 ` Pavel Begunkov
2022-09-28 10:55 ` Ben Noordhuis
2022-09-28 11:59 ` Pavel Begunkov
2022-09-28 13:49 ` Jens Axboe
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox