[PATCH v1] io_uring_cqe_get_data() only requires a const struct io_uring_cqe *cqe

[PATCH v1] io_uring_cqe_get_data() only requires a const struct io_uring_cqe *cqe

[Stefan Metzmacher]

Signed-off-by: Stefan Metzmacher <metze@samba.org>
---
 src/include/liburing.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/include/liburing.h b/src/include/liburing.h
index faed2e7..44f18fd 100644
--- a/src/include/liburing.h
+++ b/src/include/liburing.h
@@ -179,7 +179,7 @@ static inline void io_uring_sqe_set_data(struct io_uring_sqe *sqe, void *data)
 	sqe->user_data = (unsigned long) data;
 }
 
-static inline void *io_uring_cqe_get_data(struct io_uring_cqe *cqe)
+static inline void *io_uring_cqe_get_data(const struct io_uring_cqe *cqe)
 {
 	return (void *) (uintptr_t) cqe->user_data;
 }
-- 
2.17.1

Re: [PATCH v1] io_uring_cqe_get_data() only requires a const struct io_uring_cqe *cqe

[Jens Axboe]

On 2/6/20 9:02 AM, Stefan Metzmacher wrote:
> Signed-off-by: Stefan Metzmacher <metze@samba.org>
> ---
>  src/include/liburing.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/src/include/liburing.h b/src/include/liburing.h
> index faed2e7..44f18fd 100644
> --- a/src/include/liburing.h
> +++ b/src/include/liburing.h
> @@ -179,7 +179,7 @@ static inline void io_uring_sqe_set_data(struct io_uring_sqe *sqe, void *data)
>  	sqe->user_data = (unsigned long) data;
>  }
>  
> -static inline void *io_uring_cqe_get_data(struct io_uring_cqe *cqe)
> +static inline void *io_uring_cqe_get_data(const struct io_uring_cqe *cqe)
>  {
>  	return (void *) (uintptr_t) cqe->user_data;
>  }

Applied, thanks.

Unrelated to this patch, but I'd like to release a 0.4 sooner rather
than later. Let me know if you see any immediate work that needs doing
before that happens.

-- 
Jens Axboe

Re: [PATCH v1] io_uring_cqe_get_data() only requires a const struct io_uring_cqe *cqe

[Stefan Metzmacher]

[-- Attachment #1.1: Type: text/plain, Size: 2481 bytes --]

Am 06.02.20 um 17:04 schrieb Jens Axboe:
> On 2/6/20 9:02 AM, Stefan Metzmacher wrote:
>> Signed-off-by: Stefan Metzmacher <metze@samba.org>
>> ---
>>  src/include/liburing.h | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/src/include/liburing.h b/src/include/liburing.h
>> index faed2e7..44f18fd 100644
>> --- a/src/include/liburing.h
>> +++ b/src/include/liburing.h
>> @@ -179,7 +179,7 @@ static inline void io_uring_sqe_set_data(struct io_uring_sqe *sqe, void *data)
>>  	sqe->user_data = (unsigned long) data;
>>  }
>>  
>> -static inline void *io_uring_cqe_get_data(struct io_uring_cqe *cqe)
>> +static inline void *io_uring_cqe_get_data(const struct io_uring_cqe *cqe)
>>  {
>>  	return (void *) (uintptr_t) cqe->user_data;
>>  }
> 
> Applied, thanks.

Thanks!

> Unrelated to this patch, but I'd like to release a 0.4 sooner rather
> than later.

Funny, I thought about that today:-)
I prepared debian packaging for liburing-0.4 I'll send the patches soon.

While doing that I found the following incompatible change against
liburing-0.3:

 static inline void io_uring_prep_files_update(struct io_uring_sqe *sqe,
-                                             int *fds, unsigned nr_fds)
+                                             int *fds, unsigned nr_fds,
+                                             int offset)

I'm not sure if we should do something about that.

It's also strange that in src/liburing.map LIBURING_0.3 doesn't
inherit LIBURING_0.2. There's not a single symbol with @LIBURING_0.3.
Also io_uring_{setup,enter,register} are still
listed under LIBURING_0.1, but they're not in the library anymore.

> Let me know if you see any immediate work that needs doing
> before that happens.

I had the idea to have a simple version of fd compounding.
We already have IORING_OP_FILES_UPDATE in order to update
specific indexes in the files array.
I'm wondering if we could have specify an index where
IORING_OP_ACCEPT, IORING_OP_OPENAT and IORING_OP_OPENAT2
could store the generated fd into the fixed array.
The index 0 is not valid, correct? So we can have it
without a flag similar to the personality, and for
all of these buf_index is not used.

While researching that I noticed that IOSQE_FIXED_FILE
seems to be ignored for some new commands, I think that
all commands with on input fd, should be able to use that flag.
Can this be fixed before 5.6 final?

metze


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v1] io_uring_cqe_get_data() only requires a const struct io_uring_cqe *cqe

[Jens Axboe]

On 2/6/20 9:37 AM, Stefan Metzmacher wrote:
> Am 06.02.20 um 17:04 schrieb Jens Axboe:
>> On 2/6/20 9:02 AM, Stefan Metzmacher wrote:
>>> Signed-off-by: Stefan Metzmacher <metze@samba.org>
>>> ---
>>>  src/include/liburing.h | 2 +-
>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/src/include/liburing.h b/src/include/liburing.h
>>> index faed2e7..44f18fd 100644
>>> --- a/src/include/liburing.h
>>> +++ b/src/include/liburing.h
>>> @@ -179,7 +179,7 @@ static inline void io_uring_sqe_set_data(struct io_uring_sqe *sqe, void *data)
>>>  	sqe->user_data = (unsigned long) data;
>>>  }
>>>  
>>> -static inline void *io_uring_cqe_get_data(struct io_uring_cqe *cqe)
>>> +static inline void *io_uring_cqe_get_data(const struct io_uring_cqe *cqe)
>>>  {
>>>  	return (void *) (uintptr_t) cqe->user_data;
>>>  }
>>
>> Applied, thanks.
> 
> Thanks!
> 
>> Unrelated to this patch, but I'd like to release a 0.4 sooner rather
>> than later.
> 
> Funny, I thought about that today:-)
> I prepared debian packaging for liburing-0.4 I'll send the patches soon.

Great!

> While doing that I found the following incompatible change against
> liburing-0.3:
> 
>  static inline void io_uring_prep_files_update(struct io_uring_sqe *sqe,
> -                                             int *fds, unsigned nr_fds)
> +                                             int *fds, unsigned nr_fds,
> +                                             int offset)
> 
> I'm not sure if we should do something about that.

Hmm, that wasn't on purpose. But for this specific case, I think we can
just pretend that never happened.

> It's also strange that in src/liburing.map LIBURING_0.3 doesn't
> inherit LIBURING_0.2. There's not a single symbol with @LIBURING_0.3.
> Also io_uring_{setup,enter,register} are still
> listed under LIBURING_0.1, but they're not in the library anymore.

That seems like a bug, I'd happily take a patch for that...

>> Let me know if you see any immediate work that needs doing
>> before that happens.
> 
> I had the idea to have a simple version of fd compounding.
> We already have IORING_OP_FILES_UPDATE in order to update
> specific indexes in the files array.
> I'm wondering if we could have specify an index where
> IORING_OP_ACCEPT, IORING_OP_OPENAT and IORING_OP_OPENAT2
> could store the generated fd into the fixed array.
> The index 0 is not valid, correct? So we can have it
> without a flag similar to the personality, and for
> all of these buf_index is not used.

Just to make sure I'm undestanding your proposal, you want the result
from those fd instantiating calls to be added to the array of registered
files, instead of having the application do that? If so, I think this is
another case where the BPF driven links would be useful, as we could
easily do it through that with an IORING_OP_FILES_UPDATE linked to
either one of those commands.

index 0 is valid, so we can't use that trick.

> While researching that I noticed that IOSQE_FIXED_FILE
> seems to be ignored for some new commands, I think that
> all commands with on input fd, should be able to use that flag.
> Can this be fixed before 5.6 final?

Do you have specifics? Generally the file grabbing happens as part of
request prep, and the individual opcodes should not need to bother with
it.

-- 
Jens Axboe

Re: [PATCH v1] io_uring_cqe_get_data() only requires a const struct io_uring_cqe *cqe

[Stefan Metzmacher]

[-- Attachment #1.1: Type: text/plain, Size: 4370 bytes --]

Am 06.02.20 um 17:42 schrieb Jens Axboe:
> On 2/6/20 9:37 AM, Stefan Metzmacher wrote:
>> Am 06.02.20 um 17:04 schrieb Jens Axboe:
>>> On 2/6/20 9:02 AM, Stefan Metzmacher wrote:
>>>> Signed-off-by: Stefan Metzmacher <metze@samba.org>
>>>> ---
>>>>  src/include/liburing.h | 2 +-
>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/src/include/liburing.h b/src/include/liburing.h
>>>> index faed2e7..44f18fd 100644
>>>> --- a/src/include/liburing.h
>>>> +++ b/src/include/liburing.h
>>>> @@ -179,7 +179,7 @@ static inline void io_uring_sqe_set_data(struct io_uring_sqe *sqe, void *data)
>>>>  	sqe->user_data = (unsigned long) data;
>>>>  }
>>>>  
>>>> -static inline void *io_uring_cqe_get_data(struct io_uring_cqe *cqe)
>>>> +static inline void *io_uring_cqe_get_data(const struct io_uring_cqe *cqe)
>>>>  {
>>>>  	return (void *) (uintptr_t) cqe->user_data;
>>>>  }
>>>
>>> Applied, thanks.
>>
>> Thanks!
>>
>>> Unrelated to this patch, but I'd like to release a 0.4 sooner rather
>>> than later.
>>
>> Funny, I thought about that today:-)
>> I prepared debian packaging for liburing-0.4 I'll send the patches soon.
> 
> Great!
> 
>> While doing that I found the following incompatible change against
>> liburing-0.3:
>>
>>  static inline void io_uring_prep_files_update(struct io_uring_sqe *sqe,
>> -                                             int *fds, unsigned nr_fds)
>> +                                             int *fds, unsigned nr_fds,
>> +                                             int offset)
>>
>> I'm not sure if we should do something about that.
> 
> Hmm, that wasn't on purpose. But for this specific case, I think we can
> just pretend that never happened.

Ok.

>> It's also strange that in src/liburing.map LIBURING_0.3 doesn't
>> inherit LIBURING_0.2. There's not a single symbol with @LIBURING_0.3.
>> Also io_uring_{setup,enter,register} are still
>> listed under LIBURING_0.1, but they're not in the library anymore.
> 
> That seems like a bug, I'd happily take a patch for that...

I'm not sure how what to change, so I'll just leave it for now.

>>> Let me know if you see any immediate work that needs doing
>>> before that happens.
>>
>> I had the idea to have a simple version of fd compounding.
>> We already have IORING_OP_FILES_UPDATE in order to update
>> specific indexes in the files array.
>> I'm wondering if we could have specify an index where
>> IORING_OP_ACCEPT, IORING_OP_OPENAT and IORING_OP_OPENAT2
>> could store the generated fd into the fixed array.
>> The index 0 is not valid, correct? So we can have it
>> without a flag similar to the personality, and for
>> all of these buf_index is not used.
> 
> Just to make sure I'm undestanding your proposal, you want the result
> from those fd instantiating calls to be added to the array of registered
> files, instead of having the application do that?

Yes.

> If so, I think this is another case where the BPF driven links would be useful, as we could
> easily do it through that with an IORING_OP_FILES_UPDATE linked to
> either one of those commands.

Sure, it's just hard to imagine how to do BPF without it being there and
a real world examples...

> index 0 is valid, so we can't use that trick.

Ok :-)

>> While researching that I noticed that IOSQE_FIXED_FILE
>> seems to be ignored for some new commands, I think that
>> all commands with on input fd, should be able to use that flag.
>> Can this be fixed before 5.6 final?
> 
> Do you have specifics? Generally the file grabbing happens as part of
> request prep, and the individual opcodes should not need to bother with
> it.

io_statx_prep():
io_openat_prep():
io_openat2_prep():

  req->open.dfd = READ_ONCE(sqe->fd);


io_statx():
  ret = filename_lookup(ctx->dfd, ctx->filename, lookup_flags...

io_openat2():

  file = do_filp_open(req->open.dfd, req->open.filename, &op);

io_close_prep(): has this to make it clear that IOSQE_FIXED_FILE is not
supported, I guess because FILE_UPDATE with -1 needs to be used instead?

        if (sqe->flags & IOSQE_FIXED_FILE)
                return -EINVAL;

        req->close.fd = READ_ONCE(sqe->fd);


I guess at least we need if (sqe->flags & IOSQE_FIXED_FILE) in all
cases, if we can't just fix it.

metze


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v1] io_uring_cqe_get_data() only requires a const struct io_uring_cqe *cqe

[Jens Axboe]

On 2/6/20 10:05 AM, Stefan Metzmacher wrote:
>>> While researching that I noticed that IOSQE_FIXED_FILE
>>> seems to be ignored for some new commands, I think that
>>> all commands with on input fd, should be able to use that flag.
>>> Can this be fixed before 5.6 final?
>>
>> Do you have specifics? Generally the file grabbing happens as part of
>> request prep, and the individual opcodes should not need to bother with
>> it.
> 
> io_statx_prep():
> io_openat_prep():
> io_openat2_prep():
> 
>   req->open.dfd = READ_ONCE(sqe->fd);
> 
> 
> io_statx():
>   ret = filename_lookup(ctx->dfd, ctx->filename, lookup_flags...
> 
> io_openat2():
> 
>   file = do_filp_open(req->open.dfd, req->open.filename, &op);
> 
> io_close_prep(): has this to make it clear that IOSQE_FIXED_FILE is not
> supported, I guess because FILE_UPDATE with -1 needs to be used instead?
> 
>         if (sqe->flags & IOSQE_FIXED_FILE)
>                 return -EINVAL;
> 
>         req->close.fd = READ_ONCE(sqe->fd);
> 
> 
> I guess at least we need if (sqe->flags & IOSQE_FIXED_FILE) in all
> cases, if we can't just fix it.

Ah yes good point, on both honoring it and failing it for close() when
we do honor it. I'll fix that up.

-- 
Jens Axboe

Re: [PATCH v1] io_uring_cqe_get_data() only requires a const struct io_uring_cqe *cqe

[Stefan Metzmacher]

[-- Attachment #1.1: Type: text/plain, Size: 366 bytes --]

Hi Jens,

> Unrelated to this patch, but I'd like to release a 0.4 sooner rather
> than later. Let me know if you see any immediate work that needs doing
> before that happens.

I just noticed that IORING_FEAT_CUR_PERSONALITY is not yet documented,
I'm not sure if that's important enough to have in 0.4.

When do you plan to tag the release?

metze



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v1] io_uring_cqe_get_data() only requires a const struct io_uring_cqe *cqe

[Jens Axboe]

On 2/7/20 4:45 PM, Stefan Metzmacher wrote:
> Hi Jens,
> 
>> Unrelated to this patch, but I'd like to release a 0.4 sooner rather
>> than later. Let me know if you see any immediate work that needs doing
>> before that happens.
> 
> I just noticed that IORING_FEAT_CUR_PERSONALITY is not yet documented,
> I'm not sure if that's important enough to have in 0.4.

Ah good catch, I'll add that.

> When do you plan to tag the release?

Sometime next week, need to do the above doc addition and the debian
guys had something as well.

-- 
Jens Axboe