questions about io_uring buffer select feature

questions about io_uring buffer select feature

[Xiaoguang Wang]

hi,

I spent some time to learn the history of buffer select feature, especially
from https://lwn.net/Articles/813311/. According to the description in this
link:
    when doing the same IORING_OP_RECV, no buffer is passed in
    with the request. Instead, it's flagged with IOSQE_BUFFER_SELECT, and
    sqe->buf_group is filled in with a valid group ID. When the kernel can
    satisfy the receive, a buffer is selected from the specified group ID
    pool. If none are available, the IO is terminated with -ENOBUFS. On
    success, the buffer ID is passed back through the (CQE) completion
    event. This tells the application what specific buffer was used.

According to my understandings, buffer select feature is suggested to be
used with fast-poll feature, then in example of io_read(), for the first nowait
try, io_read() will always get one io_buffer even later there is no data
ready, eagain is returned and this req will enter io_arm_poll_handler().
So it seems that this behaviour violates the rule that buffer is only selected
when data is ready?

And for ENOBUFS error, how should apps handle this error? Re-provide
buffers and re-issue requests from user space again? Thanks.


Regards,
Xiaoguang Wang

Re: questions about io_uring buffer select feature

[Pavel Begunkov]

On 4/14/22 08:41, Xiaoguang Wang wrote:
> hi,
> 
> I spent some time to learn the history of buffer select feature, especially
> from https://lwn.net/Articles/813311/. According to the description in this
> link:
>      when doing the same IORING_OP_RECV, no buffer is passed in
>      with the request. Instead, it's flagged with IOSQE_BUFFER_SELECT, and
>      sqe->buf_group is filled in with a valid group ID. When the kernel can
>      satisfy the receive, a buffer is selected from the specified group ID
>      pool. If none are available, the IO is terminated with -ENOBUFS. On
>      success, the buffer ID is passed back through the (CQE) completion
>      event. This tells the application what specific buffer was used.
> 
> According to my understandings, buffer select feature is suggested to be
> used with fast-poll feature, then in example of io_read(), for the first nowait
> try, io_read() will always get one io_buffer even later there is no data
> ready, eagain is returned and this req will enter io_arm_poll_handler().
> So it seems that this behaviour violates the rule that buffer is only selected
> when data is ready?

Right, that's how it was working, but recently Jens was queueing
patches to fix it, e.g. see io_kbuf_recycle(). I think it was
for 5.18.

> And for ENOBUFS error, how should apps handle this error? Re-provide
> buffers and re-issue requests from user space again? Thanks.

It sounds just right. If the userspace can't re-provide buffers,
I assume it may want to wait for some inflight requests to complete.

-- 
Pavel Begunkov

Re: questions about io_uring buffer select feature

[Xiaoguang Wang]

hi,

> On 4/14/22 08:41, Xiaoguang Wang wrote:
>> hi,
>>
>> I spent some time to learn the history of buffer select feature, especially
>> from https://lwn.net/Articles/813311/. According to the description in this
>> link:
>>      when doing the same IORING_OP_RECV, no buffer is passed in
>>      with the request. Instead, it's flagged with IOSQE_BUFFER_SELECT, and
>>      sqe->buf_group is filled in with a valid group ID. When the kernel can
>>      satisfy the receive, a buffer is selected from the specified group ID
>>      pool. If none are available, the IO is terminated with -ENOBUFS. On
>>      success, the buffer ID is passed back through the (CQE) completion
>>      event. This tells the application what specific buffer was used.
>>
>> According to my understandings, buffer select feature is suggested to be
>> used with fast-poll feature, then in example of io_read(), for the first nowait
>> try, io_read() will always get one io_buffer even later there is no data
>> ready, eagain is returned and this req will enter io_arm_poll_handler().
>> So it seems that this behaviour violates the rule that buffer is only selected
>> when data is ready?
>
> Right, that's how it was working, but recently Jens was queueing
> patches to fix it, e.g. see io_kbuf_recycle(). I think it was
> for 5.18.
I see now, thanks for clarification.

>
>> And for ENOBUFS error, how should apps handle this error? Re-provide
>> buffers and re-issue requests from user space again? Thanks.
>
> It sounds just right. If the userspace can't re-provide buffers,
> I assume it may want to wait for some inflight requests to complete.
ok.

Regards,
Xiaoguang Wang