Re: [PATCH 2/2] io_uring: introduce non-circular SQ

[Pavel Begunkov <asml.silence@gmail.com>]

On 10/14/25 20:46, Caleb Sander Mateos wrote:
> On Tue, Oct 14, 2025 at 12:25 PM Pavel Begunkov <asml.silence@gmail.com> wrote:
>>
>> On 10/14/25 19:37, Caleb Sander Mateos wrote:
>>> On Tue, Oct 14, 2025 at 3:57 AM Pavel Begunkov <asml.silence@gmail.com> wrote:
>> ...>> + * SQEs always start at index 0 in the submission ring instead of using a
>>>> + * wrap around indexing.
>>>> + */
>>>> +#define IORING_SETUP_SQ_REWIND         (1U << 19)
>>>
>>> Keith's mixed-SQE-size patch series is already planning to use this
>>> flag: https://lore.kernel.org/io-uring/20251013180011.134131-3-kbusch@meta.com/
>>
>> I'll rebase it as ususual if that gets merged before.
>>>> -       /*
>>>> -        * Ensure any loads from the SQEs are done at this point,
>>>> -        * since once we write the new head, the application could
>>>> -        * write new data to them.
>>>> -        */
>>>> -       smp_store_release(&rings->sq.head, ctx->cached_sq_head);
>>>> +       if (ctx->flags & IORING_SETUP_SQ_REWIND) {
>>>> +               ctx->cached_sq_head = 0;
>>>
>>> The only awkward thing about this interface seems to be if
>>> io_submit_sqes() aborts early without submitting all the requested
>>> SQEs. Does userspace then need to memmove() the remaining SQEs to the
>>> start of the ring? It's certainly an unlikely case but something
>>> userspace has to handle because io_alloc_req() can fail for reasons
>>> outside its control. Seems like it might simplify the userspace side
>>> if cached_sq_head wasn't rewound if not all SQEs were consumed.
>> This kind of special rules is what usually makes interfaces a pain to
>> work with. What if you want to abort all un-submitted requests
>> instead? You can empty the queue, but then the next syscall will
>> still start from the middle. Or what if the application wants to
>> queue more requests before resubmitting previous ones? There are
>> reasons b/c the kernel will need to handle it in a less elegant way
>> than it potentially can otherwise. memmove sounds appropriate.
> 
> Maybe most convenient would be a way for userspace to pass both a head
> and a nr/tail value to the syscall instead of assuming the head is
> always 0. But it's probably difficult to modify the existing syscall

It feels fine from the API perspective, but you still need head/tail
fetching and care, additional index sanitisation (Spectre), and either
handling wrap around or extra border checks. All minor points, but
the index handling will likely be more annoying than just doing a
memmove.

> interface without an indirection to the head value, which seems to be
> a main point of this series. So always resetting to 0 and requiring
> userspace to memmove() the remaining SQEs in the rare case that
> io_uring_enter() doesn't consume all of them seems like a reasonable
> approach.

If that's what the user wants to do as there are other ways
it could be handled.

-- 
Pavel Begunkov