[RFC] io_uring: submit even with overflow backlog

[RFC] io_uring: submit even with overflow backlog

[Pavel Begunkov]

Not submitting when have requests in overflow backlog looks artificial,
and limits users for no clear purpose, especially since requests with
resources are now not locked into it but it consists for a small memory
area. Remove the restriction.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
---

Mainly for discussion. It breaks several tests, and so in theory
userspace, but can't think this restriction not being just a
nuisance to the userspace. IMHO much more convenient to allow it,
userspace can take care of it itself if needed, but for those
who don't care and use rings in parallel (e.g. different threads
for submission and completion), it will be hell of a synchronisation.

 fs/io_uring.c | 6 ------
 1 file changed, 6 deletions(-)

diff --git a/fs/io_uring.c b/fs/io_uring.c
index 357993e3e0d2..21026653e1c1 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -6673,12 +6673,6 @@ static int io_submit_sqes(struct io_ring_ctx *ctx, unsigned int nr)
 {
 	int submitted = 0;
 
-	/* if we have a backlog and couldn't flush it all, return BUSY */
-	if (test_bit(0, &ctx->sq_check_overflow)) {
-		if (!__io_cqring_overflow_flush(ctx, false))
-			return -EBUSY;
-	}
-
 	/* make sure SQ entry isn't read before tail */
 	nr = min3(nr, ctx->sq_entries, io_sqring_entries(ctx));
 
-- 
2.24.0

Re: [RFC] io_uring: submit even with overflow backlog

[Jens Axboe]

On 4/15/21 6:40 AM, Pavel Begunkov wrote:
> Not submitting when have requests in overflow backlog looks artificial,
> and limits users for no clear purpose, especially since requests with
> resources are now not locked into it but it consists for a small memory
> area. Remove the restriction.
> 
> Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
> ---
> 
> Mainly for discussion. It breaks several tests, and so in theory
> userspace, but can't think this restriction not being just a
> nuisance to the userspace. IMHO much more convenient to allow it,
> userspace can take care of it itself if needed, but for those
> who don't care and use rings in parallel (e.g. different threads
> for submission and completion), it will be hell of a synchronisation.

I think we can kill it, with the main change enabling that being the
cgroup memory accounting. This was kind of a safe guard to avoid having
silly cases just go way overboard, but I do agree that this is really
up to the application to manage. And it may have an easier time doing
so without -EBUSY on overflow being set on submit.

-- 
Jens Axboe

Re: [RFC] io_uring: submit even with overflow backlog

[Pavel Begunkov]

On 15/04/2021 14:59, Jens Axboe wrote:
> On 4/15/21 6:40 AM, Pavel Begunkov wrote:
>> Not submitting when have requests in overflow backlog looks artificial,
>> and limits users for no clear purpose, especially since requests with
>> resources are now not locked into it but it consists for a small memory
>> area. Remove the restriction.
>>
>> Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
>> ---
>>
>> Mainly for discussion. It breaks several tests, and so in theory
>> userspace, but can't think this restriction not being just a
>> nuisance to the userspace. IMHO much more convenient to allow it,
>> userspace can take care of it itself if needed, but for those
>> who don't care and use rings in parallel (e.g. different threads
>> for submission and completion), it will be hell of a synchronisation.
> 
> I think we can kill it, with the main change enabling that being the
> cgroup memory accounting. This was kind of a safe guard to avoid having

Yeah, and not keeping requests, its resources and linking requests.

> silly cases just go way overboard, but I do agree that this is really
> up to the application to manage. And it may have an easier time doing
> so without -EBUSY on overflow being set on submit.

Btw, apparently that's the main reason for getting -EBUSY in mshot test,
just another lackmustest that it's not convenient.

-- 
Pavel Begunkov

[PATCH] io_uring: don't fail submit with overflow backlog

[Pavel Begunkov]

Don't fail submission attempts if there are CQEs in the overflow
backlog, but give away the decision making to the userspace. It
might be very inconvenient to the userspace, especially if
submission and completion are done by different threads.

We can remove it because of recent changes, where requests
are now not locked by the backlog, backlog entries are allocated
separately, so they take less space and cgroup accounted. 

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
---

since RFC: commit message rewording

 fs/io_uring.c | 6 ------
 1 file changed, 6 deletions(-)

diff --git a/fs/io_uring.c b/fs/io_uring.c
index 357993e3e0d2..21026653e1c1 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -6673,12 +6673,6 @@ static int io_submit_sqes(struct io_ring_ctx *ctx, unsigned int nr)
 {
 	int submitted = 0;
 
-	/* if we have a backlog and couldn't flush it all, return BUSY */
-	if (test_bit(0, &ctx->sq_check_overflow)) {
-		if (!__io_cqring_overflow_flush(ctx, false))
-			return -EBUSY;
-	}
-
 	/* make sure SQ entry isn't read before tail */
 	nr = min3(nr, ctx->sq_entries, io_sqring_entries(ctx));
 
-- 
2.24.0