[LIBURING PATCH] sq_ring_needs_enter: check whether there are sqes when SQPOLL is not enabled

[LIBURING PATCH] sq_ring_needs_enter: check whether there are sqes when SQPOLL is not enabled

[Xiaoguang Wang]

Indeed I'm not sure this patch is necessary, robust applications
should not call io_uring_submit when there are not sqes to submmit.
But still try to add this check, I have seen some applications which
call io_uring_submit(), but there are not sqes to submit.

Signed-off-by: Xiaoguang Wang <xiaoguang.wang@linux.alibaba.com>
---
 src/queue.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/src/queue.c b/src/queue.c
index e563775..75d707d 100644
--- a/src/queue.c
+++ b/src/queue.c
@@ -19,9 +19,10 @@
  * or if IORING_SQ_NEED_WAKEUP is set, so submit thread must be explicitly
  * awakened. For the latter case, we set the thread wakeup flag.
  */
-static inline bool sq_ring_needs_enter(struct io_uring *ring, unsigned *flags)
+static inline bool sq_ring_needs_enter(struct io_uring *ring,
+				unsigned submitted, unsigned *flags)
 {
-	if (!(ring->flags & IORING_SETUP_SQPOLL))
+	if (!(ring->flags & IORING_SETUP_SQPOLL) && submitted)
 		return true;
 	if (IO_URING_READ_ONCE(*ring->sq.kflags) & IORING_SQ_NEED_WAKEUP) {
 		*flags |= IORING_ENTER_SQ_WAKEUP;
@@ -51,7 +52,7 @@ int __io_uring_get_cqe(struct io_uring *ring, struct io_uring_cqe **cqe_ptr,
 		if (wait_nr)
 			flags = IORING_ENTER_GETEVENTS;
 		if (submit)
-			sq_ring_needs_enter(ring, &flags);
+			sq_ring_needs_enter(ring, submit, &flags);
 		if (wait_nr || submit)
 			ret = __sys_io_uring_enter(ring->ring_fd, submit,
 						   wait_nr, flags, sigmask);
@@ -197,7 +198,7 @@ static int __io_uring_submit(struct io_uring *ring, unsigned submitted,
 	int ret;
 
 	flags = 0;
-	if (sq_ring_needs_enter(ring, &flags) || wait_nr) {
+	if (sq_ring_needs_enter(ring, submitted, &flags) || wait_nr) {
 		if (wait_nr || (ring->flags & IORING_SETUP_IOPOLL))
 			flags |= IORING_ENTER_GETEVENTS;
 
-- 
2.17.2

Re: [LIBURING PATCH] sq_ring_needs_enter: check whether there are sqes when SQPOLL is not enabled

[Jens Axboe]

On 4/13/20 1:19 AM, Xiaoguang Wang wrote:
> Indeed I'm not sure this patch is necessary, robust applications
> should not call io_uring_submit when there are not sqes to submmit.
> But still try to add this check, I have seen some applications which
> call io_uring_submit(), but there are not sqes to submit.

Hmm, not sure it's worth complicating the submit path for that case.
A high performant application should not call io_uring_submit() if
it didn't queue anything new. Is this a common case you've seen?

-- 
Jens Axboe

Re: [LIBURING PATCH] sq_ring_needs_enter: check whether there are sqes when SQPOLL is not enabled

[Hrvoje Zeba]

On Mon, Apr 13, 2020 at 11:29 AM Jens Axboe <axboe@kernel.dk> wrote:
>
> On 4/13/20 1:19 AM, Xiaoguang Wang wrote:
> > Indeed I'm not sure this patch is necessary, robust applications
> > should not call io_uring_submit when there are not sqes to submmit.
> > But still try to add this check, I have seen some applications which
> > call io_uring_submit(), but there are not sqes to submit.
>
> Hmm, not sure it's worth complicating the submit path for that case.
> A high performant application should not call io_uring_submit() if
> it didn't queue anything new. Is this a common case you've seen?
>

My code calls io_uring_submit() even if there are no sqes to submit to
avoid spinning if there's nothing to do:

...
uint32_t sleep = (gt::user_contexts_waiting() > 0) ? 0 : 1;
auto res = io_uring_submit_and_wait(&m_io_uring, sleep);
...

-- 
I doubt, therefore I might be.

Re: [LIBURING PATCH] sq_ring_needs_enter: check whether there are sqes when SQPOLL is not enabled

[Jens Axboe]

On 4/13/20 9:46 AM, Hrvoje Zeba wrote:
> On Mon, Apr 13, 2020 at 11:29 AM Jens Axboe <axboe@kernel.dk> wrote:
>>
>> On 4/13/20 1:19 AM, Xiaoguang Wang wrote:
>>> Indeed I'm not sure this patch is necessary, robust applications
>>> should not call io_uring_submit when there are not sqes to submmit.
>>> But still try to add this check, I have seen some applications which
>>> call io_uring_submit(), but there are not sqes to submit.
>>
>> Hmm, not sure it's worth complicating the submit path for that case.
>> A high performant application should not call io_uring_submit() if
>> it didn't queue anything new. Is this a common case you've seen?
>>
> 
> My code calls io_uring_submit() even if there are no sqes to submit to
> avoid spinning if there's nothing to do:
> 
> ...
> uint32_t sleep = (gt::user_contexts_waiting() > 0) ? 0 : 1;
> auto res = io_uring_submit_and_wait(&m_io_uring, sleep);

If you're calling with wait, then an sqe will be submitted with a
timeout operation. So that use case is fine. Or waiting in general.
But calling with {0,0} for {submit,wait} would be kind of silly.

-- 
Jens Axboe

Re: [LIBURING PATCH] sq_ring_needs_enter: check whether there are sqes when SQPOLL is not enabled

[Jens Axboe]

On 4/13/20 8:18 AM, Jens Axboe wrote:
> On 4/13/20 1:19 AM, Xiaoguang Wang wrote:
>> Indeed I'm not sure this patch is necessary, robust applications
>> should not call io_uring_submit when there are not sqes to submmit.
>> But still try to add this check, I have seen some applications which
>> call io_uring_submit(), but there are not sqes to submit.
> 
> Hmm, not sure it's worth complicating the submit path for that case.
> A high performant application should not call io_uring_submit() if
> it didn't queue anything new. Is this a common case you've seen?

Looking at it again, it actually seems quite reasonable. I've queued
it up, thanks!

-- 
Jens Axboe