public inbox for [email protected]
 help / color / mirror / Atom feed
From: David Wei <[email protected]>
To: Jens Axboe <[email protected]>, [email protected]
Subject: Re: [PATCH 3/5] io_uring: implement our own schedule timeout handling
Date: Tue, 20 Aug 2024 15:06:06 -0700	[thread overview]
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>

On 2024-08-20 14:39, Jens Axboe wrote:
> On 8/20/24 3:37 PM, David Wei wrote:
>> On 2024-08-20 14:34, Jens Axboe wrote:
>>> On 8/20/24 2:08 PM, David Wei wrote:
>>>> On 2024-08-19 16:28, Jens Axboe wrote:
>>>>> In preparation for having two distinct timeouts and avoid waking the
>>>>> task if we don't need to.
>>>>>
>>>>> Signed-off-by: Jens Axboe <[email protected]>
>>>>> ---
>>>>>  io_uring/io_uring.c | 41 ++++++++++++++++++++++++++++++++++++-----
>>>>>  io_uring/io_uring.h |  2 ++
>>>>>  2 files changed, 38 insertions(+), 5 deletions(-)
>>>>>
>>>>> diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c
>>>>> index 9e2b8d4c05db..ddfbe04c61ed 100644
>>>>> --- a/io_uring/io_uring.c
>>>>> +++ b/io_uring/io_uring.c
>>>>> @@ -2322,7 +2322,7 @@ static int io_wake_function(struct wait_queue_entry *curr, unsigned int mode,
>>>>>  	 * Cannot safely flush overflowed CQEs from here, ensure we wake up
>>>>>  	 * the task, and the next invocation will do it.
>>>>>  	 */
>>>>> -	if (io_should_wake(iowq) || io_has_work(iowq->ctx))
>>>>> +	if (io_should_wake(iowq) || io_has_work(iowq->ctx) || iowq->hit_timeout)
>>>>
>>>> iowq->hit_timeout may be modified in a timer softirq context, while this
>>>> wait_queue_func_t (AIUI) may get called from any context e.g.
>>>> net_rx_softirq for sockets. Does this need a READ_ONLY()?
>>>
>>> Yes probably not a bad idea to make it READ_ONCE().
>>>
>>>>>  		return autoremove_wake_function(curr, mode, wake_flags, key);
>>>>>  	return -1;
>>>>>  }
>>>>> @@ -2350,6 +2350,38 @@ static bool current_pending_io(void)
>>>>>  	return percpu_counter_read_positive(&tctx->inflight);
>>>>>  }
>>>>>  
>>>>> +static enum hrtimer_restart io_cqring_timer_wakeup(struct hrtimer *timer)
>>>>> +{
>>>>> +	struct io_wait_queue *iowq = container_of(timer, struct io_wait_queue, t);
>>>>> +	struct io_ring_ctx *ctx = iowq->ctx;
>>>>> +
>>>>> +	WRITE_ONCE(iowq->hit_timeout, 1);
>>>>> +	if (ctx->flags & IORING_SETUP_DEFER_TASKRUN)
>>>>> +		wake_up_process(ctx->submitter_task);
>>>>> +	else
>>>>> +		io_cqring_wake(ctx);
>>>>
>>>> This is a bit different to schedule_hrtimeout_range_clock(). Why is
>>>> io_cqring_wake() needed here for non-DEFER_TASKRUN?
>>>
>>> That's how the wakeups work - for defer taskrun, the task isn't on a
>>> waitqueue at all. Hence we need to wake the task itself. For any other
>>> setup, they will be on the waitqueue, and we just call io_cqring_wake()
>>> to wake up anyone waiting on the waitqueue. That will iterate the wake
>>> queue and call handlers for each item. Having a separate handler for
>>> that will allow to NOT wake up the task if we don't need to.
>>> taskrun, the waker
>>
>> To rephase the question, why is the original code calling
>> schedule_hrtimeout_range_clock() not needing to differentiate behaviour
>> between defer taskrun and not?
> 
> Because that part is the same, the task schedules out and goes to sleep.
> That has always been the same regardless of how the ring is setup. Only
> difference is that DEFER_TASKRUN doesn't add itself to ctx->wait, and
> hence cannot be woken by a wake_up(ctx->wait). We have to wake the task
> manually.
> 

io_cqring_timer_wakeup() is the timer expired callback which calls
wake_up_process() or io_cqring_wake() depending on DEFER_TASKRUN.

The original code calling schedule_hrtimeout_range_clock() uses
hrtimer_sleeper instead, which has a default timer expired callback set
to hrtimer_wakeup().

hrtimer_wakeup() only calls wake_up_process().

My question is: why this asymmetry? Why does the new custom callback
require io_cqring_wake()?

  parent reply	other threads:[~2024-08-20 22:06 UTC|newest]

Thread overview: 31+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-08-19 23:28 [PATCHSET v4 0/5] Add support for batched min timeout Jens Axboe
2024-08-19 23:28 ` [PATCH 1/5] io_uring: encapsulate extraneous wait flags into a separate struct Jens Axboe
2024-08-19 23:28 ` [PATCH 2/5] io_uring: move schedule wait logic into helper Jens Axboe
2024-08-19 23:28 ` [PATCH 3/5] io_uring: implement our own schedule timeout handling Jens Axboe
2024-08-20 20:08   ` David Wei
2024-08-20 21:34     ` Jens Axboe
2024-08-20 21:37       ` David Wei
2024-08-20 21:39         ` Jens Axboe
2024-08-20 22:04           ` Pavel Begunkov
2024-08-20 22:06           ` David Wei [this message]
2024-08-20 22:13             ` Pavel Begunkov
2024-08-20 22:14               ` David Wei
2024-08-20 22:19                 ` Jens Axboe
2024-08-20 22:51                   ` Jens Axboe
2024-08-20 22:54                     ` Jens Axboe
2024-08-19 23:28 ` [PATCH 4/5] io_uring: add support for batch wait timeout Jens Axboe
2024-08-20 21:10   ` David Wei
2024-08-20 21:31     ` Jens Axboe
2024-08-20 21:59       ` David Wei
2024-08-20 21:36     ` Jens Axboe
2024-08-20 22:08       ` Pavel Begunkov
2024-08-20 22:46   ` Pavel Begunkov
2024-08-20 22:47     ` Pavel Begunkov
2024-08-20 22:58       ` Jens Axboe
2024-08-21  0:08         ` Pavel Begunkov
2024-08-21 14:22           ` Jens Axboe
2024-08-19 23:28 ` [PATCH 5/5] io_uring: wire up min batch wake timeout Jens Axboe
  -- strict thread matches above, loose matches on Subject: below --
2024-08-21 14:16 [PATCHSET v5 0/5] Add support for batched min timeout Jens Axboe
2024-08-21 14:16 ` [PATCH 3/5] io_uring: implement our own schedule timeout handling Jens Axboe
2024-08-22 13:22   ` Pavel Begunkov
2024-08-22 15:27     ` Jens Axboe
2024-08-16 20:38 [PATCHSET v3 0/5] Add support for batched min timeout Jens Axboe
2024-08-16 20:38 ` [PATCH 3/5] io_uring: implement our own schedule timeout handling Jens Axboe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox