[PATCH 0/4] iopoll fixes + cleanups

[PATCH 0/4] iopoll fixes + cleanups

[Pavel Begunkov]

[1] fixes a recent for-stable patch, should be for stable as well.
[2] fixes getting mm from a wrong task in iopoll path.

[3,4] are unrelated cleanups. I don't send them separately because
they may conflict.

Pavel Begunkov (4):
  io_uring: fix hanging iopoll in case of -EAGAIN
  io_uring: handle EAGAIN iopoll
  io-wq: compact io-wq flags numbers
  io-wq: return next work from ->do_work() directly

 fs/io-wq.c    |  8 +++---
 fs/io-wq.h    | 10 ++++----
 fs/io_uring.c | 71 ++++++++++++++++++++++++---------------------------
 3 files changed, 41 insertions(+), 48 deletions(-)

-- 
2.24.0

[PATCH 1/4] io_uring: fix hanging iopoll in case of -EAGAIN

[Pavel Begunkov]

io_do_iopoll() won't do anything with a request unless
req->iopoll_completed is set. So io_complete_rw_iopoll() has to set
it, otherwise io_do_iopoll() will poll a file again and again even
though the request of interest was completed long ago.

Fixes: bbde017a32b3 ("io_uring: add memory barrier to synchronize
io_kiocb's result and iopoll_completed")
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
---
 fs/io_uring.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/fs/io_uring.c b/fs/io_uring.c
index c686061c3762..bb0dfc450db5 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -2104,10 +2104,8 @@ static void io_complete_rw_iopoll(struct kiocb *kiocb, long res, long res2)
 
 	WRITE_ONCE(req->result, res);
 	/* order with io_poll_complete() checking ->result */
-	if (res != -EAGAIN) {
-		smp_wmb();
-		WRITE_ONCE(req->iopoll_completed, 1);
-	}
+	smp_wmb();
+	WRITE_ONCE(req->iopoll_completed, 1);
 }
 
 /*
-- 
2.24.0

Re: [PATCH 1/4] io_uring: fix hanging iopoll in case of -EAGAIN

[Jens Axboe]

On 6/22/20 4:16 PM, Pavel Begunkov wrote:
> io_do_iopoll() won't do anything with a request unless
> req->iopoll_completed is set. So io_complete_rw_iopoll() has to set
> it, otherwise io_do_iopoll() will poll a file again and again even
> though the request of interest was completed long ago.

I need to look at this again, because with this change, I previously
got various use-after-free. I haven't seen any issues with it, but
I agree, from a quick look that I'm not quite sure how it's currently
not causing hangs. Yet I haven't seen any, with targeted -EAGAIN
testing.

-- 
Jens Axboe

Re: [PATCH 1/4] io_uring: fix hanging iopoll in case of -EAGAIN

[Jens Axboe]

On 6/22/20 8:07 PM, Jens Axboe wrote:
> On 6/22/20 4:16 PM, Pavel Begunkov wrote:
>> io_do_iopoll() won't do anything with a request unless
>> req->iopoll_completed is set. So io_complete_rw_iopoll() has to set
>> it, otherwise io_do_iopoll() will poll a file again and again even
>> though the request of interest was completed long ago.
> 
> I need to look at this again, because with this change, I previously
> got various use-after-free. I haven't seen any issues with it, but
> I agree, from a quick look that I'm not quite sure how it's currently
> not causing hangs. Yet I haven't seen any, with targeted -EAGAIN
> testing.

Ah I think I know what it is - if we run into:

if (req->result == -EAGAIN)
	return -EAGAIN

in io_issue_sqe() and race with it, we'll reissue twice potentially.
So the above isn't quite enough, we'll need something a bit broader.

-- 
Jens Axboe

Re: [PATCH 1/4] io_uring: fix hanging iopoll in case of -EAGAIN

[Pavel Begunkov]

On 23/06/2020 05:18, Jens Axboe wrote:
> On 6/22/20 8:07 PM, Jens Axboe wrote:
>> On 6/22/20 4:16 PM, Pavel Begunkov wrote:
>>> io_do_iopoll() won't do anything with a request unless
>>> req->iopoll_completed is set. So io_complete_rw_iopoll() has to set
>>> it, otherwise io_do_iopoll() will poll a file again and again even
>>> though the request of interest was completed long ago.
>>
>> I need to look at this again, because with this change, I previously
>> got various use-after-free. I haven't seen any issues with it, but
>> I agree, from a quick look that I'm not quite sure how it's currently
>> not causing hangs. Yet I haven't seen any, with targeted -EAGAIN
>> testing.

Can io_complete_rw_iopoll() get -EAGAIN after being successfully enqueued
(i.e. EIOCBQUEUED)? It's reliably fails for me, because my hacked nullblk
_can_ (i.e. probabilistically returns BLK_STS_AGAIN from ->iopoll()).

> 
> Ah I think I know what it is - if we run into:
> 
> if (req->result == -EAGAIN)
> 	return -EAGAIN
> 
> in io_issue_sqe() and race with it, we'll reissue twice potentially.
> So the above isn't quite enough, we'll need something a bit broader.

I see, I'll deal with it.


-- 
Pavel Begunkov

Re: [PATCH 1/4] io_uring: fix hanging iopoll in case of -EAGAIN

[Jens Axboe]

On 6/23/20 5:57 AM, Pavel Begunkov wrote:
> On 23/06/2020 05:18, Jens Axboe wrote:
>> On 6/22/20 8:07 PM, Jens Axboe wrote:
>>> On 6/22/20 4:16 PM, Pavel Begunkov wrote:
>>>> io_do_iopoll() won't do anything with a request unless
>>>> req->iopoll_completed is set. So io_complete_rw_iopoll() has to set
>>>> it, otherwise io_do_iopoll() will poll a file again and again even
>>>> though the request of interest was completed long ago.
>>>
>>> I need to look at this again, because with this change, I previously
>>> got various use-after-free. I haven't seen any issues with it, but
>>> I agree, from a quick look that I'm not quite sure how it's currently
>>> not causing hangs. Yet I haven't seen any, with targeted -EAGAIN
>>> testing.
> 
> Can io_complete_rw_iopoll() get -EAGAIN after being successfully enqueued
> (i.e. EIOCBQUEUED)? It's reliably fails for me, because my hacked nullblk
> _can_ (i.e. probabilistically returns BLK_STS_AGAIN from ->iopoll()).

Yes it can. The primary example would be a polled bio that gets split, into
let's say 4 bio's. First one queues fine, but one of the subsequent ones
run into request allocation failures and it gets marked as -EAGAIN.

>> Ah I think I know what it is - if we run into:
>>
>> if (req->result == -EAGAIN)
>> 	return -EAGAIN
>>
>> in io_issue_sqe() and race with it, we'll reissue twice potentially.
>> So the above isn't quite enough, we'll need something a bit broader.
> 
> I see, I'll deal with it.

Thanks!

-- 
Jens Axboe

Re: [PATCH 1/4] io_uring: fix hanging iopoll in case of -EAGAIN

[Pavel Begunkov]

On 23/06/2020 22:01, Jens Axboe wrote:
> On 6/23/20 5:57 AM, Pavel Begunkov wrote:
>> On 23/06/2020 05:18, Jens Axboe wrote:
>>> On 6/22/20 8:07 PM, Jens Axboe wrote:
>>>> On 6/22/20 4:16 PM, Pavel Begunkov wrote:
>>>>> io_do_iopoll() won't do anything with a request unless
>>>>> req->iopoll_completed is set. So io_complete_rw_iopoll() has to set
>>>>> it, otherwise io_do_iopoll() will poll a file again and again even
>>>>> though the request of interest was completed long ago.
>>>>
>>>> I need to look at this again, because with this change, I previously
>>>> got various use-after-free. I haven't seen any issues with it, but
>>>> I agree, from a quick look that I'm not quite sure how it's currently
>>>> not causing hangs. Yet I haven't seen any, with targeted -EAGAIN
>>>> testing.
>>
>> Can io_complete_rw_iopoll() get -EAGAIN after being successfully enqueued
>> (i.e. EIOCBQUEUED)? It's reliably fails for me, because my hacked nullblk
>> _can_ (i.e. probabilistically returns BLK_STS_AGAIN from ->iopoll()).
> 
> Yes it can. The primary example would be a polled bio that gets split, into
> let's say 4 bio's. First one queues fine, but one of the subsequent ones
> run into request allocation failures and it gets marked as -EAGAIN.

Right, thanks for the explanation. And that's the case where io_uring fails.
Now I tested all kinds of -EAGAIN to be sure.

-- 
Pavel Begunkov

[PATCH 2/4] io_uring: handle EAGAIN iopoll

[Pavel Begunkov]

req->iopoll() is not necessarily called by a task that submitted a
request. Because of that, it's dangerous to grab_env() and punt async
on -EGAIN, potentially grabbinf another task's mm and corrupting its
memory.

Do resubmit from the submitter task context.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
---
 fs/io_uring.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/fs/io_uring.c b/fs/io_uring.c
index bb0dfc450db5..595d2bbb31b1 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -884,6 +884,8 @@ enum io_mem_account {
 	ACCT_PINNED,
 };
 
+static void io_complete_rw_common(struct kiocb *kiocb, long res);
+static bool io_rw_reissue(struct io_kiocb *req, long res);
 static void io_wq_submit_work(struct io_wq_work **workptr);
 static void io_cqring_fill_event(struct io_kiocb *req, long res);
 static void io_put_req(struct io_kiocb *req);
@@ -1756,8 +1758,11 @@ static void io_iopoll_queue(struct list_head *again)
 	do {
 		req = list_first_entry(again, struct io_kiocb, list);
 		list_del(&req->list);
-		refcount_inc(&req->refs);
-		io_queue_async_work(req);
+
+		if (!io_rw_reissue(req, -EAGAIN)) {
+			io_complete_rw_common(&req->rw.kiocb, -EAGAIN);
+			io_put_req(req);
+		}
 	} while (!list_empty(again));
 }
 
@@ -1930,6 +1935,8 @@ static int io_iopoll_check(struct io_ring_ctx *ctx, unsigned *nr_events,
 		 */
 		if (!(++iters & 7)) {
 			mutex_unlock(&ctx->uring_lock);
+			if (current->task_works)
+				task_work_run();
 			mutex_lock(&ctx->uring_lock);
 		}
 
@@ -2288,6 +2295,7 @@ static int io_prep_rw(struct io_kiocb *req, const struct io_uring_sqe *sqe,
 		kiocb->ki_complete = io_complete_rw_iopoll;
 		req->result = 0;
 		req->iopoll_completed = 0;
+		io_get_req_task(req);
 	} else {
 		if (kiocb->ki_flags & IOCB_HIPRI)
 			return -EINVAL;
-- 
2.24.0

Re: [PATCH 2/4] io_uring: handle EAGAIN iopoll

[kernel test robot]

[-- Attachment #1: Type: text/plain, Size: 32302 bytes --]

Hi Pavel,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on next-20200622]
[cannot apply to linus/master v5.8-rc2 v5.8-rc1 v5.7 v5.8-rc3]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use  as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Pavel-Begunkov/iopoll-fixes-cleanups/20200623-062127
base:    27f11fea33608cbd321a97cbecfa2ef97dcc1821
config: x86_64-randconfig-a001-20200629-CONFIG_DEBUG_INFO_REDUCED (attached as .config)
compiler: clang version 11.0.0 (https://github.com/llvm/llvm-project cf1d04484344be52ada8178e41d18fd15a9b880c)
reproduce (this is a W=1 build):
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # install x86_64 cross compiling tool for clang build
        # apt-get install binutils-x86-64-linux-gnu
        # save the attached .config to linux build tree
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=x86_64 

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>

All warnings (new ones prefixed by >>):

   kernel/locking/rwsem.c:295:1: warning: unused function 'rwsem_owner_flags'
   rwsem_owner_flags(struct rw_semaphore unsigned long
   ^
   In file included from kernel/cpu.c:26:
   In file included from include/linux/suspend.h:5:
   In file included from include/linux/swap.h:9:
   include/linux/memcontrol.h:45:30: warning: implicit conversion from enumeration type 'enum memcg_stat_item' to different enumeration type 'enum node_stat_item'
   return vmstat_item_in_bytes(item);
   ~~~~~~~~~~~~~~~~~~~~ ^~~~
   In file included from kernel/exit.c:53:
   In file included from include/linux/tracehook.h:50:
   include/linux/memcontrol.h:45:30: warning: implicit conversion from enumeration type 'enum memcg_stat_item' to different enumeration type 'enum node_stat_item'
   return vmstat_item_in_bytes(item);
   ~~~~~~~~~~~~~~~~~~~~ ^~~~
   kernel/panic.c:85:13: warning: no previous prototype for function 'panic_smp_self_stop'
   void __weak panic_smp_self_stop(void)
   ^
   kernel/panic.c:85:1: note: declare 'static' if the function is not intended to be used outside of this translation unit
   void __weak panic_smp_self_stop(void)
   ^
   static
   kernel/panic.c:95:13: warning: no previous prototype for function 'nmi_panic_self_stop'
   void __weak nmi_panic_self_stop(struct pt_regs
   ^
   kernel/panic.c:95:1: note: declare 'static' if the function is not intended to be used outside of this translation unit
   void __weak nmi_panic_self_stop(struct pt_regs
   ^
   static
   kernel/panic.c:106:13: warning: no previous prototype for function 'crash_smp_send_stop'
   void __weak crash_smp_send_stop(void)
   ^
   kernel/panic.c:106:1: note: declare 'static' if the function is not intended to be used outside of this translation unit
   void __weak crash_smp_send_stop(void)
   ^
   static
   3 warnings generated.
   In file included from kernel/fork.c:53:
   In file included from include/linux/swap.h:9:
   include/linux/memcontrol.h:45:30: warning: implicit conversion from enumeration type 'enum memcg_stat_item' to different enumeration type 'enum node_stat_item'
   return vmstat_item_in_bytes(item);
   ~~~~~~~~~~~~~~~~~~~~ ^~~~
   In file included from kernel/power/user.c:10:
   In file included from include/linux/suspend.h:5:
   In file included from include/linux/swap.h:9:
   include/linux/memcontrol.h:45:30: warning: implicit conversion from enumeration type 'enum memcg_stat_item' to different enumeration type 'enum node_stat_item'
   return vmstat_item_in_bytes(item);
   ~~~~~~~~~~~~~~~~~~~~ ^~~~
   kernel/cpu.c:101:20: warning: unused function 'cpuhp_lock_acquire'
   static inline void cpuhp_lock_acquire(bool bringup) { }
   ^
   kernel/cpu.c:102:20: warning: unused function 'cpuhp_lock_release'
   static inline void cpuhp_lock_release(bool bringup) { }
   ^
   1 warning generated.
   1 warning generated.
   In file included from kernel/power/wakelock.c:23:
   In file included from kernel/power/power.h:2:
   In file included from include/linux/suspend.h:5:
   In file included from include/linux/swap.h:9:
   include/linux/memcontrol.h:45:30: warning: implicit conversion from enumeration type 'enum memcg_stat_item' to different enumeration type 'enum node_stat_item'
   return vmstat_item_in_bytes(item);
   ~~~~~~~~~~~~~~~~~~~~ ^~~~
>> kernel/power/wakelock.c:74:20: warning: unused function 'decrement_wakelocks_number'
   static inline void decrement_wakelocks_number(void)
   ^
   3 warnings generated.
   2 warnings generated.
   kernel/cpu.c:56: warning: cannot understand function prototype: 'struct cpuhp_cpu_state '
   kernel/cpu.c:113: warning: cannot understand function prototype: 'struct cpuhp_step '
   kernel/cpu.c:1884: warning: Function parameter or member 'name' not described in '__cpuhp_setup_state_cpuslocked'
   In file included from kernel/sched/core.c:9:
   In file included from kernel/sched/sched.h:63:
   In file included from include/linux/suspend.h:5:
   In file included from include/linux/swap.h:9:
   include/linux/memcontrol.h:45:30: warning: implicit conversion from enumeration type 'enum memcg_stat_item' to different enumeration type 'enum node_stat_item'
   return vmstat_item_in_bytes(item);
   ~~~~~~~~~~~~~~~~~~~~ ^~~~
>> kernel/trace/ftrace.c:6859:20: warning: unused function 'ftrace_startup_enable'
   static inline void ftrace_startup_enable(int command) { }
   ^
   1 warning generated.
   kernel/sched/core.c:4270:35: warning: no previous prototype for function 'schedule_user'
   asmlinkage __visible void __sched schedule_user(void)
   ^
   kernel/sched/core.c:4270:22: note: declare 'static' if the function is not intended to be used outside of this translation unit
   asmlinkage __visible void __sched schedule_user(void)
   ^
   static
   In file included from kernel/power/process.c:14:
   In file included from include/linux/suspend.h:5:
   In file included from include/linux/swap.h:9:
   include/linux/memcontrol.h:45:30: warning: implicit conversion from enumeration type 'enum memcg_stat_item' to different enumeration type 'enum node_stat_item'
   return vmstat_item_in_bytes(item);
   ~~~~~~~~~~~~~~~~~~~~ ^~~~
   1 warning generated.
   1 warning generated.
   In file included from kernel/power/snapshot.c:16:
   In file included from include/linux/suspend.h:5:
   In file included from include/linux/swap.h:9:
   include/linux/memcontrol.h:45:30: warning: implicit conversion from enumeration type 'enum memcg_stat_item' to different enumeration type 'enum node_stat_item'
   return vmstat_item_in_bytes(item);
   ~~~~~~~~~~~~~~~~~~~~ ^~~~
   kernel/fork.c:1153: warning: Function parameter or member 'mm' not described in 'set_mm_exe_file'
   kernel/fork.c:1153: warning: Function parameter or member 'new_exe_file' not described in 'set_mm_exe_file'
   kernel/fork.c:1177: warning: Function parameter or member 'mm' not described in 'get_mm_exe_file'
   kernel/fork.c:1197: warning: Function parameter or member 'task' not described in 'get_task_exe_file'
   kernel/fork.c:1222: warning: Function parameter or member 'task' not described in 'get_task_mm'
   kernel/sched/core.c:227:1: warning: unused function 'rq_csd_init'
   rq_csd_init(struct rq call_single_data_t smp_call_func_t func)
   ^
   kernel/sched/core.c:3822:20: warning: unused function 'sched_tick_start'
   static inline void sched_tick_start(int cpu) { }
   ^
   kernel/sched/core.c:3823:20: warning: unused function 'sched_tick_stop'
   static inline void sched_tick_stop(int cpu) { }
   ^
   kernel/trace/ftrace.c:5350: warning: Function parameter or member 'ops' not described in 'ftrace_set_filter_ip'
   kernel/trace/ftrace.c:5350: warning: Function parameter or member 'ip' not described in 'ftrace_set_filter_ip'
   kernel/trace/ftrace.c:5350: warning: Function parameter or member 'remove' not described in 'ftrace_set_filter_ip'
   kernel/trace/ftrace.c:5350: warning: Function parameter or member 'reset' not described in 'ftrace_set_filter_ip'
   kernel/trace/ftrace.c:5364: warning: Function parameter or member 'ops' not described in 'ftrace_ops_set_global_filter'
   kernel/trace/ftrace.c:5392: warning: Function parameter or member 'ops' not described in 'ftrace_set_filter'
   kernel/trace/ftrace.c:5392: warning: Function parameter or member 'buf' not described in 'ftrace_set_filter'
   kernel/trace/ftrace.c:5392: warning: Function parameter or member 'len' not described in 'ftrace_set_filter'
   kernel/trace/ftrace.c:5392: warning: Function parameter or member 'reset' not described in 'ftrace_set_filter'
   kernel/trace/ftrace.c:5411: warning: Function parameter or member 'ops' not described in 'ftrace_set_notrace'
   kernel/trace/ftrace.c:5411: warning: Function parameter or member 'buf' not described in 'ftrace_set_notrace'
   kernel/trace/ftrace.c:5411: warning: Function parameter or member 'len' not described in 'ftrace_set_notrace'
   kernel/trace/ftrace.c:5411: warning: Function parameter or member 'reset' not described in 'ftrace_set_notrace'
   kernel/trace/ftrace.c:5426: warning: Function parameter or member 'buf' not described in 'ftrace_set_global_filter'
   kernel/trace/ftrace.c:5426: warning: Function parameter or member 'len' not described in 'ftrace_set_global_filter'
   kernel/trace/ftrace.c:5426: warning: Function parameter or member 'reset' not described in 'ftrace_set_global_filter'
   kernel/trace/ftrace.c:5442: warning: Function parameter or member 'buf' not described in 'ftrace_set_global_notrace'
   kernel/trace/ftrace.c:5442: warning: Function parameter or member 'len' not described in 'ftrace_set_global_notrace'
   kernel/trace/ftrace.c:5442: warning: Function parameter or member 'reset' not described in 'ftrace_set_global_notrace'
   kernel/trace/ftrace.c:7471: warning: Function parameter or member 'ops' not described in 'register_ftrace_function'
   kernel/trace/ftrace.c:7493: warning: Function parameter or member 'ops' not described in 'unregister_ftrace_function'
   In file included from kernel/power/main.c:16:
   In file included from include/linux/suspend.h:5:
   In file included from include/linux/swap.h:9:
   include/linux/memcontrol.h:45:30: warning: implicit conversion from enumeration type 'enum memcg_stat_item' to different enumeration type 'enum node_stat_item'
   return vmstat_item_in_bytes(item);
   ~~~~~~~~~~~~~~~~~~~~ ^~~~
   In file included from kernel/power/hibernate.c:15:
   In file included from include/linux/suspend.h:5:
   In file included from include/linux/swap.h:9:
   include/linux/memcontrol.h:45:30: warning: implicit conversion from enumeration type 'enum memcg_stat_item' to different enumeration type 'enum node_stat_item'
   return vmstat_item_in_bytes(item);
   ~~~~~~~~~~~~~~~~~~~~ ^~~~
   In file included from kernel/power/swap.c:23:
   In file included from include/linux/swap.h:9:
   include/linux/memcontrol.h:45:30: warning: implicit conversion from enumeration type 'enum memcg_stat_item' to different enumeration type 'enum node_stat_item'
   return vmstat_item_in_bytes(item);
   ~~~~~~~~~~~~~~~~~~~~ ^~~~
   In file included from kernel/sched/loadavg.c:9:
   In file included from kernel/sched/sched.h:63:
   In file included from include/linux/suspend.h:5:
   In file included from include/linux/swap.h:9:
   include/linux/memcontrol.h:45:30: warning: implicit conversion from enumeration type 'enum memcg_stat_item' to different enumeration type 'enum node_stat_item'
   return vmstat_item_in_bytes(item);
   ~~~~~~~~~~~~~~~~~~~~ ^~~~
   kernel/power/snapshot.c:1266:21: warning: unused function 'saveable_highmem_page'
   static inline void zone unsigned long p)
   ^
   1 warning generated.
   In file included from kernel/sysctl.c:25:
   In file included from include/linux/swap.h:9:
   include/linux/memcontrol.h:45:30: warning: implicit conversion from enumeration type 'enum memcg_stat_item' to different enumeration type 'enum node_stat_item'
   return vmstat_item_in_bytes(item);
   ~~~~~~~~~~~~~~~~~~~~ ^~~~
   1 warning generated.
   In file included from kernel/cgroup/cgroup.c:60:
   In file included from include/net/sock.h:53:
   include/linux/memcontrol.h:45:30: warning: implicit conversion from enumeration type 'enum memcg_stat_item' to different enumeration type 'enum node_stat_item'
   return vmstat_item_in_bytes(item);
   ~~~~~~~~~~~~~~~~~~~~ ^~~~
   kernel/power/hibernate.c:272:12: warning: no previous prototype for function 'arch_resume_nosmt'
   __weak int arch_resume_nosmt(void)
--
   In file included from kernel/power/user.c:10:
   In file included from include/linux/suspend.h:5:
   In file included from include/linux/swap.h:9:
   include/linux/memcontrol.h:45:30: warning: implicit conversion from enumeration type 'enum memcg_stat_item' to different enumeration type 'enum node_stat_item' [-Wenum-conversion]
           return vmstat_item_in_bytes(item);
                  ~~~~~~~~~~~~~~~~~~~~ ^~~~
   1 warning generated.
   In file included from kernel/power/wakelock.c:23:
   In file included from kernel/power/power.h:2:
   In file included from include/linux/suspend.h:5:
   In file included from include/linux/swap.h:9:
   include/linux/memcontrol.h:45:30: warning: implicit conversion from enumeration type 'enum memcg_stat_item' to different enumeration type 'enum node_stat_item' [-Wenum-conversion]
           return vmstat_item_in_bytes(item);
                  ~~~~~~~~~~~~~~~~~~~~ ^~~~
>> kernel/power/wakelock.c:74:20: warning: unused function 'decrement_wakelocks_number' [-Wunused-function]
   static inline void decrement_wakelocks_number(void)
                      ^
   2 warnings generated.
   In file included from kernel/power/process.c:14:
   In file included from include/linux/suspend.h:5:
   In file included from include/linux/swap.h:9:
   include/linux/memcontrol.h:45:30: warning: implicit conversion from enumeration type 'enum memcg_stat_item' to different enumeration type 'enum node_stat_item' [-Wenum-conversion]
           return vmstat_item_in_bytes(item);
                  ~~~~~~~~~~~~~~~~~~~~ ^~~~
   In file included from kernel/power/snapshot.c:16:
   In file included from include/linux/suspend.h:5:
   In file included from include/linux/swap.h:9:
   include/linux/memcontrol.h:45:30: warning: implicit conversion from enumeration type 'enum memcg_stat_item' to different enumeration type 'enum node_stat_item' [-Wenum-conversion]
           return vmstat_item_in_bytes(item);
                  ~~~~~~~~~~~~~~~~~~~~ ^~~~
   In file included from kernel/power/main.c:16:
   In file included from include/linux/suspend.h:5:
   In file included from include/linux/swap.h:9:
   include/linux/memcontrol.h:45:30: warning: implicit conversion from enumeration type 'enum memcg_stat_item' to different enumeration type 'enum node_stat_item' [-Wenum-conversion]
           return vmstat_item_in_bytes(item);
                  ~~~~~~~~~~~~~~~~~~~~ ^~~~
   In file included from kernel/power/hibernate.c:15:
   In file included from include/linux/suspend.h:5:
   In file included from include/linux/swap.h:9:
   include/linux/memcontrol.h:45:30: warning: implicit conversion from enumeration type 'enum memcg_stat_item' to different enumeration type 'enum node_stat_item' [-Wenum-conversion]
           return vmstat_item_in_bytes(item);
                  ~~~~~~~~~~~~~~~~~~~~ ^~~~
   In file included from kernel/power/swap.c:23:
   In file included from include/linux/swap.h:9:
   include/linux/memcontrol.h:45:30: warning: implicit conversion from enumeration type 'enum memcg_stat_item' to different enumeration type 'enum node_stat_item' [-Wenum-conversion]
           return vmstat_item_in_bytes(item);
                  ~~~~~~~~~~~~~~~~~~~~ ^~~~
   kernel/power/snapshot.c:1266:21: warning: unused function 'saveable_highmem_page' [-Wunused-function]
   static inline void *saveable_highmem_page(struct zone *z, unsigned long p)
                       ^
   1 warning generated.
   kernel/power/hibernate.c:272:12: warning: no previous prototype for function 'arch_resume_nosmt' [-Wmissing-prototypes]
   __weak int arch_resume_nosmt(void)
              ^
   kernel/power/hibernate.c:272:8: note: declare 'static' if the function is not intended to be used outside of this translation unit
   __weak int arch_resume_nosmt(void)
          ^
          static 
   kernel/power/main.c:593: warning: Function parameter or member 'kobj' not described in 'state_show'
   kernel/power/main.c:593: warning: Function parameter or member 'attr' not described in 'state_show'
   kernel/power/main.c:593: warning: Function parameter or member 'buf' not described in 'state_show'
   2 warnings generated.
   kernel/power/snapshot.c:404: warning: Function parameter or member 'gfp_mask' not described in 'alloc_rtree_node'
   kernel/power/snapshot.c:404: warning: Function parameter or member 'safe_needed' not described in 'alloc_rtree_node'
   kernel/power/snapshot.c:404: warning: Function parameter or member 'ca' not described in 'alloc_rtree_node'
   kernel/power/snapshot.c:404: warning: Function parameter or member 'list' not described in 'alloc_rtree_node'
   kernel/power/snapshot.c:429: warning: Function parameter or member 'zone' not described in 'add_rtree_block'
   kernel/power/snapshot.c:429: warning: Function parameter or member 'gfp_mask' not described in 'add_rtree_block'
   kernel/power/snapshot.c:429: warning: Function parameter or member 'safe_needed' not described in 'add_rtree_block'
   kernel/power/snapshot.c:429: warning: Function parameter or member 'ca' not described in 'add_rtree_block'
   kernel/power/snapshot.c:502: warning: Function parameter or member 'gfp_mask' not described in 'create_zone_bm_rtree'
   kernel/power/snapshot.c:502: warning: Function parameter or member 'safe_needed' not described in 'create_zone_bm_rtree'
   kernel/power/snapshot.c:502: warning: Function parameter or member 'ca' not described in 'create_zone_bm_rtree'
   kernel/power/snapshot.c:502: warning: Function parameter or member 'start' not described in 'create_zone_bm_rtree'
   kernel/power/snapshot.c:502: warning: Function parameter or member 'end' not described in 'create_zone_bm_rtree'
   kernel/power/snapshot.c:537: warning: Function parameter or member 'zone' not described in 'free_zone_bm_rtree'
   kernel/power/snapshot.c:537: warning: Function parameter or member 'clear_nosave_free' not described in 'free_zone_bm_rtree'
   kernel/power/snapshot.c:644: warning: Function parameter or member 'bm' not described in 'memory_bm_create'
   kernel/power/snapshot.c:644: warning: Function parameter or member 'gfp_mask' not described in 'memory_bm_create'
   kernel/power/snapshot.c:644: warning: Function parameter or member 'safe_needed' not described in 'memory_bm_create'
   kernel/power/snapshot.c:686: warning: Function parameter or member 'clear_nosave_free' not described in 'memory_bm_free'
   kernel/power/snapshot.c:708: warning: Function parameter or member 'bm' not described in 'memory_bm_find_bit'
   kernel/power/snapshot.c:708: warning: Function parameter or member 'pfn' not described in 'memory_bm_find_bit'
   kernel/power/snapshot.c:708: warning: Function parameter or member 'addr' not described in 'memory_bm_find_bit'
   kernel/power/snapshot.c:708: warning: Function parameter or member 'bit_nr' not described in 'memory_bm_find_bit'
   kernel/power/snapshot.c:949: warning: Function parameter or member 'start_pfn' not described in '__register_nosave_region'
   kernel/power/snapshot.c:949: warning: Function parameter or member 'end_pfn' not described in '__register_nosave_region'
   kernel/power/snapshot.c:949: warning: Function parameter or member 'use_kmalloc' not described in '__register_nosave_region'
   kernel/power/snapshot.c:1219: warning: Function parameter or member 'zone' not described in 'saveable_highmem_page'
   kernel/power/snapshot.c:1219: warning: Function parameter or member 'pfn' not described in 'saveable_highmem_page'
   kernel/power/snapshot.c:1283: warning: Function parameter or member 'zone' not described in 'saveable_page'
   kernel/power/snapshot.c:1283: warning: Function parameter or member 'pfn' not described in 'saveable_page'
   kernel/power/snapshot.c:1354: warning: Function parameter or member 'dst' not described in 'safe_copy_page'
   kernel/power/snapshot.c:1354: warning: Function parameter or member 's_page' not described in 'safe_copy_page'
   kernel/power/snapshot.c:1567: warning: Function parameter or member 'x' not described in '__fraction'
   kernel/power/snapshot.c:1567: warning: Function parameter or member 'multiplier' not described in '__fraction'
   kernel/power/snapshot.c:1567: warning: Function parameter or member 'base' not described in '__fraction'
   kernel/power/snapshot.c:1858: warning: Function parameter or member 'nr_highmem' not described in 'count_pages_for_highmem'
   kernel/power/snapshot.c:1876: warning: Function parameter or member 'nr_pages' not described in 'enough_free_mem'
   kernel/power/snapshot.c:1876: warning: Function parameter or member 'nr_highmem' not described in 'enough_free_mem'
   kernel/power/snapshot.c:1899: warning: Function parameter or member 'safe_needed' not described in 'get_highmem_buffer'
   kernel/power/snapshot.c:1912: warning: Function parameter or member 'bm' not described in 'alloc_highmem_pages'
   kernel/power/snapshot.c:1912: warning: Function parameter or member 'nr_highmem' not described in 'alloc_highmem_pages'
   kernel/power/snapshot.c:1947: warning: Function parameter or member 'copy_bm' not described in 'swsusp_alloc'
   kernel/power/snapshot.c:1947: warning: Function parameter or member 'nr_pages' not described in 'swsusp_alloc'
   kernel/power/snapshot.c:1947: warning: Function parameter or member 'nr_highmem' not described in 'swsusp_alloc'
   kernel/power/snapshot.c:2160: warning: Function parameter or member 'bm' not described in 'mark_unsafe_pages'
   kernel/power/snapshot.c:2195: warning: Function parameter or member 'info' not described in 'load_header'
   kernel/power/snapshot.c:2349: warning: Function parameter or member 'page' not described in 'get_highmem_page_buffer'
   kernel/power/snapshot.c:2349: warning: Function parameter or member 'ca' not described in 'get_highmem_page_buffer'
   kernel/power/snapshot.c:2534: warning: Function parameter or member 'bm' not described in 'get_buffer'
   kernel/power/snapshot.c:2534: warning: Function parameter or member 'ca' not described in 'get_buffer'
   kernel/power/snapshot.c:2658: warning: Function parameter or member 'handle' not described in 'snapshot_write_finalize'
   1 warning generated.
   2 warnings generated.
--
>> kernel/trace/ftrace.c:6859:20: warning: unused function 'ftrace_startup_enable' [-Wunused-function]
   static inline void ftrace_startup_enable(int command) { }
                      ^
   1 warning generated.
   kernel/trace/ftrace.c:5350: warning: Function parameter or member 'ops' not described in 'ftrace_set_filter_ip'
   kernel/trace/ftrace.c:5350: warning: Function parameter or member 'ip' not described in 'ftrace_set_filter_ip'
   kernel/trace/ftrace.c:5350: warning: Function parameter or member 'remove' not described in 'ftrace_set_filter_ip'
   kernel/trace/ftrace.c:5350: warning: Function parameter or member 'reset' not described in 'ftrace_set_filter_ip'
   kernel/trace/ftrace.c:5364: warning: Function parameter or member 'ops' not described in 'ftrace_ops_set_global_filter'
   kernel/trace/ftrace.c:5392: warning: Function parameter or member 'ops' not described in 'ftrace_set_filter'
   kernel/trace/ftrace.c:5392: warning: Function parameter or member 'buf' not described in 'ftrace_set_filter'
   kernel/trace/ftrace.c:5392: warning: Function parameter or member 'len' not described in 'ftrace_set_filter'
   kernel/trace/ftrace.c:5392: warning: Function parameter or member 'reset' not described in 'ftrace_set_filter'
   kernel/trace/ftrace.c:5411: warning: Function parameter or member 'ops' not described in 'ftrace_set_notrace'
   kernel/trace/ftrace.c:5411: warning: Function parameter or member 'buf' not described in 'ftrace_set_notrace'
   kernel/trace/ftrace.c:5411: warning: Function parameter or member 'len' not described in 'ftrace_set_notrace'
   kernel/trace/ftrace.c:5411: warning: Function parameter or member 'reset' not described in 'ftrace_set_notrace'
   kernel/trace/ftrace.c:5426: warning: Function parameter or member 'buf' not described in 'ftrace_set_global_filter'
   kernel/trace/ftrace.c:5426: warning: Function parameter or member 'len' not described in 'ftrace_set_global_filter'
   kernel/trace/ftrace.c:5426: warning: Function parameter or member 'reset' not described in 'ftrace_set_global_filter'
   kernel/trace/ftrace.c:5442: warning: Function parameter or member 'buf' not described in 'ftrace_set_global_notrace'
   kernel/trace/ftrace.c:5442: warning: Function parameter or member 'len' not described in 'ftrace_set_global_notrace'
   kernel/trace/ftrace.c:5442: warning: Function parameter or member 'reset' not described in 'ftrace_set_global_notrace'
   kernel/trace/ftrace.c:7471: warning: Function parameter or member 'ops' not described in 'register_ftrace_function'
   kernel/trace/ftrace.c:7493: warning: Function parameter or member 'ops' not described in 'unregister_ftrace_function'
   kernel/trace/ring_buffer.c:1141: warning: Function parameter or member 'cpu_buffer' not described in 'rb_check_list'
   kernel/trace/ring_buffer.c:1141: warning: Function parameter or member 'list' not described in 'rb_check_list'
   kernel/trace/trace.c:313: warning: Function parameter or member 'this_tr' not described in 'trace_array_put'
   kernel/trace/trace.c:392: warning: Function parameter or member 'filtered_no_pids' not described in 'trace_ignore_this_task'
   kernel/trace/trace_seq.c:142: warning: Function parameter or member 'args' not described in 'trace_seq_vprintf'
   kernel/trace/trace_preemptirq.c:88:16: warning: no previous prototype for function 'trace_hardirqs_on_caller' [-Wmissing-prototypes]
   __visible void trace_hardirqs_on_caller(unsigned long caller_addr)
                  ^
   kernel/trace/trace_preemptirq.c:88:11: note: declare 'static' if the function is not intended to be used outside of this translation unit
   __visible void trace_hardirqs_on_caller(unsigned long caller_addr)
             ^
             static 
   kernel/trace/trace_preemptirq.c:103:16: warning: no previous prototype for function 'trace_hardirqs_off_caller' [-Wmissing-prototypes]
   __visible void trace_hardirqs_off_caller(unsigned long caller_addr)
                  ^
   kernel/trace/trace_preemptirq.c:103:11: note: declare 'static' if the function is not intended to be used outside of this translation unit
   __visible void trace_hardirqs_off_caller(unsigned long caller_addr)
             ^
             static 
   2 warnings generated.
   In file included from kernel/trace/fgraph.c:10:
   In file included from include/linux/suspend.h:5:
   In file included from include/linux/swap.h:9:
   include/linux/memcontrol.h:45:30: warning: implicit conversion from enumeration type 'enum memcg_stat_item' to different enumeration type 'enum node_stat_item' [-Wenum-conversion]
           return vmstat_item_in_bytes(item);
                  ~~~~~~~~~~~~~~~~~~~~ ^~~~
   kernel/trace/fgraph.c:232:15: warning: no previous prototype for function 'ftrace_return_to_handler' [-Wmissing-prototypes]
   unsigned long ftrace_return_to_handler(unsigned long frame_pointer)
                 ^
   kernel/trace/fgraph.c:232:1: note: declare 'static' if the function is not intended to be used outside of this translation unit
   unsigned long ftrace_return_to_handler(unsigned long frame_pointer)
   ^
   static 
   kernel/trace/fgraph.c:348:6: warning: no previous prototype for function 'ftrace_graph_sleep_time_control' [-Wmissing-prototypes]
   void ftrace_graph_sleep_time_control(bool enable)
        ^
   kernel/trace/fgraph.c:348:1: note: declare 'static' if the function is not intended to be used outside of this translation unit
   void ftrace_graph_sleep_time_control(bool enable)
   ^
   static 
   3kernel/trace/trace_events_filter.c:1565:1:  warningwarning: unused function 'event_set_no_set_filter_flag' [-Wunused-function]s
    generated.
   event_set_no_set_filter_flag(struct trace_event_file *file)
   ^
   kernel/trace/trace_events_filter.c:1571:1: warning: unused function 'event_clear_no_set_filter_flag' [-Wunused-function]
   event_clear_no_set_filter_flag(struct trace_event_file *file)
   ^
   kernel/trace/trace_events_filter.c:1577:1: warning: unused function 'event_no_set_filter_flag' [-Wunused-function]
   event_no_set_filter_flag(struct trace_event_file *file)
   ^
   kernel/trace/fgraph.c:298: warning: Function parameter or member 'task' not described in 'ftrace_graph_ret_addr'
   kernel/trace/fgraph.c:298: warning: Function parameter or member 'idx' not described in 'ftrace_graph_ret_addr'
   kernel/trace/fgraph.c:298: warning: Function parameter or member 'ret' not described in 'ftrace_graph_ret_addr'
   kernel/trace/fgraph.c:298: warning: Function parameter or member 'retp' not described in 'ftrace_graph_ret_addr'
   kernel/trace/trace_branch.c:205:6: warning: no previous prototype for function 'ftrace_likely_update' [-Wmissing-prototypes]
   void ftrace_likely_update(struct ftrace_likely_data *f, int val,
        ^
   kernel/trace/trace_branch.c:205:1: note: declare 'static' if the function is not intended to be used outside of this translation unit
   void ftrace_likely_update(struct ftrace_likely_data *f, int val,
   ^
   static 
   1 warning generated.
   kernel/trace/trace_events_trigger.c:58: warning: Function parameter or member 'event' not described in 'event_triggers_call'
   kernel/trace/trace_events_trigger.c:917: warning: Function parameter or member 'named_data' not described in 'set_named_trigger_data'
   3 warnings generated.
   kernel/trace/trace_events_filter.c:99: warning: cannot understand function prototype: 'struct prog_entry '
   kernel/trace/trace_events_filter.c:118: warning: Function parameter or member 'invert' not described in 'update_preds'
   kernel/trace/trace_events_filter.c:118: warning: Excess function parameter 'when_to_branch' description in 'update_preds'
   kernel/trace/trace_events_filter.c:1736: warning: Function parameter or member 'tr' not described in 'create_filter'
   kernel/trace/trace_events_filter.c:1736: warning: Function parameter or member 'filter_string' not described in 'create_filter'
   kernel/trace/trace_events_filter.c:1736: warning: Excess function parameter 'filter_str' description in 'create_filter'
   kernel/trace/trace_events_filter.c:1776: warning: Function parameter or member 'dir' not described in 'create_system_filter'
   kernel/trace/trace_events_filter.c:1776: warning: Function parameter or member 'tr' not described in 'create_system_filter'
   kernel/trace/trace_events_filter.c:1776: warning: Excess function parameter 'system' description in 'create_system_filter'
..

vim +/decrement_wakelocks_number +74 kernel/power/wakelock.c

c73893e2ca731b Rafael J. Wysocki 2012-05-05  73  
c73893e2ca731b Rafael J. Wysocki 2012-05-05 @74  static inline void decrement_wakelocks_number(void)
c73893e2ca731b Rafael J. Wysocki 2012-05-05  75  {
c73893e2ca731b Rafael J. Wysocki 2012-05-05  76  	number_of_wakelocks--;
c73893e2ca731b Rafael J. Wysocki 2012-05-05  77  }
c73893e2ca731b Rafael J. Wysocki 2012-05-05  78  #else /* CONFIG_PM_WAKELOCKS_LIMIT = 0 */
c73893e2ca731b Rafael J. Wysocki 2012-05-05  79  static inline bool wakelocks_limit_exceeded(void) { return false; }
c73893e2ca731b Rafael J. Wysocki 2012-05-05  80  static inline void increment_wakelocks_number(void) {}
c73893e2ca731b Rafael J. Wysocki 2012-05-05  81  static inline void decrement_wakelocks_number(void) {}
c73893e2ca731b Rafael J. Wysocki 2012-05-05  82  #endif /* CONFIG_PM_WAKELOCKS_LIMIT */
c73893e2ca731b Rafael J. Wysocki 2012-05-05  83  

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 36861 bytes --]

[PATCH 3/4] io-wq: compact io-wq flags numbers

[Pavel Begunkov]

Renumerate IO_WQ flags, so they take adjacent bits

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
---
 fs/io-wq.h | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/fs/io-wq.h b/fs/io-wq.h
index 071f1a997800..04239dfb12b0 100644
--- a/fs/io-wq.h
+++ b/fs/io-wq.h
@@ -5,10 +5,10 @@ struct io_wq;
 
 enum {
 	IO_WQ_WORK_CANCEL	= 1,
-	IO_WQ_WORK_HASHED	= 4,
-	IO_WQ_WORK_UNBOUND	= 32,
-	IO_WQ_WORK_NO_CANCEL	= 256,
-	IO_WQ_WORK_CONCURRENT	= 512,
+	IO_WQ_WORK_HASHED	= 2,
+	IO_WQ_WORK_UNBOUND	= 4,
+	IO_WQ_WORK_NO_CANCEL	= 8,
+	IO_WQ_WORK_CONCURRENT	= 16,
 
 	IO_WQ_HASH_SHIFT	= 24,	/* upper 8 bits are used for hash key */
 };
-- 
2.24.0

[PATCH 4/4] io-wq: return next work from ->do_work() directly

[Pavel Begunkov]

It's easier to return next work from ->do_work() than
having an in-out argument. Looks nicer and easier to compile.
Also, merge io_wq_assign_next() into its only user.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
---
 fs/io-wq.c    |  8 +++-----
 fs/io-wq.h    |  2 +-
 fs/io_uring.c | 53 ++++++++++++++++++++-------------------------------
 3 files changed, 25 insertions(+), 38 deletions(-)

diff --git a/fs/io-wq.c b/fs/io-wq.c
index 47c5f3aeb460..72f759e1d6eb 100644
--- a/fs/io-wq.c
+++ b/fs/io-wq.c
@@ -523,9 +523,8 @@ static void io_worker_handle_work(struct io_worker *worker)
 				work->flags |= IO_WQ_WORK_CANCEL;
 
 			hash = io_get_work_hash(work);
-			linked = old_work = work;
-			wq->do_work(&linked);
-			linked = (old_work == linked) ? NULL : linked;
+			old_work = work;
+			linked = wq->do_work(work);
 
 			work = next_hashed;
 			if (!work && linked && !io_wq_is_hashed(linked)) {
@@ -781,8 +780,7 @@ static void io_run_cancel(struct io_wq_work *work, struct io_wqe *wqe)
 		struct io_wq_work *old_work = work;
 
 		work->flags |= IO_WQ_WORK_CANCEL;
-		wq->do_work(&work);
-		work = (work == old_work) ? NULL : work;
+		work = wq->do_work(work);
 		wq->free_work(old_work);
 	} while (work);
 }
diff --git a/fs/io-wq.h b/fs/io-wq.h
index 04239dfb12b0..114f12ec2d65 100644
--- a/fs/io-wq.h
+++ b/fs/io-wq.h
@@ -101,7 +101,7 @@ static inline struct io_wq_work *wq_next_work(struct io_wq_work *work)
 }
 
 typedef void (free_work_fn)(struct io_wq_work *);
-typedef void (io_wq_work_fn)(struct io_wq_work **);
+typedef struct io_wq_work *(io_wq_work_fn)(struct io_wq_work *);
 
 struct io_wq_data {
 	struct user_struct *user;
diff --git a/fs/io_uring.c b/fs/io_uring.c
index 595d2bbb31b1..5282711eeeaf 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -886,7 +886,6 @@ enum io_mem_account {
 
 static void io_complete_rw_common(struct kiocb *kiocb, long res);
 static bool io_rw_reissue(struct io_kiocb *req, long res);
-static void io_wq_submit_work(struct io_wq_work **workptr);
 static void io_cqring_fill_event(struct io_kiocb *req, long res);
 static void io_put_req(struct io_kiocb *req);
 static void __io_double_put_req(struct io_kiocb *req);
@@ -1624,20 +1623,6 @@ static void io_free_req(struct io_kiocb *req)
 		io_queue_async_work(nxt);
 }
 
-static void io_wq_assign_next(struct io_wq_work **workptr, struct io_kiocb *nxt)
-{
-	struct io_kiocb *link;
-	const struct io_op_def *def = &io_op_defs[nxt->opcode];
-
-	if ((nxt->flags & REQ_F_ISREG) && def->hash_reg_file)
-		io_wq_hash_work(&nxt->work, file_inode(nxt->file));
-
-	*workptr = &nxt->work;
-	link = io_prep_linked_timeout(nxt);
-	if (link)
-		nxt->flags |= REQ_F_QUEUE_TIMEOUT;
-}
-
 /*
  * Drop reference to request, return next in chain (if there is one) if this
  * was the last reference to this request.
@@ -1657,24 +1642,29 @@ static void io_put_req(struct io_kiocb *req)
 		io_free_req(req);
 }
 
-static void io_steal_work(struct io_kiocb *req,
-			  struct io_wq_work **workptr)
+static struct io_wq_work *io_steal_work(struct io_kiocb *req)
 {
+	struct io_kiocb *link, *nxt = NULL;
+
 	/*
-	 * It's in an io-wq worker, so there always should be at least
-	 * one reference, which will be dropped in io_put_work() just
-	 * after the current handler returns.
-	 *
-	 * It also means, that if the counter dropped to 1, then there is
-	 * no asynchronous users left, so it's safe to steal the next work.
+	 * A ref is owned by io-wq in which context we're. So, if that's the
+	 * last one, it's safe to steal next work. False negatives are Ok,
+	 * it just will be re-punted async in io_put_work()
 	 */
-	if (refcount_read(&req->refs) == 1) {
-		struct io_kiocb *nxt = NULL;
+	if (refcount_read(&req->refs) != 1)
+		return NULL;
 
-		io_req_find_next(req, &nxt);
-		if (nxt)
-			io_wq_assign_next(workptr, nxt);
-	}
+	io_req_find_next(req, &nxt);
+	if (!nxt)
+		return NULL;
+
+	if ((nxt->flags & REQ_F_ISREG) && io_op_defs[nxt->opcode].hash_reg_file)
+		io_wq_hash_work(&nxt->work, file_inode(nxt->file));
+
+	link = io_prep_linked_timeout(nxt);
+	if (link)
+		nxt->flags |= REQ_F_QUEUE_TIMEOUT;
+	return &nxt->work;
 }
 
 /*
@@ -5626,9 +5616,8 @@ static void io_arm_async_linked_timeout(struct io_kiocb *req)
 	io_queue_linked_timeout(link);
 }
 
-static void io_wq_submit_work(struct io_wq_work **workptr)
+static struct io_wq_work *io_wq_submit_work(struct io_wq_work *work)
 {
-	struct io_wq_work *work = *workptr;
 	struct io_kiocb *req = container_of(work, struct io_kiocb, work);
 	int ret = 0;
 
@@ -5660,7 +5649,7 @@ static void io_wq_submit_work(struct io_wq_work **workptr)
 		io_put_req(req);
 	}
 
-	io_steal_work(req, workptr);
+	return io_steal_work(req);
 }
 
 static inline struct file *io_file_from_index(struct io_ring_ctx *ctx,
-- 
2.24.0