Re: [RFC 0/9] fixed worker: a new way to handle io works

[Hao Xu <haoxu@linux.alibaba.com>]

在 2021/11/25 下午11:09, Pavel Begunkov 写道:
> On 11/24/21 04:46, Hao Xu wrote:
>> There is big contension in current io-wq implementation. Introduce a new
>> type io-worker called fixed-worker to solve this problem. it is also a
>> new way to handle works. In this new system, works are dispatched to
>> different private queues rather than a long shared queue.
> 
> It's really great to temper the contention here, even though it looks
> we are stepping onto the path of reinventing all the optimisations
> solved long ago in other thread pools. Work stealing is probably
Hmm, hope io_uring can do it better, a powerful iowq! :)
> the next, but guess it's inevitable :)
Probably yes :)
> 
> First four patchhes sound like a good idea, they will probably go
> first. However, IIUC, the hashing is crucial and it's a must have.
> Are you planning to add it? If not, is there an easy way to leave
I'm planning to add it, still need some time to make it robust.
> hashing working even if hashed reqs not going through those new
> per-worker queues? E.g. (if it's not already as this...)
> 
> if (hashed) {
>      // fixed workers don't support hashing, so go through the
>      // old path and place into the shared queue.
>      enqueue_shared_queue();
> } else
>      enqueue_new_path();
>
Good idea.


> And last note, just fyi, it's easier to sell patches if you put
> numbers in the cover letter
Thanks Pavel, that's definitely clearer for people to review.

Cheers,
Hao
> 
> 
>> Hao Xu (9):
>>    io-wq: decouple work_list protection from the big wqe->lock
>>    io-wq: reduce acct->lock crossing functions lock/unlock
>>    io-wq: update check condition for lock
>>    io-wq: use IO_WQ_ACCT_NR rather than hardcoded number
>>    io-wq: move hash wait entry to io_wqe_acct
>>    io-wq: add infra data structure for fix workers
>>    io-wq: implement fixed worker logic
>>    io-wq: batch the handling of fixed worker private works
>>    io-wq: small optimization for __io_worker_busy()
>>
>>   fs/io-wq.c | 415 ++++++++++++++++++++++++++++++++++++++---------------
>>   fs/io-wq.h |   5 +
>>   2 files changed, 308 insertions(+), 112 deletions(-)
>>
>