[RFC PATCH 00/11] fixed worker

[Hao Xu <hao.xu@linux.dev>]

From: Hao Xu <howeyxu@tencent.com>

The initial feature request by users is here:
https://github.com/axboe/liburing/issues/296

Fixed worker provide a way for users to control the io-wq threads. A
fixed worker is worker thread which exists no matter there are works
to do or not. We provide a new register api to register fixed workers,
and a register api to unregister them as well. The parameter of the
register api is the number of fixed workers users want.

For example:

```c
io_uring_register_iowq_fixed_workers(&ring, { .nr_workers = 5 })
do I/O works
io_uring_unregister_iowq_fixed_workers(&ring)

```

After registration, there will be 5 fixed workers. User can setup their
affinity, priority etc. freely, without adding any new register api to
set up attributions. These workers won't be destroyed until users call
unregister api.

Note, registering some fixed workers doesn't mean no creating normal
workers. When there is no free workers, new normal workers can be
created when works come. So a work may be picked up by fixed workers or
normal workers.

If users want to offload works only to fixed workers, they can specify
a flag FIXED_ONLY when registering fixed workers.

```c
io_uring_register_iowq_fixed_workers(&ring, { .nr_workers = 5, .flags |=
FIXED_ONLY })

```

In above case, no normal workers will be created before calling
io_uring_register_iowq_fixed_workers().

Note:
 - When registering fixed workers, those fixed workers are per io-wq.
   So if an io_uring instance is shared by multiple tasks, and you want
   all tasks to use fixed workers, all tasks have to call the regitser
   api.
 - if specifying FIXED_ONLY when registering fixed workers, that is per
   io_uring instance. all works in this instance are handled by fixed
   workers.

Therefore, if an io_uring instance is shared by two tasks, and you want
all requests in this instance to be handled only by fixed workers, you
have to call the register api in these two tasks and specify FIXED_ONLY
at least once when calling register api.


Hao Xu (11):
  io-wq: fix worker counting after worker received exit signal
  io-wq: add a new worker flag to indicate worker exit
  io-wq: add a new type io-wq worker
  io-wq: add fixed worker members in io_wq_acct
  io-wq: add a new parameter for creating a new fixed worker
  io-wq: return io_worker after successful inline worker creation
  io_uring: add new api to register fixed workers
  io_uring: add function to unregister fixed workers
  io-wq: add strutures to allow to wait fixed workers exit
  io-wq: distinguish fixed worker by its name
  io_uring: add IORING_SETUP_FIXED_WORKER_ONLY and its friend

 include/uapi/linux/io_uring.h |  20 +++
 io_uring/io-wq.c              | 275 ++++++++++++++++++++++++++++++----
 io_uring/io-wq.h              |   3 +
 io_uring/io_uring.c           | 132 +++++++++++++++-
 4 files changed, 397 insertions(+), 33 deletions(-)

-- 
2.25.1