public inbox for [email protected]
 help / color / mirror / Atom feed
* [PATCH RFC 00/17] playing around req alloc
@ 2021-02-10  0:03 Pavel Begunkov
  2021-02-10  0:03 ` [PATCH 01/17] io_uring: replace force_nonblock with flags Pavel Begunkov
                   ` (17 more replies)
  0 siblings, 18 replies; 27+ messages in thread
From: Pavel Begunkov @ 2021-02-10  0:03 UTC (permalink / raw)
  To: Jens Axboe, io-uring

Unfolding previous ideas on persistent req caches. 4-7 including
slashed ~20% of overhead for nops benchmark, haven't done benchmarking
personally for this yet, but according to perf should be ~30-40% in
total. That's for IOPOLL + inline completion cases, obviously w/o
async/IRQ completions.

Jens,
1. 11/17 removes deallocations on end of submit_sqes. Looks you
forgot or just didn't do that.

2. lists are slow and not great cache-wise, that why at I want at least
a combined approach from 12/17.

3. Instead of lists in "use persistent request cache" I had in mind a
slightly different way: to grow the req alloc cache to 32-128 (or hint
from the userspace), batch-alloc by 8 as before, and recycle _all_ reqs
right into it. If  overflows, do kfree().
It should give probabilistically high hit rate, amortising out most of
allocations. Pros: it doesn't grow ~infinitely as lists can. Cons: there
are always counter examples. But as I don't have numbers to back it, I
took your implementation. Maybe, we'll reconsider later.

I'll revise tomorrow on a fresh head + do some performance testing,
and is leaving it RFC until then.

Jens Axboe (3):
  io_uring: use persistent request cache
  io_uring: provide FIFO ordering for task_work
  io_uring: enable req cache for task_work items

Pavel Begunkov (14):
  io_uring: replace force_nonblock with flags
  io_uring: make op handlers always take issue flags
  io_uring: don't propagate io_comp_state
  io_uring: don't keep submit_state on stack
  io_uring: remove ctx from comp_state
  io_uring: don't reinit submit state every time
  io_uring: replace list with array for compl batch
  io_uring: submit-completion free batching
  io_uring: remove fallback_req
  io_uring: count ctx refs separately from reqs
  io_uring: persistent req cache
  io_uring: feed reqs back into alloc cache
  io_uring: take comp_state from ctx
  io_uring: defer flushing cached reqs

 fs/io-wq.h               |   9 -
 fs/io_uring.c            | 716 ++++++++++++++++++++++-----------------
 include/linux/io_uring.h |  14 +
 3 files changed, 425 insertions(+), 314 deletions(-)

-- 
2.24.0


^ permalink raw reply	[flat|nested] 27+ messages in thread

end of thread, other threads:[~2021-02-10 14:28 UTC | newest]

Thread overview: 27+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-02-10  0:03 [PATCH RFC 00/17] playing around req alloc Pavel Begunkov
2021-02-10  0:03 ` [PATCH 01/17] io_uring: replace force_nonblock with flags Pavel Begunkov
2021-02-10  0:03 ` [PATCH 02/17] io_uring: make op handlers always take issue flags Pavel Begunkov
2021-02-10  0:03 ` [PATCH 03/17] io_uring: don't propagate io_comp_state Pavel Begunkov
2021-02-10 14:00   ` Pavel Begunkov
2021-02-10 14:27     ` Jens Axboe
2021-02-10  0:03 ` [PATCH 04/17] io_uring: don't keep submit_state on stack Pavel Begunkov
2021-02-10  0:03 ` [PATCH 05/17] io_uring: remove ctx from comp_state Pavel Begunkov
2021-02-10  0:03 ` [PATCH 06/17] io_uring: don't reinit submit state every time Pavel Begunkov
2021-02-10  0:03 ` [PATCH 07/17] io_uring: replace list with array for compl batch Pavel Begunkov
2021-02-10  0:03 ` [PATCH 08/17] io_uring: submit-completion free batching Pavel Begunkov
2021-02-10  0:03 ` [PATCH 09/17] io_uring: remove fallback_req Pavel Begunkov
2021-02-10  0:03 ` [PATCH 10/17] io_uring: count ctx refs separately from reqs Pavel Begunkov
2021-02-10  0:03 ` [PATCH 11/17] io_uring: persistent req cache Pavel Begunkov
2021-02-10  0:03 ` [PATCH 12/17] io_uring: feed reqs back into alloc cache Pavel Begunkov
2021-02-10  0:03 ` [PATCH 13/17] io_uring: use persistent request cache Pavel Begunkov
2021-02-10  2:14   ` Jens Axboe
2021-02-10  0:03 ` [PATCH 14/17] io_uring: provide FIFO ordering for task_work Pavel Begunkov
2021-02-10  0:03 ` [PATCH 15/17] io_uring: enable req cache for task_work items Pavel Begunkov
2021-02-10  0:03 ` [PATCH 16/17] io_uring: take comp_state from ctx Pavel Begunkov
2021-02-10  0:03 ` [PATCH 17/17] io_uring: defer flushing cached reqs Pavel Begunkov
2021-02-10  2:10   ` Jens Axboe
2021-02-10  2:08 ` [PATCH RFC 00/17] playing around req alloc Jens Axboe
2021-02-10  3:14   ` Pavel Begunkov
2021-02-10  3:23     ` Jens Axboe
2021-02-10 11:53       ` Pavel Begunkov
2021-02-10 14:27         ` Jens Axboe

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox