Relationship between io-uring asynchronous idioms and mmap/LRU paging.

[Steve <steve_iouring_list@shic.co.uk>]

I hope my post is appropriate for this list. Relative to other recent 
posts on this list, my interests are high-level.

I want to develop efficient, scalable, low-latency, asynchronous 
services in user-space. I've dabbled with liburing in the context of an 
experimental service involving network request/responses.  For the 
purpose of this post, assume calculating responses, to requests, 
requires looking-up pages in a huge read-only file.  In order to reap 
all the performance benefits of io-uring, I know I should avoid blocking 
calls in my event loop.

If I were to use a multithreaded (c.f. asynchronous) paradigm... my 
strategy, to look-up pages, would have been to mmap 
<https://man7.org/linux/man-pages/man2/mmap.2.html> the huge file and 
rely upon the kernel LRU cache. Cache misses, relative to the memory 
mapped file will result in a page fault and a blocked thread. This could 
be OK, if cache-misses are rare events... but, while cache hits are 
expected to be frequent, I can't assume cache misses will be rare.

Options I have considered:

   1. Introduce a thread-pool, with task-request and task-response 
queues... using tasks to de-couple reading requests from writing 
responses... the strategy would be to avoid the io-uring event loop 
thread interacting with the memory mapped file. Intuitively, this seems 
cumbersome - compared with using a 'more asynchronous' idiom to avoid 
having to depend upon multithreaded concurrency and thread synchronisation.

   2. Implement an explicit application-layer page cache. Pages could be 
retrieved, into explicitly allocated memory, asynchronously... using 
io-uring read requests. I could suspend request/response processing on 
any cache miss... then resume processing when the io-uring completion 
queue informs that each page has been loaded.  A C++20 coroutine, for 
example, could allow this asynchronous suspension and resumption of 
calculation of responses to requests. This approach seems to undermine 
resource-use cooperation between processes. A single page on disk could 
end-up cached separately by each process instance (inefficient) and 
there would be difficulties efficiently managing appropriate sizes for 
application layer caches.

In an ideal world, I would like to fuse the benefits of mmap's 
kernel-managed cache, with the advantages of an io-uring asynchronous 
idiom.  I find myself wishing there were kernel-level APIs to:

   * Determine if a page, at a virtual address, is already cached in 
RAM. [ Perhaps mincore() 
<https://man7.org/linux/man-pages/man2/mincore.2.html> could be adequate? ]
   * Submit an asynchronous io-uring request with comparable (but 
non-blocking) effect to a page-fault for the virtual address whose page 
was not in core.
   * Receive notification, on the io-uring completion queue, that an 
requested page has now been cached.

If such facilities were to exist, I can imagine a process, using 
io-uring asynchronous idioms, that retains the memory management 
advantages associated with mmap... without introducing dependence upon 
threads.  I've not found any documentation to suggest that my imagined 
io-uring features exist.  Am I overlooking something? Are there plans to 
implement asynchronous features involving the kernel page-cache and 
io-uring scheduling?  Would io-uring experts consider option 1 a 
sensible, pragmatic, choice... in a circumstance where kernel-level 
caching of the mapped file seems desirable... or would a different 
approach be more appropriate?

Thanks in advance for any comments.

Steve