Re: Relationship between io-uring asynchronous idioms and mmap/LRU paging.

[Jens Axboe <axboe@kernel.dk>]

On 7/11/25 12:12 PM, Steve wrote:
> I hope my post is appropriate for this list. Relative to other recent
> posts on this list, my interests are high-level.

Certainly is.

> I want to develop efficient, scalable, low-latency, asynchronous
> services in user-space. I've dabbled with liburing in the context of
> an experimental service involving network request/responses.  For the
> purpose of this post, assume calculating responses, to requests,
> requires looking-up pages in a huge read-only file.  In order to reap
> all the performance benefits of io-uring, I know I should avoid
> blocking calls in my event loop.
> 
> If I were to use a multithreaded (c.f. asynchronous) paradigm... my
> strategy, to look-up pages, would have been to mmap
> <https://man7.org/linux/man-pages/man2/mmap.2.html> the huge file and
> rely upon the kernel LRU cache. Cache misses, relative to the memory
> mapped file will result in a page fault and a blocked thread. This
> could be OK, if cache-misses are rare events... but, while cache hits
> are expected to be frequent, I can't assume cache misses will be rare.
> 
> Options I have considered:
> 
>   1. Introduce a thread-pool, with task-request and task-response
>   queues... using tasks to de-couple reading requests from writing
>   responses... the strategy would be to avoid the io-uring event loop
>   thread interacting with the memory mapped file. Intuitively, this
>   seems cumbersome - compared with using a 'more asynchronous' idiom
>   to avoid having to depend upon multithreaded concurrency and thread
>   synchronisation.
> 
>   2. Implement an explicit application-layer page cache. Pages could
>   be retrieved, into explicitly allocated memory, asynchronously...
>   using io-uring read requests. I could suspend request/response
>   processing on any cache miss... then resume processing when the
>   io-uring completion queue informs that each page has been loaded.  A
>   C++20 coroutine, for example, could allow this asynchronous
>   suspension and resumption of calculation of responses to requests.
>   This approach seems to undermine resource-use cooperation between
>   processes. A single page on disk could end-up cached separately by
>   each process instance (inefficient) and there would be difficulties
>   efficiently managing appropriate sizes for application layer caches.
> 
> In an ideal world, I would like to fuse the benefits of mmap's
> kernel-managed cache, with the advantages of an io-uring asynchronous
> idiom.  I find myself wishing there were kernel-level APIs to:
> 
>   * Determine if a page, at a virtual address, is already cached in
>   RAM. [ Perhaps mincore()
>   <https://man7.org/linux/man-pages/man2/mincore.2.html> could be
>   adequate? ]
>   * Submit an asynchronous io-uring request with comparable (but
>   non-blocking) effect to a page-fault for the virtual address whose
>   page was not in core.
>   * Receive notification, on the io-uring completion queue, that an
>   requested page has now been cached.
> 
> If such facilities were to exist, I can imagine a process, using
> io-uring asynchronous idioms, that retains the memory management
> advantages associated with mmap... without introducing dependence upon
> threads.  I've not found any documentation to suggest that my imagined
> io-uring features exist.  Am I overlooking something? Are there plans
> to implement asynchronous features involving the kernel page-cache and
> io-uring scheduling?  Would io-uring experts consider option 1 a
> sensible, pragmatic, choice... in a circumstance where kernel-level
> caching of the mapped file seems desirable... or would a different
> approach be more appropriate?

Just a heads-up then I'm OOO for a bit, and since it looks like nobody
else has replied to this, I'll take a closer look when I'm back next
week.

-- 
Jens Axboe