public inbox for [email protected]
 help / color / mirror / Atom feed
* [PATCH v4 0/2] io_uring: add support for IORING_OP_GETDENTS
@ 2021-03-12 15:48 Lennert Buytenhek
  2021-03-12 15:49 ` [PATCH v4 1/2] readdir: split the core of getdents64(2) out into vfs_getdents() Lennert Buytenhek
  2021-03-12 15:50 ` [PATCH v4 2/2] io_uring: add support for IORING_OP_GETDENTS Lennert Buytenhek
  0 siblings, 2 replies; 6+ messages in thread
From: Lennert Buytenhek @ 2021-03-12 15:48 UTC (permalink / raw)
  To: io-uring

(These patches depend on IORING_OP_MKDIRAT going in first -- see
the changelog entry for v2 below.)

These patches add support for IORING_OP_GETDENTS, which is a new io_uring
opcode that more or less does an lseek(sqe->fd, sqe->off, SEEK_SET)
followed by a getdents64(sqe->fd, (void *)sqe->addr, sqe->len).

A dumb test program which recursively scans through a directory tree
and prints the names of all directories and files it encounters along
the way is available here:

	https://krautbox.wantstofly.org/~buytenh/uringfind-v3.c

Changes since v3 RFC:

- Made locking in io_getdents() unconditional, as the prior
  optimization was racy.  (Pointed out by Pavel Begunkov.)

- Rebase onto for-5.13/io_uring as of 2021/03/12 plus a manually
  applied version of the mkdirat patch.

Changes since v2 RFC:

- Rebase onto io_uring-2021-02-17 plus a manually applied version of
  the mkdirat patch.  The latter is needed because userland (liburing)
  has already merged the opcode for IORING_OP_MKDIRAT (in commit
  "io_uring.h: 5.12 pending kernel sync") while this opcode isn't in
  the kernel yet (as of io_uring-2021-02-17), and this means that this
  can't be merged until IORING_OP_MKDIRAT is merged.

- Adapt to changes made in "io_uring: replace force_nonblock with flags"
  that are in io_uring-2021-02-17.

Changes since v1 RFC:

- Drop the trailing '64' from IORING_OP_GETDENTS64 (suggested by
  Matthew Wilcox).

- Instead of requiring that sqe->off be zero, use this field to pass
  in a directory offset to start reading from.  For the first
  IORING_OP_GETDENTS call on a directory, this can be set to zero,
  and for subsequent calls, it can be set to the ->d_off field of
  the last struct linux_dirent64 returned by the previous call.

Lennert Buytenhek (2):
  readdir: split the core of getdents64(2) out into vfs_getdents()
  io_uring: add support for IORING_OP_GETDENTS

 fs/io_uring.c                 |   66 ++++++++++++++++++++++++++++++++++++++++++
 fs/readdir.c                  |   25 ++++++++++-----
 include/linux/fs.h            |    4 ++
 include/uapi/linux/io_uring.h |    1
 4 files changed, 88 insertions(+), 8 deletions(-)

^ permalink raw reply	[flat|nested] 6+ messages in thread
* RE: [PATCH v4 2/2] io_uring: add support for IORING_OP_GETDENTS
@ 2021-03-16 15:39 Tavian Barnes
  2021-03-16 16:11 ` Lennert Buytenhek
  0 siblings, 1 reply; 6+ messages in thread
From: Tavian Barnes @ 2021-03-16 15:39 UTC (permalink / raw)
  To: buytenh; +Cc: io-uring

 > IORING_OP_GETDENTS behaves much like getdents64(2) and takes the same
 > arguments, but with a small twist: it takes an additional offset
 > argument, and reading from the specified directory starts at the given
 > offset.
 >
 > For the first IORING_OP_GETDENTS call on a directory, the offset
 > parameter can be set to zero, and for subsequent calls, it can be
 > set to the ->d_off field of the last struct linux_dirent64 returned
 > by the previous IORING_OP_GETDENTS call.
 >
 > Internally, if necessary, IORING_OP_GETDENTS will vfs_llseek() to
 > the right directory position before calling vfs_getdents().

For my envisioned use case it would be important to support reading from
the current file position when offset == -1 (IORING_FEAT_RW_CUR_POS).
Among other things, this would let me fully read a directory with fewer
round-trips.  Normally after the first getdents(), another one must be
issued to distinguish between EOF and a short first read.  It would be
nice to do both calls with linked SQEs so I could immediately know that
I've reached the end of the directory.

 > IORING_OP_GETDENTS may or may not update the specified directory's
 > file offset, and the file offset should not be relied upon having
 > any particular value during or after an IORING_OP_GETDENTS call.

Obviously for the above to work, we'd have to get rid of this 
limitation.  Is that possible?

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2021-03-29 20:40 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-03-12 15:48 [PATCH v4 0/2] io_uring: add support for IORING_OP_GETDENTS Lennert Buytenhek
2021-03-12 15:49 ` [PATCH v4 1/2] readdir: split the core of getdents64(2) out into vfs_getdents() Lennert Buytenhek
2021-03-29 20:38   ` Jens Axboe
2021-03-12 15:50 ` [PATCH v4 2/2] io_uring: add support for IORING_OP_GETDENTS Lennert Buytenhek
  -- strict thread matches above, loose matches on Subject: below --
2021-03-16 15:39 Tavian Barnes
2021-03-16 16:11 ` Lennert Buytenhek

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox