public inbox for [email protected]
 help / color / mirror / Atom feed
* [PATCH v6 00/11] io_uring getdents
@ 2023-08-27 13:28 Hao Xu
  2023-08-27 13:28 ` [PATCH 01/11] fs: split off vfs_getdents function of getdents64 syscall Hao Xu
                   ` (11 more replies)
  0 siblings, 12 replies; 28+ messages in thread
From: Hao Xu @ 2023-08-27 13:28 UTC (permalink / raw)
  To: io-uring, Jens Axboe
  Cc: Dominique Martinet, Pavel Begunkov, Christian Brauner,
	Alexander Viro, Stefan Roesch, Clay Harris, Dave Chinner,
	Darrick J . Wong, linux-fsdevel, linux-xfs, linux-ext4,
	linux-cachefs, ecryptfs, linux-nfs, linux-unionfs, bpf, netdev,
	linux-s390, linux-kernel, linux-block, linux-btrfs, codalist,
	linux-f2fs-devel, cluster-devel, linux-mm, linux-nilfs, devel,
	linux-cifs, samba-technical, linux-mtd, Wanpeng Li

From: Hao Xu <[email protected]>

This series introduce getdents64 to io_uring, the code logic is similar
with the snychronized version's. It first try nowait issue, and offload
it to io-wq threads if the first try fails.

Patch1 and Patch2 are some preparation
Patch3 supports nowait for xfs getdents code
Patch4-11 are vfs change, include adding helpers and trylock for locks

Tests I've done:
A liburing test case for functional test:
https://github.com/HowHsu/liburing/commit/39dc9a8e19c06a8cebf8c2301b85320eb45c061e?diff=unified

Tested it with a liburing performance test:
https://github.com/HowHsu/liburing/blob/getdents/test/getdents2.c

The test is controlled by the below script[2] which runs getdents2.t 100
times and calulate the avg.
The result show that io_uring version is about 2.6% faster:

note:
[1] the number of getdents call/request in io_uring and normal sync version
are made sure to be same beforehand.

[2] run_getdents.py

```python3

import subprocess

N = 100
sum = 0.0
args = ["/data/home/howeyxu/tmpdir", "sync"]

for i in range(N):
    output = subprocess.check_output(["./liburing/test/getdents2.t"] + args)
    sum += float(output)

average = sum / N
print("Average of sync:", average)

sum = 0.0
args = ["/data/home/howeyxu/tmpdir", "iouring"]

for i in range(N):
    output = subprocess.check_output(["./liburing/test/getdents2.t"] + args)
    sum += float(output)

average = sum / N
print("Average of iouring:", average)

```

v5->v6:
 - remove xfs journal stuff since there are fundamental issues in the
   design.

v4->v5:
 - move atime update to the beginning of getdents operation
 - trylock for i_rwsem
 - nowait semantics for involved xfs journal stuff

v3->v4:
 - add Dave's xfs nowait code and fix a deadlock problem, with some code
   style tweak.
 - disable fixed file to avoid a race problem for now
 - add a test program.

v2->v3:
 - removed the kernfs patches
 - add f_pos_lock logic
 - remove the "reduce last EOF getdents try" optimization since
   Dominique reports that doesn't make difference
 - remove the rewind logic, I think the right way is to introduce lseek
   to io_uring not to patch this logic to getdents.
 - add Singed-off-by of Stefan Roesch for patch 1 since checkpatch
   complained that Co-developed-by someone should be accompanied with
   Signed-off-by same person, I can remove them if Stefan thinks that's
   not proper.


Dominique Martinet (1):
  fs: split off vfs_getdents function of getdents64 syscall

Hao Xu (10):
  xfs: add NOWAIT semantics for readdir
  vfs: add nowait flag for struct dir_context
  vfs: add a vfs helper for io_uring file pos lock
  vfs: add file_pos_unlock() for io_uring usage
  vfs: add a nowait parameter for touch_atime()
  vfs: add nowait parameter for file_accessed()
  vfs: move file_accessed() to the beginning of iterate_dir()
  vfs: error out -EAGAIN if atime needs to be updated
  vfs: trylock inode->i_rwsem in iterate_dir() to support nowait
  io_uring: add support for getdents

 arch/s390/hypfs/inode.c        |  2 +-
 block/fops.c                   |  2 +-
 fs/btrfs/file.c                |  2 +-
 fs/btrfs/inode.c               |  2 +-
 fs/cachefiles/namei.c          |  2 +-
 fs/coda/dir.c                  |  4 +--
 fs/ecryptfs/file.c             |  4 +--
 fs/ext2/file.c                 |  4 +--
 fs/ext4/file.c                 |  6 ++--
 fs/f2fs/file.c                 |  4 +--
 fs/file.c                      | 13 ++++++++
 fs/fuse/dax.c                  |  2 +-
 fs/fuse/file.c                 |  4 +--
 fs/gfs2/file.c                 |  2 +-
 fs/hugetlbfs/inode.c           |  2 +-
 fs/inode.c                     | 10 ++++--
 fs/internal.h                  |  8 +++++
 fs/namei.c                     |  4 +--
 fs/nfsd/vfs.c                  |  2 +-
 fs/nilfs2/file.c               |  2 +-
 fs/orangefs/file.c             |  2 +-
 fs/orangefs/inode.c            |  2 +-
 fs/overlayfs/file.c            |  2 +-
 fs/overlayfs/inode.c           |  2 +-
 fs/pipe.c                      |  2 +-
 fs/ramfs/file-nommu.c          |  2 +-
 fs/readdir.c                   | 61 ++++++++++++++++++++++++++--------
 fs/smb/client/cifsfs.c         |  2 +-
 fs/splice.c                    |  2 +-
 fs/stat.c                      |  2 +-
 fs/ubifs/file.c                |  2 +-
 fs/udf/file.c                  |  2 +-
 fs/xfs/libxfs/xfs_da_btree.c   | 16 +++++++++
 fs/xfs/libxfs/xfs_da_btree.h   |  1 +
 fs/xfs/libxfs/xfs_dir2_block.c |  7 ++--
 fs/xfs/libxfs/xfs_dir2_priv.h  |  2 +-
 fs/xfs/scrub/dir.c             |  2 +-
 fs/xfs/scrub/readdir.c         |  2 +-
 fs/xfs/xfs_dir2_readdir.c      | 49 +++++++++++++++++++++------
 fs/xfs/xfs_file.c              |  6 ++--
 fs/xfs/xfs_inode.c             | 27 +++++++++++++++
 fs/xfs/xfs_inode.h             | 17 ++++++----
 fs/zonefs/file.c               |  4 +--
 include/linux/file.h           |  7 ++++
 include/linux/fs.h             | 15 +++++++--
 include/uapi/linux/io_uring.h  |  1 +
 io_uring/fs.c                  | 53 +++++++++++++++++++++++++++++
 io_uring/fs.h                  |  3 ++
 io_uring/opdef.c               |  8 +++++
 kernel/bpf/inode.c             |  4 +--
 mm/filemap.c                   |  8 ++---
 mm/shmem.c                     |  6 ++--
 net/unix/af_unix.c             |  4 +--
 53 files changed, 310 insertions(+), 96 deletions(-)

-- 
2.25.1


^ permalink raw reply	[flat|nested] 28+ messages in thread

end of thread, other threads:[~2023-09-10 22:01 UTC | newest]

Thread overview: 28+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-08-27 13:28 [PATCH v6 00/11] io_uring getdents Hao Xu
2023-08-27 13:28 ` [PATCH 01/11] fs: split off vfs_getdents function of getdents64 syscall Hao Xu
2023-08-27 13:28 ` [PATCH 02/11] xfs: add NOWAIT semantics for readdir Hao Xu
2023-08-27 20:44   ` Matthew Wilcox
2023-08-29  7:41     ` Hao Xu
2023-08-29 13:05       ` Matthew Wilcox
2023-09-04  1:02   ` Dave Chinner
2023-08-27 13:28 ` [PATCH 03/11] vfs: add nowait flag for struct dir_context Hao Xu
2023-08-27 13:28 ` [PATCH 04/11] vfs: add a vfs helper for io_uring file pos lock Hao Xu
2023-08-27 20:47   ` Matthew Wilcox
2023-08-27 13:28 ` [PATCH 05/11] vfs: add file_pos_unlock() for io_uring usage Hao Xu
2023-08-27 13:28 ` [PATCH 06/11] vfs: add a nowait parameter for touch_atime() Hao Xu
2023-08-27 13:28 ` [PATCH 07/11] vfs: add nowait parameter for file_accessed() Hao Xu
2023-08-27 21:32   ` Matthew Wilcox
2023-08-29  7:46     ` Hao Xu
2023-08-29 11:53       ` Matthew Wilcox
2023-08-30  6:11         ` Hao Xu
2023-09-03 22:30           ` Dave Chinner
2023-09-08  0:29             ` Pavel Begunkov
2023-09-10 22:01               ` Dave Chinner
2023-09-04  9:51   ` Christian Brauner
2023-08-27 13:28 ` [PATCH 08/11] vfs: move file_accessed() to the beginning of iterate_dir() Hao Xu
2023-08-27 13:28 ` [PATCH 09/11] vfs: error out -EAGAIN if atime needs to be updated Hao Xu
2023-08-27 20:51   ` Matthew Wilcox
2023-08-27 13:28 ` [PATCH 10/11] vfs: trylock inode->i_rwsem in iterate_dir() to support nowait Hao Xu
2023-09-04  9:37   ` Christian Brauner
2023-08-27 13:28 ` [PATCH 11/11] io_uring: add support for getdents Hao Xu
2023-09-04  9:57 ` [PATCH v6 00/11] io_uring getdents Christian Brauner

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox