* [RFC 0/5] support nonblock submission for splice pipe to pipe
@ 2022-06-07 8:06 Hao Xu
2022-06-07 8:06 ` [PATCH 1/5] io_uring: move sp->len check up for splice and tee Hao Xu
` (7 more replies)
0 siblings, 8 replies; 15+ messages in thread
From: Hao Xu @ 2022-06-07 8:06 UTC (permalink / raw)
To: io-uring
Cc: Jens Axboe, Pavel Begunkov, linux-fsdevel, Alexander Viro,
Andrew Morton, Luis Chamberlain, Kuniyuki Iwashima
From: Hao Xu <[email protected]>
splice from pipe to pipe is a trivial case, and we can support nonblock
try for it easily. splice depends on iowq at all which is slow. Let's
build a fast submission path for it by supporting nonblock.
Wrote a simple test to test time spent of splicing from pipe to pipe:
Did 50 times test for each, ignore the highest and lowest number,
calculate the average number:
before patchset: 119.85 usec
with patchset: 29.5 usec
----------------
I'm not sure if we should use a io_uring specific flag rather than
SPLICE_F_NONBLOCK since from mutex_lock_nest to mutex_trylock changes
the behavior under debug environment I guess. Or maybe there is another
better option than mutex_trylock?
Hao Xu (5):
io_uring: move sp->len check up for splice and tee
pipe: add trylock helpers for pipe lock
splice: support nonblock for splice from pipe to pipe
io_uring: support nonblock try for splicing from pipe to pipe
io_uring: add file_in in io_splice{} to avoid duplicate calculation
fs/pipe.c | 29 +++++++++++++++++++++
fs/splice.c | 21 +++++++++++++---
include/linux/pipe_fs_i.h | 2 ++
io_uring/splice.c | 53 +++++++++++++++++++++++++++++----------
4 files changed, 89 insertions(+), 16 deletions(-)
base-commit: d8271bf021438f468dab3cd84fe5279b5bbcead8
--
2.25.1
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH 1/5] io_uring: move sp->len check up for splice and tee
2022-06-07 8:06 [RFC 0/5] support nonblock submission for splice pipe to pipe Hao Xu
@ 2022-06-07 8:06 ` Hao Xu
2022-06-07 8:06 ` [PATCH 2/5] pipe: add trylock helpers for pipe lock Hao Xu
` (6 subsequent siblings)
7 siblings, 0 replies; 15+ messages in thread
From: Hao Xu @ 2022-06-07 8:06 UTC (permalink / raw)
To: io-uring
Cc: Jens Axboe, Pavel Begunkov, linux-fsdevel, Alexander Viro,
Andrew Morton, Luis Chamberlain, Kuniyuki Iwashima
From: Hao Xu <[email protected]>
The traditional sync splice code return 0 if len is 0 at the beginning
of syscall, similar thing for tee. So move up sp->len zero check so that
it reaches quick ending when len is 0.
Signed-off-by: Hao Xu <[email protected]>
---
io_uring/splice.c | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/io_uring/splice.c b/io_uring/splice.c
index 0e19d6330345..b2cd1044c3ee 100644
--- a/io_uring/splice.c
+++ b/io_uring/splice.c
@@ -53,6 +53,9 @@ int io_tee(struct io_kiocb *req, unsigned int issue_flags)
struct file *in;
long ret = 0;
+ if (unlikely(!sp->len))
+ goto done;
+
if (issue_flags & IO_URING_F_NONBLOCK)
return -EAGAIN;
@@ -65,8 +68,7 @@ int io_tee(struct io_kiocb *req, unsigned int issue_flags)
goto done;
}
- if (sp->len)
- ret = do_tee(in, out, sp->len, flags);
+ ret = do_tee(in, out, sp->len, flags);
if (!(sp->flags & SPLICE_F_FD_IN_FIXED))
io_put_file(in);
@@ -95,6 +97,9 @@ int io_splice(struct io_kiocb *req, unsigned int issue_flags)
struct file *in;
long ret = 0;
+ if (unlikely(!sp->len))
+ goto done;
+
if (issue_flags & IO_URING_F_NONBLOCK)
return -EAGAIN;
@@ -110,8 +115,7 @@ int io_splice(struct io_kiocb *req, unsigned int issue_flags)
poff_in = (sp->off_in == -1) ? NULL : &sp->off_in;
poff_out = (sp->off_out == -1) ? NULL : &sp->off_out;
- if (sp->len)
- ret = do_splice(in, poff_in, out, poff_out, sp->len, flags);
+ ret = do_splice(in, poff_in, out, poff_out, sp->len, flags);
if (!(sp->flags & SPLICE_F_FD_IN_FIXED))
io_put_file(in);
--
2.25.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 2/5] pipe: add trylock helpers for pipe lock
2022-06-07 8:06 [RFC 0/5] support nonblock submission for splice pipe to pipe Hao Xu
2022-06-07 8:06 ` [PATCH 1/5] io_uring: move sp->len check up for splice and tee Hao Xu
@ 2022-06-07 8:06 ` Hao Xu
2022-06-07 8:06 ` [PATCH 3/5] splice: support nonblock for splice from pipe to pipe Hao Xu
` (5 subsequent siblings)
7 siblings, 0 replies; 15+ messages in thread
From: Hao Xu @ 2022-06-07 8:06 UTC (permalink / raw)
To: io-uring
Cc: Jens Axboe, Pavel Begunkov, linux-fsdevel, Alexander Viro,
Andrew Morton, Luis Chamberlain, Kuniyuki Iwashima
From: Hao Xu <[email protected]>
Add two helpers pipe_trylock() and pipe_double_trylock(), they are used
in nonblock splice(pipe to pipe) scenario in next patches.
Signed-off-by: Hao Xu <[email protected]>
---
fs/pipe.c | 29 +++++++++++++++++++++++++++++
include/linux/pipe_fs_i.h | 2 ++
2 files changed, 31 insertions(+)
diff --git a/fs/pipe.c b/fs/pipe.c
index 74ae9fafd25a..033736eb61fb 100644
--- a/fs/pipe.c
+++ b/fs/pipe.c
@@ -98,6 +98,11 @@ void pipe_unlock(struct pipe_inode_info *pipe)
}
EXPORT_SYMBOL(pipe_unlock);
+int pipe_trylock(struct pipe_inode_info *pipe)
+{
+ return mutex_trylock(&pipe->mutex);
+}
+
static inline void __pipe_lock(struct pipe_inode_info *pipe)
{
mutex_lock_nested(&pipe->mutex, I_MUTEX_PARENT);
@@ -122,6 +127,30 @@ void pipe_double_lock(struct pipe_inode_info *pipe1,
}
}
+int pipe_double_trylock(struct pipe_inode_info *pipe1,
+ struct pipe_inode_info *pipe2)
+{
+ BUG_ON(pipe1 == pipe2);
+
+ if (pipe1 < pipe2) {
+ if (!pipe_trylock(pipe1))
+ return 0;
+ if (!pipe_trylock(pipe2)) {
+ pipe_unlock(pipe1);
+ return 0;
+ }
+ } else {
+ if (!pipe_trylock(pipe2))
+ return 0;
+ if (!pipe_trylock(pipe1)) {
+ pipe_unlock(pipe2);
+ return 0;
+ }
+ }
+
+ return 1;
+}
+
static void anon_pipe_buf_release(struct pipe_inode_info *pipe,
struct pipe_buffer *buf)
{
diff --git a/include/linux/pipe_fs_i.h b/include/linux/pipe_fs_i.h
index cb0fd633a610..78dc2c7c999f 100644
--- a/include/linux/pipe_fs_i.h
+++ b/include/linux/pipe_fs_i.h
@@ -235,8 +235,10 @@ static inline bool pipe_buf_try_steal(struct pipe_inode_info *pipe,
/* Pipe lock and unlock operations */
void pipe_lock(struct pipe_inode_info *);
+int pipe_trylock(struct pipe_inode_info *);
void pipe_unlock(struct pipe_inode_info *);
void pipe_double_lock(struct pipe_inode_info *, struct pipe_inode_info *);
+int pipe_double_trylock(struct pipe_inode_info *, struct pipe_inode_info *);
/* Wait for a pipe to be readable/writable while dropping the pipe lock */
void pipe_wait_readable(struct pipe_inode_info *);
--
2.25.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 3/5] splice: support nonblock for splice from pipe to pipe
2022-06-07 8:06 [RFC 0/5] support nonblock submission for splice pipe to pipe Hao Xu
2022-06-07 8:06 ` [PATCH 1/5] io_uring: move sp->len check up for splice and tee Hao Xu
2022-06-07 8:06 ` [PATCH 2/5] pipe: add trylock helpers for pipe lock Hao Xu
@ 2022-06-07 8:06 ` Hao Xu
2022-06-07 8:06 ` [PATCH 4/5] io_uring: support nonblock try for splicing " Hao Xu
` (4 subsequent siblings)
7 siblings, 0 replies; 15+ messages in thread
From: Hao Xu @ 2022-06-07 8:06 UTC (permalink / raw)
To: io-uring
Cc: Jens Axboe, Pavel Begunkov, linux-fsdevel, Alexander Viro,
Andrew Morton, Luis Chamberlain, Kuniyuki Iwashima
From: Hao Xu <[email protected]>
When SPLICE_F_NONBLOCK is set, splice() still may be blocked by pipe
lock in pipe to pipe scenario. Add trylock logic to make it more
nonblock
Signed-off-by: Hao Xu <[email protected]>
---
fs/splice.c | 21 ++++++++++++++++++---
1 file changed, 18 insertions(+), 3 deletions(-)
diff --git a/fs/splice.c b/fs/splice.c
index 047b79db8eb5..b087e00ed079 100644
--- a/fs/splice.c
+++ b/fs/splice.c
@@ -1372,7 +1372,12 @@ static int ipipe_prep(struct pipe_inode_info *pipe, unsigned int flags)
return 0;
ret = 0;
- pipe_lock(pipe);
+ if (flags & SPLICE_F_NONBLOCK) {
+ if (!pipe_trylock(pipe))
+ return -EAGAIN;
+ } else {
+ pipe_lock(pipe);
+ }
while (pipe_empty(pipe->head, pipe->tail)) {
if (signal_pending(current)) {
@@ -1408,7 +1413,12 @@ static int opipe_prep(struct pipe_inode_info *pipe, unsigned int flags)
return 0;
ret = 0;
- pipe_lock(pipe);
+ if (flags & SPLICE_F_NONBLOCK) {
+ if (!pipe_trylock(pipe))
+ return -EAGAIN;
+ } else {
+ pipe_lock(pipe);
+ }
while (pipe_full(pipe->head, pipe->tail, pipe->max_usage)) {
if (!pipe->readers) {
@@ -1460,7 +1470,12 @@ static int splice_pipe_to_pipe(struct pipe_inode_info *ipipe,
* grabbing by pipe info address. Otherwise two different processes
* could deadlock (one doing tee from A -> B, the other from B -> A).
*/
- pipe_double_lock(ipipe, opipe);
+ if (flags & SPLICE_F_NONBLOCK) {
+ if (!pipe_double_trylock(ipipe, opipe))
+ return -EAGAIN;
+ } else {
+ pipe_double_lock(ipipe, opipe);
+ }
i_tail = ipipe->tail;
i_mask = ipipe->ring_size - 1;
--
2.25.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 4/5] io_uring: support nonblock try for splicing from pipe to pipe
2022-06-07 8:06 [RFC 0/5] support nonblock submission for splice pipe to pipe Hao Xu
` (2 preceding siblings ...)
2022-06-07 8:06 ` [PATCH 3/5] splice: support nonblock for splice from pipe to pipe Hao Xu
@ 2022-06-07 8:06 ` Hao Xu
2022-06-07 21:38 ` kernel test robot
` (2 more replies)
2022-06-07 8:06 ` [PATCH 5/5] io_uring: add file_in in io_splice{} to avoid duplicate calculation Hao Xu
` (3 subsequent siblings)
7 siblings, 3 replies; 15+ messages in thread
From: Hao Xu @ 2022-06-07 8:06 UTC (permalink / raw)
To: io-uring
Cc: Jens Axboe, Pavel Begunkov, linux-fsdevel, Alexander Viro,
Andrew Morton, Luis Chamberlain, Kuniyuki Iwashima
From: Hao Xu <[email protected]>
splice() in io_uring is running in a slow way since it fully depends on
io workers. While splicing from pipe to pipe is a simpler case compared
with file to pipe and pipe to file. Let's make it support nonblock
first. This way we get a fast path for splicing fom pipe to pipe.
Signed-off-by: Hao Xu <[email protected]>
---
io_uring/splice.c | 24 +++++++++++++++++++++---
1 file changed, 21 insertions(+), 3 deletions(-)
diff --git a/io_uring/splice.c b/io_uring/splice.c
index b2cd1044c3ee..650c70e3dde1 100644
--- a/io_uring/splice.c
+++ b/io_uring/splice.c
@@ -88,6 +88,14 @@ int io_splice_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
return __io_splice_prep(req, sqe);
}
+bool io_splice_support_nowait(struct file *in, struct file *out)
+{
+ if (get_pipe_info(in, true) && get_pipe_info(out, true))
+ return true;
+
+ return false;
+}
+
int io_splice(struct io_kiocb *req, unsigned int issue_flags)
{
struct io_splice *sp = io_kiocb_to_cmd(req);
@@ -100,9 +108,6 @@ int io_splice(struct io_kiocb *req, unsigned int issue_flags)
if (unlikely(!sp->len))
goto done;
- if (issue_flags & IO_URING_F_NONBLOCK)
- return -EAGAIN;
-
if (sp->flags & SPLICE_F_FD_IN_FIXED)
in = io_file_get_fixed(req, sp->splice_fd_in, issue_flags);
else
@@ -112,6 +117,16 @@ int io_splice(struct io_kiocb *req, unsigned int issue_flags)
goto done;
}
+ if (issue_flags & IO_URING_F_NONBLOCK) {
+ if (io_splice_support_nowait(in, out)) {
+ flags |= SPLICE_F_NONBLOCK;
+ } else {
+ if (!(sp->flags & SPLICE_F_FD_IN_FIXED))
+ io_put_file(in);
+ return -EAGAIN;
+ }
+ }
+
poff_in = (sp->off_in == -1) ? NULL : &sp->off_in;
poff_out = (sp->off_out == -1) ? NULL : &sp->off_out;
@@ -119,6 +134,9 @@ int io_splice(struct io_kiocb *req, unsigned int issue_flags)
if (!(sp->flags & SPLICE_F_FD_IN_FIXED))
io_put_file(in);
+ if (ret == -EAGAIN)
+ return ret;
+
done:
if (ret != sp->len)
req_set_fail(req);
--
2.25.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 5/5] io_uring: add file_in in io_splice{} to avoid duplicate calculation
2022-06-07 8:06 [RFC 0/5] support nonblock submission for splice pipe to pipe Hao Xu
` (3 preceding siblings ...)
2022-06-07 8:06 ` [PATCH 4/5] io_uring: support nonblock try for splicing " Hao Xu
@ 2022-06-07 8:06 ` Hao Xu
2022-06-07 8:13 ` [RFC 0/5] support nonblock submission for splice pipe to pipe Hao Xu
` (2 subsequent siblings)
7 siblings, 0 replies; 15+ messages in thread
From: Hao Xu @ 2022-06-07 8:06 UTC (permalink / raw)
To: io-uring
Cc: Jens Axboe, Pavel Begunkov, linux-fsdevel, Alexander Viro,
Andrew Morton, Luis Chamberlain, Kuniyuki Iwashima
From: Hao Xu <[email protected]>
Add a member file_in in io_splice{} to avoid duplicate calculation of
input file for splice. This is for the case where we do splice from
pipe to pipe and get -EAGAIN for the inline submission.
Signed-off-by: Hao Xu <[email protected]>
---
io_uring/splice.c | 33 +++++++++++++++++++--------------
1 file changed, 19 insertions(+), 14 deletions(-)
diff --git a/io_uring/splice.c b/io_uring/splice.c
index 650c70e3dde1..c97f2971fe7e 100644
--- a/io_uring/splice.c
+++ b/io_uring/splice.c
@@ -21,6 +21,7 @@ struct io_splice {
loff_t off_in;
u64 len;
int splice_fd_in;
+ struct file *file_in;
unsigned int flags;
};
@@ -35,6 +36,7 @@ static int __io_splice_prep(struct io_kiocb *req,
if (unlikely(sp->flags & ~valid_flags))
return -EINVAL;
sp->splice_fd_in = READ_ONCE(sqe->splice_fd_in);
+ sp->file_in = NULL;
return 0;
}
@@ -108,34 +110,37 @@ int io_splice(struct io_kiocb *req, unsigned int issue_flags)
if (unlikely(!sp->len))
goto done;
- if (sp->flags & SPLICE_F_FD_IN_FIXED)
- in = io_file_get_fixed(req, sp->splice_fd_in, issue_flags);
- else
- in = io_file_get_normal(req, sp->splice_fd_in);
- if (!in) {
- ret = -EBADF;
- goto done;
+ if (!sp->file_in) {
+ if (sp->flags & SPLICE_F_FD_IN_FIXED)
+ in = io_file_get_fixed(req, sp->splice_fd_in, issue_flags);
+ else
+ in = io_file_get_normal(req, sp->splice_fd_in);
+
+ if (!in) {
+ ret = -EBADF;
+ goto done;
+ }
+ sp->file_in = in;
+ } else {
+ in = sp->file_in;
}
if (issue_flags & IO_URING_F_NONBLOCK) {
- if (io_splice_support_nowait(in, out)) {
+ if (io_splice_support_nowait(in, out))
flags |= SPLICE_F_NONBLOCK;
- } else {
- if (!(sp->flags & SPLICE_F_FD_IN_FIXED))
- io_put_file(in);
+ else
return -EAGAIN;
- }
}
poff_in = (sp->off_in == -1) ? NULL : &sp->off_in;
poff_out = (sp->off_out == -1) ? NULL : &sp->off_out;
ret = do_splice(in, poff_in, out, poff_out, sp->len, flags);
+ if (ret == -EAGAIN)
+ return ret;
if (!(sp->flags & SPLICE_F_FD_IN_FIXED))
io_put_file(in);
- if (ret == -EAGAIN)
- return ret;
done:
if (ret != sp->len)
--
2.25.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [RFC 0/5] support nonblock submission for splice pipe to pipe
2022-06-07 8:06 [RFC 0/5] support nonblock submission for splice pipe to pipe Hao Xu
` (4 preceding siblings ...)
2022-06-07 8:06 ` [PATCH 5/5] io_uring: add file_in in io_splice{} to avoid duplicate calculation Hao Xu
@ 2022-06-07 8:13 ` Hao Xu
2022-06-07 9:27 ` Pavel Begunkov
2022-06-14 13:23 ` Hao Xu
7 siblings, 0 replies; 15+ messages in thread
From: Hao Xu @ 2022-06-07 8:13 UTC (permalink / raw)
To: io-uring
Cc: Jens Axboe, Pavel Begunkov, linux-fsdevel, Alexander Viro,
Andrew Morton, Luis Chamberlain, Kuniyuki Iwashima
Hi,
On 6/7/22 16:06, Hao Xu wrote:
> From: Hao Xu <[email protected]>
>
> splice from pipe to pipe is a trivial case, and we can support nonblock
> try for it easily. splice depends on iowq at all which is slow. Let's
^ should be 'splice in io_uring'
> build a fast submission path for it by supporting nonblock.
>
> Wrote a simple test to test time spent of splicing from pipe to pipe:
>
Sorry, forgot to attach the test:
https://github.com/HowHsu/liburing/commit/3ff901b4fb80caf66a520742c6542e35111493a0
Regards,
Hao
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC 0/5] support nonblock submission for splice pipe to pipe
2022-06-07 8:06 [RFC 0/5] support nonblock submission for splice pipe to pipe Hao Xu
` (5 preceding siblings ...)
2022-06-07 8:13 ` [RFC 0/5] support nonblock submission for splice pipe to pipe Hao Xu
@ 2022-06-07 9:27 ` Pavel Begunkov
2022-06-08 4:19 ` Hao Xu
2022-06-14 13:23 ` Hao Xu
7 siblings, 1 reply; 15+ messages in thread
From: Pavel Begunkov @ 2022-06-07 9:27 UTC (permalink / raw)
To: Hao Xu, io-uring
Cc: Jens Axboe, linux-fsdevel, Alexander Viro, Andrew Morton,
Luis Chamberlain, Kuniyuki Iwashima
On 6/7/22 09:06, Hao Xu wrote:
> From: Hao Xu <[email protected]>
>
> splice from pipe to pipe is a trivial case, and we can support nonblock
> try for it easily. splice depends on iowq at all which is slow. Let's
> build a fast submission path for it by supporting nonblock.
fwiw,
https://www.spinics.net/lists/kernel/msg3652757.html
> Wrote a simple test to test time spent of splicing from pipe to pipe:
>
>
> Did 50 times test for each, ignore the highest and lowest number,
> calculate the average number:
>
> before patchset: 119.85 usec
> with patchset: 29.5 usec
>
> ----------------
> I'm not sure if we should use a io_uring specific flag rather than
> SPLICE_F_NONBLOCK since from mutex_lock_nest to mutex_trylock changes
> the behavior under debug environment I guess. Or maybe there is another
> better option than mutex_trylock?
>
>
> Hao Xu (5):
> io_uring: move sp->len check up for splice and tee
> pipe: add trylock helpers for pipe lock
> splice: support nonblock for splice from pipe to pipe
> io_uring: support nonblock try for splicing from pipe to pipe
> io_uring: add file_in in io_splice{} to avoid duplicate calculation
>
> fs/pipe.c | 29 +++++++++++++++++++++
> fs/splice.c | 21 +++++++++++++---
> include/linux/pipe_fs_i.h | 2 ++
> io_uring/splice.c | 53 +++++++++++++++++++++++++++++----------
> 4 files changed, 89 insertions(+), 16 deletions(-)
>
>
> base-commit: d8271bf021438f468dab3cd84fe5279b5bbcead8
--
Pavel Begunkov
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH 4/5] io_uring: support nonblock try for splicing from pipe to pipe
2022-06-07 8:06 ` [PATCH 4/5] io_uring: support nonblock try for splicing " Hao Xu
@ 2022-06-07 21:38 ` kernel test robot
2022-06-08 14:18 ` kernel test robot
2022-06-09 2:24 ` kernel test robot
2 siblings, 0 replies; 15+ messages in thread
From: kernel test robot @ 2022-06-07 21:38 UTC (permalink / raw)
To: Hao Xu, io-uring
Cc: llvm, kbuild-all, Jens Axboe, Pavel Begunkov, linux-fsdevel,
Alexander Viro, Andrew Morton, Linux Memory Management List,
Luis Chamberlain, Kuniyuki Iwashima
Hi Hao,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on d8271bf021438f468dab3cd84fe5279b5bbcead8]
url: https://github.com/intel-lab-lkp/linux/commits/Hao-Xu/support-nonblock-submission-for-splice-pipe-to-pipe/20220607-161605
base: d8271bf021438f468dab3cd84fe5279b5bbcead8
config: mips-buildonly-randconfig-r001-20220607 (https://download.01.org/0day-ci/archive/20220608/[email protected]/config)
compiler: clang version 15.0.0 (https://github.com/llvm/llvm-project b92436efcb7813fc481b30f2593a4907568d917a)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# install mips cross compiling tool for clang build
# apt-get install binutils-mips-linux-gnu
# https://github.com/intel-lab-lkp/linux/commit/96683840c3f19b77a536a259094d24e0cd93ebc0
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Hao-Xu/support-nonblock-submission-for-splice-pipe-to-pipe/20220607-161605
git checkout 96683840c3f19b77a536a259094d24e0cd93ebc0
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=mips SHELL=/bin/bash
If you fix the issue, kindly add following tag where applicable
Reported-by: kernel test robot <[email protected]>
All warnings (new ones prefixed by >>):
>> io_uring/splice.c:91:6: warning: no previous prototype for function 'io_splice_support_nowait' [-Wmissing-prototypes]
bool io_splice_support_nowait(struct file *in, struct file *out)
^
io_uring/splice.c:91:1: note: declare 'static' if the function is not intended to be used outside of this translation unit
bool io_splice_support_nowait(struct file *in, struct file *out)
^
static
1 warning generated.
vim +/io_splice_support_nowait +91 io_uring/splice.c
90
> 91 bool io_splice_support_nowait(struct file *in, struct file *out)
92 {
93 if (get_pipe_info(in, true) && get_pipe_info(out, true))
94 return true;
95
96 return false;
97 }
98
--
0-DAY CI Kernel Test Service
https://01.org/lkp
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC 0/5] support nonblock submission for splice pipe to pipe
2022-06-07 9:27 ` Pavel Begunkov
@ 2022-06-08 4:19 ` Hao Xu
2022-06-08 4:31 ` Jens Axboe
2022-06-08 11:33 ` Pavel Begunkov
0 siblings, 2 replies; 15+ messages in thread
From: Hao Xu @ 2022-06-08 4:19 UTC (permalink / raw)
To: Pavel Begunkov, io-uring
Cc: Jens Axboe, linux-fsdevel, Alexander Viro, Andrew Morton,
Luis Chamberlain, Kuniyuki Iwashima
On 6/7/22 17:27, Pavel Begunkov wrote:
> On 6/7/22 09:06, Hao Xu wrote:
>> From: Hao Xu <[email protected]>
>>
>> splice from pipe to pipe is a trivial case, and we can support nonblock
>> try for it easily. splice depends on iowq at all which is slow. Let's
>> build a fast submission path for it by supporting nonblock.
>
> fwiw,
>
> https://www.spinics.net/lists/kernel/msg3652757.html
>
Thanks, Pavel. Seems it has been discussed for a long time but the
result remains unclear...For me, I think this patch is necessary for
getting a good SPLICE_F_NONBLOCK user experience.
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC 0/5] support nonblock submission for splice pipe to pipe
2022-06-08 4:19 ` Hao Xu
@ 2022-06-08 4:31 ` Jens Axboe
2022-06-08 11:33 ` Pavel Begunkov
1 sibling, 0 replies; 15+ messages in thread
From: Jens Axboe @ 2022-06-08 4:31 UTC (permalink / raw)
To: Hao Xu, Pavel Begunkov, io-uring
Cc: linux-fsdevel, Alexander Viro, Andrew Morton, Luis Chamberlain,
Kuniyuki Iwashima
On 6/7/22 10:19 PM, Hao Xu wrote:
> On 6/7/22 17:27, Pavel Begunkov wrote:
>> On 6/7/22 09:06, Hao Xu wrote:
>>> From: Hao Xu <[email protected]>
>>>
>>> splice from pipe to pipe is a trivial case, and we can support nonblock
>>> try for it easily. splice depends on iowq at all which is slow. Let's
>>> build a fast submission path for it by supporting nonblock.
>>
>> fwiw,
>>
>> https://www.spinics.net/lists/kernel/msg3652757.html
>>
>
> Thanks, Pavel. Seems it has been discussed for a long time but the
> result remains unclear...For me, I think this patch is necessary for
> getting a good SPLICE_F_NONBLOCK user experience.
I'd just take it up again, this is something we do need to get done to
avoid io-wq offload. Which is important...
--
Jens Axboe
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC 0/5] support nonblock submission for splice pipe to pipe
2022-06-08 4:19 ` Hao Xu
2022-06-08 4:31 ` Jens Axboe
@ 2022-06-08 11:33 ` Pavel Begunkov
1 sibling, 0 replies; 15+ messages in thread
From: Pavel Begunkov @ 2022-06-08 11:33 UTC (permalink / raw)
To: Hao Xu, io-uring
Cc: Jens Axboe, linux-fsdevel, Alexander Viro, Andrew Morton,
Luis Chamberlain, Kuniyuki Iwashima
On 6/8/22 05:19, Hao Xu wrote:
> On 6/7/22 17:27, Pavel Begunkov wrote:
>> On 6/7/22 09:06, Hao Xu wrote:
>>> From: Hao Xu <[email protected]>
>>>
>>> splice from pipe to pipe is a trivial case, and we can support nonblock
>>> try for it easily. splice depends on iowq at all which is slow. Let's
>>> build a fast submission path for it by supporting nonblock.
>>
>> fwiw,
>>
>> https://www.spinics.net/lists/kernel/msg3652757.html
>>
>
> Thanks, Pavel. Seems it has been discussed for a long time but the
> result remains unclear...For me, I think this patch is necessary for getting a good SPLICE_F_NONBLOCK user experience.
I quite agree here, something like this is much needed.
The performance of io_uring pipe I/O is miserably slow
--
Pavel Begunkov
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH 4/5] io_uring: support nonblock try for splicing from pipe to pipe
2022-06-07 8:06 ` [PATCH 4/5] io_uring: support nonblock try for splicing " Hao Xu
2022-06-07 21:38 ` kernel test robot
@ 2022-06-08 14:18 ` kernel test robot
2022-06-09 2:24 ` kernel test robot
2 siblings, 0 replies; 15+ messages in thread
From: kernel test robot @ 2022-06-08 14:18 UTC (permalink / raw)
To: Hao Xu, io-uring
Cc: kbuild-all, Jens Axboe, Pavel Begunkov, linux-fsdevel,
Alexander Viro, Andrew Morton, Linux Memory Management List,
Luis Chamberlain, Kuniyuki Iwashima
Hi Hao,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on d8271bf021438f468dab3cd84fe5279b5bbcead8]
url: https://github.com/intel-lab-lkp/linux/commits/Hao-Xu/support-nonblock-submission-for-splice-pipe-to-pipe/20220607-161605
base: d8271bf021438f468dab3cd84fe5279b5bbcead8
config: x86_64-defconfig (https://download.01.org/0day-ci/archive/20220608/[email protected]/config)
compiler: gcc-11 (Debian 11.3.0-1) 11.3.0
reproduce (this is a W=1 build):
# https://github.com/intel-lab-lkp/linux/commit/96683840c3f19b77a536a259094d24e0cd93ebc0
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Hao-Xu/support-nonblock-submission-for-splice-pipe-to-pipe/20220607-161605
git checkout 96683840c3f19b77a536a259094d24e0cd93ebc0
# save the config file
mkdir build_dir && cp config build_dir/.config
make W=1 O=build_dir ARCH=x86_64 SHELL=/bin/bash
If you fix the issue, kindly add following tag where applicable
Reported-by: kernel test robot <[email protected]>
All warnings (new ones prefixed by >>):
>> io_uring/splice.c:91:6: warning: no previous prototype for 'io_splice_support_nowait' [-Wmissing-prototypes]
91 | bool io_splice_support_nowait(struct file *in, struct file *out)
| ^~~~~~~~~~~~~~~~~~~~~~~~
vim +/io_splice_support_nowait +91 io_uring/splice.c
90
> 91 bool io_splice_support_nowait(struct file *in, struct file *out)
92 {
93 if (get_pipe_info(in, true) && get_pipe_info(out, true))
94 return true;
95
96 return false;
97 }
98
--
0-DAY CI Kernel Test Service
https://01.org/lkp
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH 4/5] io_uring: support nonblock try for splicing from pipe to pipe
2022-06-07 8:06 ` [PATCH 4/5] io_uring: support nonblock try for splicing " Hao Xu
2022-06-07 21:38 ` kernel test robot
2022-06-08 14:18 ` kernel test robot
@ 2022-06-09 2:24 ` kernel test robot
2 siblings, 0 replies; 15+ messages in thread
From: kernel test robot @ 2022-06-09 2:24 UTC (permalink / raw)
To: Hao Xu, io-uring
Cc: kbuild-all, Jens Axboe, Pavel Begunkov, linux-fsdevel,
Alexander Viro, Andrew Morton, Linux Memory Management List,
Luis Chamberlain, Kuniyuki Iwashima
Hi Hao,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on d8271bf021438f468dab3cd84fe5279b5bbcead8]
url: https://github.com/intel-lab-lkp/linux/commits/Hao-Xu/support-nonblock-submission-for-splice-pipe-to-pipe/20220607-161605
base: d8271bf021438f468dab3cd84fe5279b5bbcead8
config: i386-randconfig-s001 (https://download.01.org/0day-ci/archive/20220609/[email protected]/config)
compiler: gcc-11 (Debian 11.3.0-1) 11.3.0
reproduce:
# apt-get install sparse
# sparse version: v0.6.4-26-gb3cf30ba-dirty
# https://github.com/intel-lab-lkp/linux/commit/96683840c3f19b77a536a259094d24e0cd93ebc0
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Hao-Xu/support-nonblock-submission-for-splice-pipe-to-pipe/20220607-161605
git checkout 96683840c3f19b77a536a259094d24e0cd93ebc0
# save the config file
mkdir build_dir && cp config build_dir/.config
make W=1 C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__' O=build_dir ARCH=i386 SHELL=/bin/bash
If you fix the issue, kindly add following tag where applicable
Reported-by: kernel test robot <[email protected]>
sparse warnings: (new ones prefixed by >>)
>> io_uring/splice.c:91:6: sparse: sparse: symbol 'io_splice_support_nowait' was not declared. Should it be static?
--
0-DAY CI Kernel Test Service
https://01.org/lkp
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC 0/5] support nonblock submission for splice pipe to pipe
2022-06-07 8:06 [RFC 0/5] support nonblock submission for splice pipe to pipe Hao Xu
` (6 preceding siblings ...)
2022-06-07 9:27 ` Pavel Begunkov
@ 2022-06-14 13:23 ` Hao Xu
7 siblings, 0 replies; 15+ messages in thread
From: Hao Xu @ 2022-06-14 13:23 UTC (permalink / raw)
To: io-uring
Cc: Jens Axboe, Pavel Begunkov, linux-fsdevel, Alexander Viro,
Andrew Morton, Luis Chamberlain, Kuniyuki Iwashima
On 6/7/22 16:06, Hao Xu wrote:
> From: Hao Xu <[email protected]>
>
> splice from pipe to pipe is a trivial case, and we can support nonblock
> try for it easily. splice depends on iowq at all which is slow. Let's
> build a fast submission path for it by supporting nonblock.
>
> Wrote a simple test to test time spent of splicing from pipe to pipe:
>
>
> Did 50 times test for each, ignore the highest and lowest number,
> calculate the average number:
>
> before patchset: 119.85 usec
> with patchset: 29.5 usec
>
Hi Al,
Any comments on this one?
Regards,
Hao
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2022-06-14 13:23 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-06-07 8:06 [RFC 0/5] support nonblock submission for splice pipe to pipe Hao Xu
2022-06-07 8:06 ` [PATCH 1/5] io_uring: move sp->len check up for splice and tee Hao Xu
2022-06-07 8:06 ` [PATCH 2/5] pipe: add trylock helpers for pipe lock Hao Xu
2022-06-07 8:06 ` [PATCH 3/5] splice: support nonblock for splice from pipe to pipe Hao Xu
2022-06-07 8:06 ` [PATCH 4/5] io_uring: support nonblock try for splicing " Hao Xu
2022-06-07 21:38 ` kernel test robot
2022-06-08 14:18 ` kernel test robot
2022-06-09 2:24 ` kernel test robot
2022-06-07 8:06 ` [PATCH 5/5] io_uring: add file_in in io_splice{} to avoid duplicate calculation Hao Xu
2022-06-07 8:13 ` [RFC 0/5] support nonblock submission for splice pipe to pipe Hao Xu
2022-06-07 9:27 ` Pavel Begunkov
2022-06-08 4:19 ` Hao Xu
2022-06-08 4:31 ` Jens Axboe
2022-06-08 11:33 ` Pavel Begunkov
2022-06-14 13:23 ` Hao Xu
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox