public inbox for [email protected]
 help / color / mirror / Atom feed
* [PATCH 0/3] rw optimisation partial resend
@ 2021-10-16 23:07 Pavel Begunkov
  2021-10-16 23:07 ` [PATCH 1/3] io_uring: arm poll for non-nowait files Pavel Begunkov
                   ` (4 more replies)
  0 siblings, 5 replies; 7+ messages in thread
From: Pavel Begunkov @ 2021-10-16 23:07 UTC (permalink / raw)
  To: io-uring; +Cc: Jens Axboe, asml.silence

Screwed commit messages with rebase, it returns back the intended
structure: splitting 1/3 as a separate patch, 2/3 gets an actual
explanation.

Also, merge a change reported by kernel test robot about
set but not used variable rw.

Pavel Begunkov (3):
  io_uring: arm poll for non-nowait files
  io_uring: combine REQ_F_NOWAIT_{READ,WRITE} flags
  io_uring: simplify io_file_supports_nowait()

 fs/io_uring.c | 88 +++++++++++++++++++++------------------------------
 1 file changed, 36 insertions(+), 52 deletions(-)

-- 
2.33.0


^ permalink raw reply	[flat|nested] 7+ messages in thread

* [PATCH 1/3] io_uring: arm poll for non-nowait files
  2021-10-16 23:07 [PATCH 0/3] rw optimisation partial resend Pavel Begunkov
@ 2021-10-16 23:07 ` Pavel Begunkov
  2021-10-16 23:07 ` [PATCH 2/3] io_uring: combine REQ_F_NOWAIT_{READ,WRITE} flags Pavel Begunkov
                   ` (3 subsequent siblings)
  4 siblings, 0 replies; 7+ messages in thread
From: Pavel Begunkov @ 2021-10-16 23:07 UTC (permalink / raw)
  To: io-uring; +Cc: Jens Axboe, asml.silence

Don't check if we can do nowait before arming apoll, there are several
reasons for that. First, we don't care much about files that don't
support nowait. Second, it may be useful -- we don't want to be taking
away extra workers from io-wq when it can go in some async. Even if it
will go through io-wq eventually, it make difference in the numbers of
workers actually used. And the last one, it's needed to clean nowait in
future commits.

[kernel test robot: fix unused-var]
Signed-off-by: Pavel Begunkov <[email protected]>
---
 fs/io_uring.c | 7 -------
 1 file changed, 7 deletions(-)

diff --git a/fs/io_uring.c b/fs/io_uring.c
index ce9a1b89da3f..14566d1bf174 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -5591,7 +5591,6 @@ static int io_arm_poll_handler(struct io_kiocb *req)
 	struct async_poll *apoll;
 	struct io_poll_table ipt;
 	__poll_t ret, mask = EPOLLONESHOT | POLLERR | POLLPRI;
-	int rw;
 
 	if (!req->file || !file_can_poll(req->file))
 		return IO_APOLL_ABORTED;
@@ -5601,7 +5600,6 @@ static int io_arm_poll_handler(struct io_kiocb *req)
 		return IO_APOLL_ABORTED;
 
 	if (def->pollin) {
-		rw = READ;
 		mask |= POLLIN | POLLRDNORM;
 
 		/* If reading from MSG_ERRQUEUE using recvmsg, ignore POLLIN */
@@ -5609,14 +5607,9 @@ static int io_arm_poll_handler(struct io_kiocb *req)
 		    (req->sr_msg.msg_flags & MSG_ERRQUEUE))
 			mask &= ~POLLIN;
 	} else {
-		rw = WRITE;
 		mask |= POLLOUT | POLLWRNORM;
 	}
 
-	/* if we can't nonblock try, then no point in arming a poll handler */
-	if (!io_file_supports_nowait(req, rw))
-		return IO_APOLL_ABORTED;
-
 	apoll = kmalloc(sizeof(*apoll), GFP_ATOMIC);
 	if (unlikely(!apoll))
 		return IO_APOLL_ABORTED;
-- 
2.33.0


^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [PATCH 2/3] io_uring: combine REQ_F_NOWAIT_{READ,WRITE} flags
  2021-10-16 23:07 [PATCH 0/3] rw optimisation partial resend Pavel Begunkov
  2021-10-16 23:07 ` [PATCH 1/3] io_uring: arm poll for non-nowait files Pavel Begunkov
@ 2021-10-16 23:07 ` Pavel Begunkov
  2021-10-16 23:07 ` [PATCH 3/3] io_uring: simplify io_file_supports_nowait() Pavel Begunkov
                   ` (2 subsequent siblings)
  4 siblings, 0 replies; 7+ messages in thread
From: Pavel Begunkov @ 2021-10-16 23:07 UTC (permalink / raw)
  To: io-uring; +Cc: Jens Axboe, asml.silence

Merge REQ_F_NOWAIT_READ and REQ_F_NOWAIT_WRITE into one flag, i.e.
REQ_F_SUPPORT_NOWAIT. First it gets rid of dependence on CONFIG_64BIT
but also simplifies the code.

One thing to consider is when we don't have ->{read,write}_iter and go
through loop_rw_iter(). Just fail it with -EAGAIN if we expect nowait
behaviour but not sure whether it supports it.

Signed-off-by: Pavel Begunkov <[email protected]>
---
 fs/io_uring.c | 61 ++++++++++++++++++---------------------------------
 1 file changed, 21 insertions(+), 40 deletions(-)

diff --git a/fs/io_uring.c b/fs/io_uring.c
index 14566d1bf174..06444b2f9a32 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -732,8 +732,7 @@ enum {
 	REQ_F_ARM_LTIMEOUT_BIT,
 	REQ_F_ASYNC_DATA_BIT,
 	/* keep async read/write and isreg together and in order */
-	REQ_F_NOWAIT_READ_BIT,
-	REQ_F_NOWAIT_WRITE_BIT,
+	REQ_F_SUPPORT_NOWAIT_BIT,
 	REQ_F_ISREG_BIT,
 
 	/* not a real bit, just to check we're not overflowing the space */
@@ -774,10 +773,8 @@ enum {
 	REQ_F_COMPLETE_INLINE	= BIT(REQ_F_COMPLETE_INLINE_BIT),
 	/* caller should reissue async */
 	REQ_F_REISSUE		= BIT(REQ_F_REISSUE_BIT),
-	/* supports async reads */
-	REQ_F_NOWAIT_READ	= BIT(REQ_F_NOWAIT_READ_BIT),
-	/* supports async writes */
-	REQ_F_NOWAIT_WRITE	= BIT(REQ_F_NOWAIT_WRITE_BIT),
+	/* supports async reads/writes */
+	REQ_F_SUPPORT_NOWAIT	= BIT(REQ_F_SUPPORT_NOWAIT_BIT),
 	/* regular file */
 	REQ_F_ISREG		= BIT(REQ_F_ISREG_BIT),
 	/* has creds assigned */
@@ -1390,18 +1387,13 @@ static bool req_need_defer(struct io_kiocb *req, u32 seq)
 	return false;
 }
 
-#define FFS_ASYNC_READ		0x1UL
-#define FFS_ASYNC_WRITE		0x2UL
-#ifdef CONFIG_64BIT
-#define FFS_ISREG		0x4UL
-#else
-#define FFS_ISREG		0x0UL
-#endif
-#define FFS_MASK		~(FFS_ASYNC_READ|FFS_ASYNC_WRITE|FFS_ISREG)
+#define FFS_NOWAIT		0x1UL
+#define FFS_ISREG		0x2UL
+#define FFS_MASK		~(FFS_NOWAIT|FFS_ISREG)
 
 static inline bool io_req_ffs_set(struct io_kiocb *req)
 {
-	return IS_ENABLED(CONFIG_64BIT) && (req->flags & REQ_F_FIXED_FILE);
+	return req->flags & REQ_F_FIXED_FILE;
 }
 
 static inline void io_req_track_inflight(struct io_kiocb *req)
@@ -2775,7 +2767,7 @@ static bool io_bdev_nowait(struct block_device *bdev)
  * any file. For now, just ensure that anything potentially problematic is done
  * inline.
  */
-static bool __io_file_supports_nowait(struct file *file, int rw)
+static bool __io_file_supports_nowait(struct file *file)
 {
 	umode_t mode = file_inode(file)->i_mode;
 
@@ -2798,24 +2790,14 @@ static bool __io_file_supports_nowait(struct file *file, int rw)
 	/* any ->read/write should understand O_NONBLOCK */
 	if (file->f_flags & O_NONBLOCK)
 		return true;
-
-	if (!(file->f_mode & FMODE_NOWAIT))
-		return false;
-
-	if (rw == READ)
-		return file->f_op->read_iter != NULL;
-
-	return file->f_op->write_iter != NULL;
+	return file->f_mode & FMODE_NOWAIT;
 }
 
-static bool io_file_supports_nowait(struct io_kiocb *req, int rw)
+static inline bool io_file_supports_nowait(struct io_kiocb *req)
 {
-	if (rw == READ && (req->flags & REQ_F_NOWAIT_READ))
+	if (likely(req->flags & REQ_F_SUPPORT_NOWAIT))
 		return true;
-	else if (rw == WRITE && (req->flags & REQ_F_NOWAIT_WRITE))
-		return true;
-
-	return __io_file_supports_nowait(req->file, rw);
+	return __io_file_supports_nowait(req->file);
 }
 
 static int io_prep_rw(struct io_kiocb *req, const struct io_uring_sqe *sqe,
@@ -2847,7 +2829,7 @@ static int io_prep_rw(struct io_kiocb *req, const struct io_uring_sqe *sqe,
 	 * reliably. If not, or it IOCB_NOWAIT is set, don't retry.
 	 */
 	if ((kiocb->ki_flags & IOCB_NOWAIT) ||
-	    ((file->f_flags & O_NONBLOCK) && !io_file_supports_nowait(req, rw)))
+	    ((file->f_flags & O_NONBLOCK) && !io_file_supports_nowait(req)))
 		req->flags |= REQ_F_NOWAIT;
 
 	ioprio = READ_ONCE(sqe->ioprio);
@@ -3238,7 +3220,8 @@ static ssize_t loop_rw_iter(int rw, struct io_kiocb *req, struct iov_iter *iter)
 	 */
 	if (kiocb->ki_flags & IOCB_HIPRI)
 		return -EOPNOTSUPP;
-	if (kiocb->ki_flags & IOCB_NOWAIT)
+	if ((kiocb->ki_flags & IOCB_NOWAIT) &&
+	    !(kiocb->ki_filp->f_flags & O_NONBLOCK))
 		return -EAGAIN;
 
 	while (iov_iter_count(iter)) {
@@ -3478,7 +3461,7 @@ static int io_read(struct io_kiocb *req, unsigned int issue_flags)
 
 	if (force_nonblock) {
 		/* If the file doesn't support async, just async punt */
-		if (unlikely(!io_file_supports_nowait(req, READ))) {
+		if (unlikely(!io_file_supports_nowait(req))) {
 			ret = io_setup_async_rw(req, iovec, s, true);
 			return ret ?: -EAGAIN;
 		}
@@ -3602,7 +3585,7 @@ static int io_write(struct io_kiocb *req, unsigned int issue_flags)
 
 	if (force_nonblock) {
 		/* If the file doesn't support async, just async punt */
-		if (unlikely(!io_file_supports_nowait(req, WRITE)))
+		if (unlikely(!io_file_supports_nowait(req)))
 			goto copy_iov;
 
 		/* file path doesn't support NOWAIT for non-direct_IO */
@@ -3634,7 +3617,7 @@ static int io_write(struct io_kiocb *req, unsigned int issue_flags)
 	}
 	kiocb->ki_flags |= IOCB_WRITE;
 
-	if (req->file->f_op->write_iter)
+	if (likely(req->file->f_op->write_iter))
 		ret2 = call_write_iter(req->file, kiocb, &s->iter);
 	else if (req->file->f_op->write)
 		ret2 = loop_rw_iter(WRITE, req, &s->iter);
@@ -6781,10 +6764,8 @@ static void io_fixed_file_set(struct io_fixed_file *file_slot, struct file *file
 {
 	unsigned long file_ptr = (unsigned long) file;
 
-	if (__io_file_supports_nowait(file, READ))
-		file_ptr |= FFS_ASYNC_READ;
-	if (__io_file_supports_nowait(file, WRITE))
-		file_ptr |= FFS_ASYNC_WRITE;
+	if (__io_file_supports_nowait(file))
+		file_ptr |= FFS_NOWAIT;
 	if (S_ISREG(file_inode(file)->i_mode))
 		file_ptr |= FFS_ISREG;
 	file_slot->file_ptr = file_ptr;
@@ -6803,7 +6784,7 @@ static inline struct file *io_file_get_fixed(struct io_ring_ctx *ctx,
 	file = (struct file *) (file_ptr & FFS_MASK);
 	file_ptr &= ~FFS_MASK;
 	/* mask in overlapping REQ_F and FFS bits */
-	req->flags |= (file_ptr << REQ_F_NOWAIT_READ_BIT);
+	req->flags |= (file_ptr << REQ_F_SUPPORT_NOWAIT_BIT);
 	io_req_set_rsrc_node(req, ctx);
 	return file;
 }
-- 
2.33.0


^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [PATCH 3/3] io_uring: simplify io_file_supports_nowait()
  2021-10-16 23:07 [PATCH 0/3] rw optimisation partial resend Pavel Begunkov
  2021-10-16 23:07 ` [PATCH 1/3] io_uring: arm poll for non-nowait files Pavel Begunkov
  2021-10-16 23:07 ` [PATCH 2/3] io_uring: combine REQ_F_NOWAIT_{READ,WRITE} flags Pavel Begunkov
@ 2021-10-16 23:07 ` Pavel Begunkov
  2021-10-16 23:11 ` [PATCH 0/3] rw optimisation partial resend Pavel Begunkov
  2021-10-17 14:25 ` Jens Axboe
  4 siblings, 0 replies; 7+ messages in thread
From: Pavel Begunkov @ 2021-10-16 23:07 UTC (permalink / raw)
  To: io-uring; +Cc: Jens Axboe, asml.silence

Make sure that REQ_F_SUPPORT_NOWAIT is always set io_prep_rw(), and so
we can stop caring about setting it down the line simplifying
io_file_supports_nowait().

Signed-off-by: Pavel Begunkov <[email protected]>
---
 fs/io_uring.c | 34 ++++++++++++++++++++++------------
 1 file changed, 22 insertions(+), 12 deletions(-)

diff --git a/fs/io_uring.c b/fs/io_uring.c
index 06444b2f9a32..18dac2aece59 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -2767,10 +2767,8 @@ static bool io_bdev_nowait(struct block_device *bdev)
  * any file. For now, just ensure that anything potentially problematic is done
  * inline.
  */
-static bool __io_file_supports_nowait(struct file *file)
+static bool __io_file_supports_nowait(struct file *file, umode_t mode)
 {
-	umode_t mode = file_inode(file)->i_mode;
-
 	if (S_ISBLK(mode)) {
 		if (IS_ENABLED(CONFIG_BLOCK) &&
 		    io_bdev_nowait(I_BDEV(file->f_mapping->host)))
@@ -2793,11 +2791,26 @@ static bool __io_file_supports_nowait(struct file *file)
 	return file->f_mode & FMODE_NOWAIT;
 }
 
+/*
+ * If we tracked the file through the SCM inflight mechanism, we could support
+ * any file. For now, just ensure that anything potentially problematic is done
+ * inline.
+ */
+static unsigned int io_file_get_flags(struct file *file)
+{
+	umode_t mode = file_inode(file)->i_mode;
+	unsigned int res = 0;
+
+	if (S_ISREG(mode))
+		res |= FFS_ISREG;
+	if (__io_file_supports_nowait(file, mode))
+		res |= FFS_NOWAIT;
+	return res;
+}
+
 static inline bool io_file_supports_nowait(struct io_kiocb *req)
 {
-	if (likely(req->flags & REQ_F_SUPPORT_NOWAIT))
-		return true;
-	return __io_file_supports_nowait(req->file);
+	return req->flags & REQ_F_SUPPORT_NOWAIT;
 }
 
 static int io_prep_rw(struct io_kiocb *req, const struct io_uring_sqe *sqe,
@@ -2809,8 +2822,8 @@ static int io_prep_rw(struct io_kiocb *req, const struct io_uring_sqe *sqe,
 	unsigned ioprio;
 	int ret;
 
-	if (!io_req_ffs_set(req) && S_ISREG(file_inode(file)->i_mode))
-		req->flags |= REQ_F_ISREG;
+	if (!io_req_ffs_set(req))
+		req->flags |= io_file_get_flags(file) << REQ_F_SUPPORT_NOWAIT_BIT;
 
 	kiocb->ki_pos = READ_ONCE(sqe->off);
 	if (kiocb->ki_pos == -1 && !(file->f_mode & FMODE_STREAM)) {
@@ -6764,10 +6777,7 @@ static void io_fixed_file_set(struct io_fixed_file *file_slot, struct file *file
 {
 	unsigned long file_ptr = (unsigned long) file;
 
-	if (__io_file_supports_nowait(file))
-		file_ptr |= FFS_NOWAIT;
-	if (S_ISREG(file_inode(file)->i_mode))
-		file_ptr |= FFS_ISREG;
+	file_ptr |= io_file_get_flags(file);
 	file_slot->file_ptr = file_ptr;
 }
 
-- 
2.33.0


^ permalink raw reply related	[flat|nested] 7+ messages in thread

* Re: [PATCH 0/3] rw optimisation partial resend
  2021-10-16 23:07 [PATCH 0/3] rw optimisation partial resend Pavel Begunkov
                   ` (2 preceding siblings ...)
  2021-10-16 23:07 ` [PATCH 3/3] io_uring: simplify io_file_supports_nowait() Pavel Begunkov
@ 2021-10-16 23:11 ` Pavel Begunkov
  2021-10-16 23:17   ` Pavel Begunkov
  2021-10-17 14:25 ` Jens Axboe
  4 siblings, 1 reply; 7+ messages in thread
From: Pavel Begunkov @ 2021-10-16 23:11 UTC (permalink / raw)
  To: io-uring; +Cc: Jens Axboe

On 10/17/21 00:07, Pavel Begunkov wrote:
> Screwed commit messages with rebase, it returns back the intended
> structure: splitting 1/3 as a separate patch, 2/3 gets an actual
> explanation.

Ok, let me resend it with changes Noah mentioned


> Also, merge a change reported by kernel test robot about
> set but not used variable rw.
> 
> Pavel Begunkov (3):
>    io_uring: arm poll for non-nowait files
>    io_uring: combine REQ_F_NOWAIT_{READ,WRITE} flags
>    io_uring: simplify io_file_supports_nowait()
> 
>   fs/io_uring.c | 88 +++++++++++++++++++++------------------------------
>   1 file changed, 36 insertions(+), 52 deletions(-)
> 

-- 
Pavel Begunkov

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH 0/3] rw optimisation partial resend
  2021-10-16 23:11 ` [PATCH 0/3] rw optimisation partial resend Pavel Begunkov
@ 2021-10-16 23:17   ` Pavel Begunkov
  0 siblings, 0 replies; 7+ messages in thread
From: Pavel Begunkov @ 2021-10-16 23:17 UTC (permalink / raw)
  To: io-uring; +Cc: Jens Axboe

On 10/17/21 00:11, Pavel Begunkov wrote:
> On 10/17/21 00:07, Pavel Begunkov wrote:
>> Screwed commit messages with rebase, it returns back the intended
>> structure: splitting 1/3 as a separate patch, 2/3 gets an actual
>> explanation.
> 
> Ok, let me resend it with changes Noah mentioned

not for this series though, iow good to go

> 
> 
>> Also, merge a change reported by kernel test robot about
>> set but not used variable rw.
>>
>> Pavel Begunkov (3):
>>    io_uring: arm poll for non-nowait files
>>    io_uring: combine REQ_F_NOWAIT_{READ,WRITE} flags
>>    io_uring: simplify io_file_supports_nowait()
>>
>>   fs/io_uring.c | 88 +++++++++++++++++++++------------------------------
>>   1 file changed, 36 insertions(+), 52 deletions(-)
>>
> 

-- 
Pavel Begunkov

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH 0/3] rw optimisation partial resend
  2021-10-16 23:07 [PATCH 0/3] rw optimisation partial resend Pavel Begunkov
                   ` (3 preceding siblings ...)
  2021-10-16 23:11 ` [PATCH 0/3] rw optimisation partial resend Pavel Begunkov
@ 2021-10-17 14:25 ` Jens Axboe
  4 siblings, 0 replies; 7+ messages in thread
From: Jens Axboe @ 2021-10-17 14:25 UTC (permalink / raw)
  To: Pavel Begunkov, io-uring; +Cc: Jens Axboe

On Sun, 17 Oct 2021 00:07:07 +0100, Pavel Begunkov wrote:
> Screwed commit messages with rebase, it returns back the intended
> structure: splitting 1/3 as a separate patch, 2/3 gets an actual
> explanation.
> 
> Also, merge a change reported by kernel test robot about
> set but not used variable rw.
> 
> [...]

Applied, thanks!

[1/3] io_uring: arm poll for non-nowait files
      commit: feabed278b191505df0ab7a382bc04b270ffb1f4
[2/3] io_uring: combine REQ_F_NOWAIT_{READ,WRITE} flags
      commit: c142f8627b24af12b958acd79c55761d52eab548
[3/3] io_uring: simplify io_file_supports_nowait()
      commit: c533d6e48e8a1d20e46c54231052b574005c2725

Best regards,
-- 
Jens Axboe



^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2021-10-17 14:25 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-10-16 23:07 [PATCH 0/3] rw optimisation partial resend Pavel Begunkov
2021-10-16 23:07 ` [PATCH 1/3] io_uring: arm poll for non-nowait files Pavel Begunkov
2021-10-16 23:07 ` [PATCH 2/3] io_uring: combine REQ_F_NOWAIT_{READ,WRITE} flags Pavel Begunkov
2021-10-16 23:07 ` [PATCH 3/3] io_uring: simplify io_file_supports_nowait() Pavel Begunkov
2021-10-16 23:11 ` [PATCH 0/3] rw optimisation partial resend Pavel Begunkov
2021-10-16 23:17   ` Pavel Begunkov
2021-10-17 14:25 ` Jens Axboe

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox