public inbox for [email protected]
 help / color / mirror / Atom feed
* [PATCH v2] io_uring: kbuf: inline io_kbuf_recycle_ring()
@ 2022-06-23 13:01 Hao Xu
  2022-06-23 13:59 ` Jens Axboe
  0 siblings, 1 reply; 2+ messages in thread
From: Hao Xu @ 2022-06-23 13:01 UTC (permalink / raw)
  To: io-uring; +Cc: Jens Axboe, Pavel Begunkov

From: Hao Xu <[email protected]>

Make io_kbuf_recycle_ring() inline since it is the fast path of
provided buffer.

Signed-off-by: Hao Xu <[email protected]>
---
 io_uring/kbuf.c | 28 ----------------------------
 io_uring/kbuf.h | 28 +++++++++++++++++++++++++++-
 2 files changed, 27 insertions(+), 29 deletions(-)

diff --git a/io_uring/kbuf.c b/io_uring/kbuf.c
index 4b7f2aa99e38..8e4f1e8aaf4a 100644
--- a/io_uring/kbuf.c
+++ b/io_uring/kbuf.c
@@ -74,34 +74,6 @@ void io_kbuf_recycle_legacy(struct io_kiocb *req, unsigned issue_flags)
 	return;
 }
 
-void io_kbuf_recycle_ring(struct io_kiocb *req)
-{
-	/*
-	 * We don't need to recycle for REQ_F_BUFFER_RING, we can just clear
-	 * the flag and hence ensure that bl->head doesn't get incremented.
-	 * If the tail has already been incremented, hang on to it.
-	 * The exception is partial io, that case we should increment bl->head
-	 * to monopolize the buffer.
-	 */
-	if (req->buf_list) {
-		if (req->flags & REQ_F_PARTIAL_IO) {
-			/*
-			 * If we end up here, then the io_uring_lock has
-			 * been kept held since we retrieved the buffer.
-			 * For the io-wq case, we already cleared
-			 * req->buf_list when the buffer was retrieved,
-			 * hence it cannot be set here for that case.
-			 */
-			req->buf_list->head++;
-			req->buf_list = NULL;
-		} else {
-			req->buf_index = req->buf_list->bgid;
-			req->flags &= ~REQ_F_BUFFER_RING;
-		}
-	}
-	return;
-}
-
 unsigned int __io_put_kbuf(struct io_kiocb *req, unsigned issue_flags)
 {
 	unsigned int cflags;
diff --git a/io_uring/kbuf.h b/io_uring/kbuf.h
index b5a89ffadf31..3d48f1ab5439 100644
--- a/io_uring/kbuf.h
+++ b/io_uring/kbuf.h
@@ -49,7 +49,33 @@ int io_unregister_pbuf_ring(struct io_ring_ctx *ctx, void __user *arg);
 unsigned int __io_put_kbuf(struct io_kiocb *req, unsigned issue_flags);
 
 void io_kbuf_recycle_legacy(struct io_kiocb *req, unsigned issue_flags);
-void io_kbuf_recycle_ring(struct io_kiocb *req);
+
+static inline void io_kbuf_recycle_ring(struct io_kiocb *req)
+{
+	/*
+	 * We don't need to recycle for REQ_F_BUFFER_RING, we can just clear
+	 * the flag and hence ensure that bl->head doesn't get incremented.
+	 * If the tail has already been incremented, hang on to it.
+	 * The exception is partial io, that case we should increment bl->head
+	 * to monopolize the buffer.
+	 */
+	if (req->buf_list) {
+		if (req->flags & REQ_F_PARTIAL_IO) {
+			/*
+			 * If we end up here, then the io_uring_lock has
+			 * been kept held since we retrieved the buffer.
+			 * For the io-wq case, we already cleared
+			 * req->buf_list when the buffer was retrieved,
+			 * hence it cannot be set here for that case.
+			 */
+			req->buf_list->head++;
+			req->buf_list = NULL;
+		} else {
+			req->buf_index = req->buf_list->bgid;
+			req->flags &= ~REQ_F_BUFFER_RING;
+		}
+	}
+}
 
 static inline bool io_do_buffer_select(struct io_kiocb *req)
 {

base-commit: 5ec69c3a15ae6e904d76545d9a9c686eb758def0
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 2+ messages in thread

* Re: [PATCH v2] io_uring: kbuf: inline io_kbuf_recycle_ring()
  2022-06-23 13:01 [PATCH v2] io_uring: kbuf: inline io_kbuf_recycle_ring() Hao Xu
@ 2022-06-23 13:59 ` Jens Axboe
  0 siblings, 0 replies; 2+ messages in thread
From: Jens Axboe @ 2022-06-23 13:59 UTC (permalink / raw)
  To: hao.xu, io-uring; +Cc: asml.silence

On Thu, 23 Jun 2022 21:01:26 +0800, Hao Xu wrote:
> From: Hao Xu <[email protected]>
> 
> Make io_kbuf_recycle_ring() inline since it is the fast path of
> provided buffer.
> 
> 

Applied, thanks!

[1/1] io_uring: kbuf: inline io_kbuf_recycle_ring()
      commit: 0c7e8750e623d48fb39f1284fe77299edbdbc2bc

Best regards,
-- 
Jens Axboe



^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2022-06-23 13:59 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-06-23 13:01 [PATCH v2] io_uring: kbuf: inline io_kbuf_recycle_ring() Hao Xu
2022-06-23 13:59 ` Jens Axboe

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox