* [PATCH v2] io_uring: kbuf: inline io_kbuf_recycle_ring()
@ 2022-06-23 13:01 Hao Xu
2022-06-23 13:59 ` Jens Axboe
0 siblings, 1 reply; 2+ messages in thread
From: Hao Xu @ 2022-06-23 13:01 UTC (permalink / raw)
To: io-uring; +Cc: Jens Axboe, Pavel Begunkov
From: Hao Xu <[email protected]>
Make io_kbuf_recycle_ring() inline since it is the fast path of
provided buffer.
Signed-off-by: Hao Xu <[email protected]>
---
io_uring/kbuf.c | 28 ----------------------------
io_uring/kbuf.h | 28 +++++++++++++++++++++++++++-
2 files changed, 27 insertions(+), 29 deletions(-)
diff --git a/io_uring/kbuf.c b/io_uring/kbuf.c
index 4b7f2aa99e38..8e4f1e8aaf4a 100644
--- a/io_uring/kbuf.c
+++ b/io_uring/kbuf.c
@@ -74,34 +74,6 @@ void io_kbuf_recycle_legacy(struct io_kiocb *req, unsigned issue_flags)
return;
}
-void io_kbuf_recycle_ring(struct io_kiocb *req)
-{
- /*
- * We don't need to recycle for REQ_F_BUFFER_RING, we can just clear
- * the flag and hence ensure that bl->head doesn't get incremented.
- * If the tail has already been incremented, hang on to it.
- * The exception is partial io, that case we should increment bl->head
- * to monopolize the buffer.
- */
- if (req->buf_list) {
- if (req->flags & REQ_F_PARTIAL_IO) {
- /*
- * If we end up here, then the io_uring_lock has
- * been kept held since we retrieved the buffer.
- * For the io-wq case, we already cleared
- * req->buf_list when the buffer was retrieved,
- * hence it cannot be set here for that case.
- */
- req->buf_list->head++;
- req->buf_list = NULL;
- } else {
- req->buf_index = req->buf_list->bgid;
- req->flags &= ~REQ_F_BUFFER_RING;
- }
- }
- return;
-}
-
unsigned int __io_put_kbuf(struct io_kiocb *req, unsigned issue_flags)
{
unsigned int cflags;
diff --git a/io_uring/kbuf.h b/io_uring/kbuf.h
index b5a89ffadf31..3d48f1ab5439 100644
--- a/io_uring/kbuf.h
+++ b/io_uring/kbuf.h
@@ -49,7 +49,33 @@ int io_unregister_pbuf_ring(struct io_ring_ctx *ctx, void __user *arg);
unsigned int __io_put_kbuf(struct io_kiocb *req, unsigned issue_flags);
void io_kbuf_recycle_legacy(struct io_kiocb *req, unsigned issue_flags);
-void io_kbuf_recycle_ring(struct io_kiocb *req);
+
+static inline void io_kbuf_recycle_ring(struct io_kiocb *req)
+{
+ /*
+ * We don't need to recycle for REQ_F_BUFFER_RING, we can just clear
+ * the flag and hence ensure that bl->head doesn't get incremented.
+ * If the tail has already been incremented, hang on to it.
+ * The exception is partial io, that case we should increment bl->head
+ * to monopolize the buffer.
+ */
+ if (req->buf_list) {
+ if (req->flags & REQ_F_PARTIAL_IO) {
+ /*
+ * If we end up here, then the io_uring_lock has
+ * been kept held since we retrieved the buffer.
+ * For the io-wq case, we already cleared
+ * req->buf_list when the buffer was retrieved,
+ * hence it cannot be set here for that case.
+ */
+ req->buf_list->head++;
+ req->buf_list = NULL;
+ } else {
+ req->buf_index = req->buf_list->bgid;
+ req->flags &= ~REQ_F_BUFFER_RING;
+ }
+ }
+}
static inline bool io_do_buffer_select(struct io_kiocb *req)
{
base-commit: 5ec69c3a15ae6e904d76545d9a9c686eb758def0
--
2.25.1
^ permalink raw reply related [flat|nested] 2+ messages in thread
* Re: [PATCH v2] io_uring: kbuf: inline io_kbuf_recycle_ring()
2022-06-23 13:01 [PATCH v2] io_uring: kbuf: inline io_kbuf_recycle_ring() Hao Xu
@ 2022-06-23 13:59 ` Jens Axboe
0 siblings, 0 replies; 2+ messages in thread
From: Jens Axboe @ 2022-06-23 13:59 UTC (permalink / raw)
To: hao.xu, io-uring; +Cc: asml.silence
On Thu, 23 Jun 2022 21:01:26 +0800, Hao Xu wrote:
> From: Hao Xu <[email protected]>
>
> Make io_kbuf_recycle_ring() inline since it is the fast path of
> provided buffer.
>
>
Applied, thanks!
[1/1] io_uring: kbuf: inline io_kbuf_recycle_ring()
commit: 0c7e8750e623d48fb39f1284fe77299edbdbc2bc
Best regards,
--
Jens Axboe
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2022-06-23 13:59 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-06-23 13:01 [PATCH v2] io_uring: kbuf: inline io_kbuf_recycle_ring() Hao Xu
2022-06-23 13:59 ` Jens Axboe
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox