From: Stefan Roesch <[email protected]>
To: [email protected], [email protected]
Cc: [email protected], [email protected], [email protected],
[email protected], [email protected], [email protected]
Subject: [PATCH v13 2/7] net: introduce napi_busy_loop_rcu()
Date: Thu, 18 May 2023 14:17:46 -0700 [thread overview]
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
This introduces the napi_busy_loop_rcu() function. If the caller of
napi_busy_loop() function is also taking the rcu read lock, it is possible
that napi_busy_loop() is releasing the read lock if it invokes schedule.
However the caller is expecting that the rcu read lock is not released
until the function completes. This new function avoids that problem. It
expects that the caller MUST hold the rcu_read_lock while calling this
function.
Signed-off-by: Stefan Roesch <[email protected]>
---
include/net/busy_poll.h | 4 ++++
net/core/dev.c | 38 ++++++++++++++++++++++++++++++++++++++
2 files changed, 42 insertions(+)
diff --git a/include/net/busy_poll.h b/include/net/busy_poll.h
index f90f0021f5f2..622623f5740e 100644
--- a/include/net/busy_poll.h
+++ b/include/net/busy_poll.h
@@ -47,6 +47,10 @@ void napi_busy_loop(unsigned int napi_id,
bool (*loop_end)(void *, unsigned long),
void *loop_end_arg, bool prefer_busy_poll, u16 budget);
+void napi_busy_loop_rcu(unsigned int napi_id,
+ bool (*loop_end)(void *, unsigned long),
+ void *loop_end_arg, bool prefer_busy_poll, u16 budget);
+
#else /* CONFIG_NET_RX_BUSY_POLL */
static inline unsigned long net_busy_loop_on(void)
{
diff --git a/net/core/dev.c b/net/core/dev.c
index f4677aa20f84..51a1c70a70a2 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -6213,6 +6213,44 @@ static inline void __napi_busy_poll(struct napi_busy_poll_ctx *ctx,
LINUX_MIB_BUSYPOLLRXPACKETS, work);
local_bh_enable();
}
+
+/*
+ * napi_busy_loop_rcu() - busy poll (caller holds rcu read lock)
+ * @napi_id : napi id
+ * @loop_end : function to check for loop end
+ * @loop_end_arg : argument for loop end function
+ * @prefer_busy_poll: prefer busy poll
+ * @budget : budget for busy poll
+ *
+ * Warning: caller must hold rcu read lock.
+ */
+void napi_busy_loop_rcu(unsigned int napi_id,
+ bool (*loop_end)(void *, unsigned long),
+ void *loop_end_arg, bool prefer_busy_poll, u16 budget)
+{
+ unsigned long start_time = loop_end ? busy_loop_current_time() : 0;
+ struct napi_busy_poll_ctx ctx = {};
+
+ ctx.napi = napi_by_id(napi_id);
+ if (!ctx.napi)
+ return;
+
+ preempt_disable();
+ for (;;) {
+ __napi_busy_poll(&ctx, prefer_busy_poll, budget);
+
+ if (!loop_end || loop_end(loop_end_arg, start_time))
+ break;
+ if (unlikely(need_resched()))
+ break;
+
+ cpu_relax();
+ }
+ if (ctx.napi_poll)
+ busy_poll_stop(ctx.napi, ctx.have_poll_lock, prefer_busy_poll, budget);
+ preempt_enable();
+}
+
void napi_busy_loop(unsigned int napi_id,
bool (*loop_end)(void *, unsigned long),
void *loop_end_arg, bool prefer_busy_poll, u16 budget)
--
2.39.1
next prev parent reply other threads:[~2023-05-18 21:18 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-05-18 21:17 [PATCH v13 0/7] io_uring: add napi busy polling support Stefan Roesch
2023-05-18 21:17 ` [PATCH v13 1/7] net: split off __napi_busy_poll from napi_busy_poll Stefan Roesch
2023-05-31 17:26 ` Jakub Kicinski
2023-06-05 17:47 ` Stefan Roesch
2023-06-05 18:00 ` Jakub Kicinski
[not found] ` <[email protected]>
[not found] ` <[email protected]>
2023-06-01 4:15 ` Jakub Kicinski
2023-06-02 4:12 ` Stefan Roesch
2023-06-02 4:26 ` Jakub Kicinski
2023-05-18 21:17 ` Stefan Roesch [this message]
[not found] ` <[email protected]>
2023-05-31 17:38 ` [PATCH v13 2/7] net: introduce napi_busy_loop_rcu() Jakub Kicinski
2023-06-05 17:45 ` Stefan Roesch
2023-05-18 21:17 ` [PATCH v13 3/7] io-uring: move io_wait_queue definition to header file Stefan Roesch
2023-05-18 21:17 ` [PATCH v13 4/7] io-uring: add napi busy poll support Stefan Roesch
2023-05-19 1:26 ` Jens Axboe
2023-05-19 23:11 ` Stefan Roesch
2023-05-19 9:53 ` Simon Horman
2023-05-19 23:17 ` Stefan Roesch
2023-05-18 21:17 ` [PATCH v13 5/7] io-uring: add sqpoll support for napi busy poll Stefan Roesch
2023-05-19 0:11 ` kernel test robot
2023-05-19 1:13 ` Jens Axboe
2023-05-19 23:29 ` Stefan Roesch
2023-05-19 4:35 ` kernel test robot
2023-05-18 21:17 ` [PATCH v13 6/7] io_uring: add register/unregister napi function Stefan Roesch
2023-05-19 1:30 ` Jens Axboe
2023-05-18 21:17 ` [PATCH v13 7/7] io_uring: add prefer busy poll to register and unregister napi api Stefan Roesch
2023-05-19 1:31 ` [PATCH v13 0/7] io_uring: add napi busy polling support Jens Axboe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox