public inbox for io-uring@vger.kernel.org
 help / color / mirror / Atom feed
From: Willem de Bruijn <willemdebruijn.kernel@gmail.com>
To: Pavel Begunkov <asml.silence@gmail.com>,
	 io-uring@vger.kernel.org,
	 Vadim Fedorenko <vadim.fedorenko@linux.dev>
Cc: asml.silence@gmail.com,  netdev@vger.kernel.org,
	 Eric Dumazet <edumazet@google.com>,
	 Kuniyuki Iwashima <kuniyu@amazon.com>,
	 Paolo Abeni <pabeni@redhat.com>,
	 Willem de Bruijn <willemb@google.com>,
	 "David S . Miller" <davem@davemloft.net>,
	 Jakub Kicinski <kuba@kernel.org>,
	 Richard Cochran <richardcochran@gmail.com>,
	 Stanislav Fomichev <sdf@fomichev.me>,
	 Jason Xing <kerneljasonxing@gmail.com>
Subject: Re: [PATCH v3 5/5] io_uring/netcmd: add tx timestamping cmd support
Date: Thu, 12 Jun 2025 17:35:38 -0400	[thread overview]
Message-ID: <684b482a1a03f_cb2792944c@willemb.c.googlers.com.notmuch> (raw)
In-Reply-To: <1e9c0e393d6d207ba438da3ad5bf7e4125b28cb7.1749657325.git.asml.silence@gmail.com>

Pavel Begunkov wrote:
> Add a new socket command which returns tx time stamps to the user. It
> provide an alternative to the existing error queue recvmsg interface.
> The command works in a polled multishot mode, which means io_uring will
> poll the socket and keep posting timestamps until the request is
> cancelled or fails in any other way (e.g. with no space in the CQ). It
> reuses the net infra and grabs timestamps from the socket's error queue.
> 
> The command requires IORING_SETUP_CQE32. All non-final CQEs (marked with
> IORING_CQE_F_MORE) have cqe->res set to the tskey, and the upper 16 bits
> of cqe->flags keep tstype (i.e. offset by IORING_CQE_BUFFER_SHIFT). The
> timevalue is store in the upper part of the extended CQE. The final
> completion won't have IORING_CQR_F_MORE and will have cqe->res storing
> 0/error.
> 
> Suggested-by: Vadim Fedorenko <vadim.fedorenko@linux.dev>
> Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
> ---
>  include/uapi/linux/io_uring.h |  9 ++++
>  io_uring/cmd_net.c            | 82 +++++++++++++++++++++++++++++++++++
>  2 files changed, 91 insertions(+)
> 
> diff --git a/include/uapi/linux/io_uring.h b/include/uapi/linux/io_uring.h
> index cfd17e382082..5c89e6f6d624 100644
> --- a/include/uapi/linux/io_uring.h
> +++ b/include/uapi/linux/io_uring.h
> @@ -968,6 +968,15 @@ enum io_uring_socket_op {
>  	SOCKET_URING_OP_SIOCOUTQ,
>  	SOCKET_URING_OP_GETSOCKOPT,
>  	SOCKET_URING_OP_SETSOCKOPT,
> +	SOCKET_URING_OP_TX_TIMESTAMP,
> +};
> +
> +#define IORING_CQE_F_TIMESTAMP_HW	((__u32)1 << IORING_CQE_BUFFER_SHIFT)
> +#define IORING_TIMESTAMP_TSTYPE_SHIFT	(IORING_CQE_BUFFER_SHIFT + 1)
> +

Perhaps instead of these shifts define an actual struct, e.g.,
io_uring_cqe_tstamp.

One question is the number of bits to reserve for the tstype.
Currently only 2 are needed. But that can grow. The current
approach conveniently leaves that open.

Alternatively, perhaps make the dependency between the shifts more
obvious:

+#define IORING_TIMESTAMP_HW_SHIFT	IORING_CQE_BUFFER_SHIFT
+#define IORING_TIMESTAMP_TYPE_SHIFT	(IORING_CQE_BUFFER_SHIFT + 1)

+#define IORING_CQE_F_TSTAMP_HW		((__u32)1 << IORING_TIMESTAMP_HW_SHIFT);

> +struct io_timespec {
> +	__u64		tv_sec;
> +	__u64		tv_nsec;
>  };
>  
>  /* Zero copy receive refill queue entry */
> diff --git a/io_uring/cmd_net.c b/io_uring/cmd_net.c
> index e99170c7d41a..bc2d33ea2db3 100644
> --- a/io_uring/cmd_net.c
> +++ b/io_uring/cmd_net.c
> @@ -1,5 +1,6 @@
>  #include <asm/ioctls.h>
>  #include <linux/io_uring/net.h>
> +#include <linux/errqueue.h>
>  #include <net/sock.h>
>  
>  #include "uring_cmd.h"
> @@ -51,6 +52,85 @@ static inline int io_uring_cmd_setsockopt(struct socket *sock,
>  				  optlen);
>  }
>  
> +static bool io_process_timestamp_skb(struct io_uring_cmd *cmd, struct sock *sk,
> +				     struct sk_buff *skb, unsigned issue_flags)
> +{
> +	struct sock_exterr_skb *serr = SKB_EXT_ERR(skb);
> +	struct io_uring_cqe cqe[2];
> +	struct io_timespec *iots;
> +	struct timespec64 ts;
> +	u32 tstype, tskey;
> +	int ret;
> +
> +	BUILD_BUG_ON(sizeof(struct io_uring_cqe) != sizeof(struct io_timespec));
> +
> +	ret = skb_get_tx_timestamp(skb, sk, &ts);
> +	if (ret < 0)
> +		return false;
> +
> +	tskey = serr->ee.ee_data;
> +	tstype = serr->ee.ee_info;
> +
> +	cqe->user_data = 0;
> +	cqe->res = tskey;
> +	cqe->flags = IORING_CQE_F_MORE;
> +	cqe->flags |= tstype << IORING_TIMESTAMP_TSTYPE_SHIFT;
> +	if (ret == NET_TIMESTAMP_ORIGIN_HW)
> +		cqe->flags |= IORING_CQE_F_TIMESTAMP_HW;
> +
> +	iots = (struct io_timespec *)&cqe[1];
> +	iots->tv_sec = ts.tv_sec;
> +	iots->tv_nsec = ts.tv_nsec;
> +	return io_uring_cmd_post_mshot_cqe32(cmd, issue_flags, cqe);
> +}
> +
> +static int io_uring_cmd_timestamp(struct socket *sock,
> +				  struct io_uring_cmd *cmd,
> +				  unsigned int issue_flags)
> +{
> +	struct sock *sk = sock->sk;
> +	struct sk_buff_head *q = &sk->sk_error_queue;
> +	struct sk_buff *skb, *tmp;
> +	struct sk_buff_head list;
> +	int ret;
> +
> +	if (!(issue_flags & IO_URING_F_CQE32))
> +		return -EINVAL;
> +	ret = io_cmd_poll_multishot(cmd, issue_flags, EPOLLERR);
> +	if (unlikely(ret))
> +		return ret;
> +
> +	if (skb_queue_empty_lockless(q))
> +		return -EAGAIN;
> +	__skb_queue_head_init(&list);
> +
> +	scoped_guard(spinlock_irq, &q->lock) {
> +		skb_queue_walk_safe(q, skb, tmp) {
> +			/* don't support skbs with payload */
> +			if (!skb_has_tx_timestamp(skb, sk) || skb->len)
> +				continue;
> +			__skb_unlink(skb, q);
> +			__skb_queue_tail(&list, skb);
> +		}
> +	}
> +
> +	while (1) {
> +		skb = skb_peek(&list);
> +		if (!skb)
> +			break;
> +		if (!io_process_timestamp_skb(cmd, sk, skb, issue_flags))
> +			break;
> +		__skb_dequeue(&list);
> +		consume_skb(skb);
> +	}
> +
> +	if (!unlikely(skb_queue_empty(&list))) {
> +		scoped_guard(spinlock_irqsave, &q->lock)
> +			skb_queue_splice(q, &list);
> +	}
> +	return -EAGAIN;
> +}
> +
>  int io_uring_cmd_sock(struct io_uring_cmd *cmd, unsigned int issue_flags)
>  {
>  	struct socket *sock = cmd->file->private_data;
> @@ -76,6 +156,8 @@ int io_uring_cmd_sock(struct io_uring_cmd *cmd, unsigned int issue_flags)
>  		return io_uring_cmd_getsockopt(sock, cmd, issue_flags);
>  	case SOCKET_URING_OP_SETSOCKOPT:
>  		return io_uring_cmd_setsockopt(sock, cmd, issue_flags);
> +	case SOCKET_URING_OP_TX_TIMESTAMP:
> +		return io_uring_cmd_timestamp(sock, cmd, issue_flags);
>  	default:
>  		return -EOPNOTSUPP;
>  	}
> -- 
> 2.49.0
> 



  parent reply	other threads:[~2025-06-12 21:35 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-06-12  9:09 [PATCH v3 0/5] io_uring cmd for tx timestamps Pavel Begunkov
2025-06-12  9:09 ` [PATCH v3 1/5] net: timestamp: add helper returning skb's tx tstamp Pavel Begunkov
2025-06-12 21:20   ` Willem de Bruijn
2025-06-12  9:09 ` [PATCH v3 2/5] io_uring/poll: introduce io_arm_apoll() Pavel Begunkov
2025-06-12  9:09 ` [PATCH v3 3/5] io_uring/cmd: allow multishot polled commands Pavel Begunkov
2025-06-12  9:09 ` [PATCH v3 4/5] io_uring: add mshot helper for posting CQE32 Pavel Begunkov
2025-06-12  9:09 ` [PATCH v3 5/5] io_uring/netcmd: add tx timestamping cmd support Pavel Begunkov
2025-06-12 14:12   ` Jens Axboe
2025-06-12 14:26     ` Pavel Begunkov
2025-06-12 14:31       ` Jens Axboe
2025-06-12 15:01         ` Pavel Begunkov
2025-06-12 21:35   ` Willem de Bruijn [this message]
2025-06-13 18:29     ` Pavel Begunkov
2025-06-12  9:15 ` [PATCH v3 0/5] io_uring cmd for tx timestamps Pavel Begunkov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=684b482a1a03f_cb2792944c@willemb.c.googlers.com.notmuch \
    --to=willemdebruijn.kernel@gmail.com \
    --cc=asml.silence@gmail.com \
    --cc=davem@davemloft.net \
    --cc=edumazet@google.com \
    --cc=io-uring@vger.kernel.org \
    --cc=kerneljasonxing@gmail.com \
    --cc=kuba@kernel.org \
    --cc=kuniyu@amazon.com \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=richardcochran@gmail.com \
    --cc=sdf@fomichev.me \
    --cc=vadim.fedorenko@linux.dev \
    --cc=willemb@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox