* [PATCH liburing v3 0/5] Add basic test for nvme uring passthrough commands
[not found] <CGME20220728093902epcas5p40813f72b828e68e192f98819d29b2863@epcas5p4.samsung.com>
@ 2022-07-28 9:33 ` Ankit Kumar
[not found] ` <CGME20220728093904epcas5p4eed789c1eda441c223795dd026dc5d3f@epcas5p4.samsung.com>
` (6 more replies)
0 siblings, 7 replies; 10+ messages in thread
From: Ankit Kumar @ 2022-07-28 9:33 UTC (permalink / raw)
To: axboe; +Cc: io-uring, joshi.k, Ankit Kumar
This patchset adds a way to test NVMe uring passthrough commands with
nvme-ns character device. The uring passthrough was introduced with 5.19
io_uring.
To send nvme uring passthrough commands we require helpers to fetch NVMe
char device (/dev/ngXnY) specific fields such as namespace id, lba size etc.
How to run:
./test/io_uring_passthrough.t /dev/ng0n1
It requires argument to be NVMe device, if not the test will be skipped.
The test covers write/read with verify for sqthread poll, vectored / nonvectored
and fixed IO buffers, which can be extended in future. As of now iopoll is not
supported for passthrough commands, there is a test for such case.
Changes from v2 to v3
- Skip test if argument is not nvme device and remove prints, as
suggested by Jens.
- change nvme helper function name, as pointed by Jens.
- Remove wrong comment about command size, as per Kanchan's review
Ankit Kumar (5):
configure: check for nvme uring command support
io_uring.h: sync sqe entry with 5.20 io_uring
nvme: add nvme opcodes, structures and helper functions
test: add io_uring passthrough test
test/io_uring_passthrough: add test case for poll IO
configure | 20 ++
src/include/liburing/io_uring.h | 17 +-
test/Makefile | 1 +
test/io_uring_passthrough.c | 390 ++++++++++++++++++++++++++++++++
test/nvme.h | 168 ++++++++++++++
5 files changed, 594 insertions(+), 2 deletions(-)
create mode 100644 test/io_uring_passthrough.c
create mode 100644 test/nvme.h
--
2.17.1
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH liburing v3 1/5] configure: check for nvme uring command support
[not found] ` <CGME20220728093904epcas5p4eed789c1eda441c223795dd026dc5d3f@epcas5p4.samsung.com>
@ 2022-07-28 9:33 ` Ankit Kumar
0 siblings, 0 replies; 10+ messages in thread
From: Ankit Kumar @ 2022-07-28 9:33 UTC (permalink / raw)
To: axboe; +Cc: io-uring, joshi.k, Ankit Kumar
Modify configure to check availability of nvme_uring_cmd.
The follow up patch will have uring passthrough tests.
Signed-off-by: Ankit Kumar <[email protected]>
---
configure | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/configure b/configure
index 43071dd..1b0cc50 100755
--- a/configure
+++ b/configure
@@ -367,6 +367,23 @@ if compile_prog "" "" "has_ucontext"; then
fi
print_config "has_ucontext" "$has_ucontext"
+##########################################
+# Check NVME_URING_CMD support
+nvme_uring_cmd="no"
+cat > $TMPC << EOF
+#include <linux/nvme_ioctl.h>
+int main(void)
+{
+ struct nvme_uring_cmd *cmd;
+
+ return sizeof(struct nvme_uring_cmd);
+}
+EOF
+if compile_prog "" "" "nvme uring cmd"; then
+ nvme_uring_cmd="yes"
+fi
+print_config "NVMe uring command support" "$nvme_uring_cmd"
+
#############################################################################
if test "$liburing_nolibc" = "yes"; then
output_sym "CONFIG_NOLIBC"
@@ -402,6 +419,9 @@ fi
if test "$array_bounds" = "yes"; then
output_sym "CONFIG_HAVE_ARRAY_BOUNDS"
fi
+if test "$nvme_uring_cmd" = "yes"; then
+ output_sym "CONFIG_HAVE_NVME_URING"
+fi
echo "CC=$cc" >> $config_host_mak
print_config "CC" "$cc"
--
2.17.1
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH liburing v3 2/5] io_uring.h: sync sqe entry with 5.20 io_uring
[not found] ` <CGME20220728093905epcas5p22963dd2dadb73bdabfaffc55cb2edef5@epcas5p2.samsung.com>
@ 2022-07-28 9:33 ` Ankit Kumar
0 siblings, 0 replies; 10+ messages in thread
From: Ankit Kumar @ 2022-07-28 9:33 UTC (permalink / raw)
To: axboe; +Cc: io-uring, joshi.k, Ankit Kumar
Add a few missing fields which was added for uring command
Signed-off-by: Ankit Kumar <[email protected]>
---
src/include/liburing/io_uring.h | 17 +++++++++++++++--
1 file changed, 15 insertions(+), 2 deletions(-)
diff --git a/src/include/liburing/io_uring.h b/src/include/liburing/io_uring.h
index 3953807..c923f5c 100644
--- a/src/include/liburing/io_uring.h
+++ b/src/include/liburing/io_uring.h
@@ -26,6 +26,10 @@ struct io_uring_sqe {
union {
__u64 off; /* offset into file */
__u64 addr2;
+ struct {
+ __u32 cmd_op;
+ __u32 __pad1;
+ };
};
union {
__u64 addr; /* pointer to buffer or iovecs */
@@ -69,8 +73,17 @@ struct io_uring_sqe {
__u16 addr_len;
};
};
- __u64 addr3;
- __u64 __pad2[1];
+ union {
+ struct {
+ __u64 addr3;
+ __u64 __pad2[1];
+ };
+ /*
+ * If the ring is initialized with IORING_SETUP_SQE128, then
+ * this field is used for 80 bytes of arbitrary command data
+ */
+ __u8 cmd[0];
+ };
};
/*
--
2.17.1
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH liburing v3 3/5] nvme: add nvme opcodes, structures and helper functions
[not found] ` <CGME20220728093906epcas5p3fb607ddd16054ca7e6cbf26b53306062@epcas5p3.samsung.com>
@ 2022-07-28 9:33 ` Ankit Kumar
0 siblings, 0 replies; 10+ messages in thread
From: Ankit Kumar @ 2022-07-28 9:33 UTC (permalink / raw)
To: axboe; +Cc: io-uring, joshi.k, Ankit Kumar
Add bare minimum structures and helper functions required for
io_uring passthrough commands. This will enable the follow up
patch to add tests for nvme-ns generic character device.
Signed-off-by: Ankit Kumar <[email protected]>
---
test/nvme.h | 168 ++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 168 insertions(+)
create mode 100644 test/nvme.h
diff --git a/test/nvme.h b/test/nvme.h
new file mode 100644
index 0000000..14dc338
--- /dev/null
+++ b/test/nvme.h
@@ -0,0 +1,168 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Description: Helpers for NVMe uring passthrough commands
+ */
+#ifndef LIBURING_NVME_H
+#define LIBURING_NVME_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <sys/ioctl.h>
+#include <linux/nvme_ioctl.h>
+
+/*
+ * If the uapi headers installed on the system lacks nvme uring command
+ * support, use the local version to prevent compilation issues.
+ */
+#ifndef CONFIG_HAVE_NVME_URING
+struct nvme_uring_cmd {
+ __u8 opcode;
+ __u8 flags;
+ __u16 rsvd1;
+ __u32 nsid;
+ __u32 cdw2;
+ __u32 cdw3;
+ __u64 metadata;
+ __u64 addr;
+ __u32 metadata_len;
+ __u32 data_len;
+ __u32 cdw10;
+ __u32 cdw11;
+ __u32 cdw12;
+ __u32 cdw13;
+ __u32 cdw14;
+ __u32 cdw15;
+ __u32 timeout_ms;
+ __u32 rsvd2;
+};
+
+#define NVME_URING_CMD_IO _IOWR('N', 0x80, struct nvme_uring_cmd)
+#define NVME_URING_CMD_IO_VEC _IOWR('N', 0x81, struct nvme_uring_cmd)
+#endif /* CONFIG_HAVE_NVME_URING */
+
+#define NVME_DEFAULT_IOCTL_TIMEOUT 0
+#define NVME_IDENTIFY_DATA_SIZE 4096
+#define NVME_IDENTIFY_CSI_SHIFT 24
+#define NVME_IDENTIFY_CNS_NS 0
+#define NVME_CSI_NVM 0
+
+enum nvme_admin_opcode {
+ nvme_admin_identify = 0x06,
+};
+
+enum nvme_io_opcode {
+ nvme_cmd_write = 0x01,
+ nvme_cmd_read = 0x02,
+};
+
+int nsid;
+__u32 lba_shift;
+
+struct nvme_lbaf {
+ __le16 ms;
+ __u8 ds;
+ __u8 rp;
+};
+
+struct nvme_id_ns {
+ __le64 nsze;
+ __le64 ncap;
+ __le64 nuse;
+ __u8 nsfeat;
+ __u8 nlbaf;
+ __u8 flbas;
+ __u8 mc;
+ __u8 dpc;
+ __u8 dps;
+ __u8 nmic;
+ __u8 rescap;
+ __u8 fpi;
+ __u8 dlfeat;
+ __le16 nawun;
+ __le16 nawupf;
+ __le16 nacwu;
+ __le16 nabsn;
+ __le16 nabo;
+ __le16 nabspf;
+ __le16 noiob;
+ __u8 nvmcap[16];
+ __le16 npwg;
+ __le16 npwa;
+ __le16 npdg;
+ __le16 npda;
+ __le16 nows;
+ __le16 mssrl;
+ __le32 mcl;
+ __u8 msrc;
+ __u8 rsvd81[11];
+ __le32 anagrpid;
+ __u8 rsvd96[3];
+ __u8 nsattr;
+ __le16 nvmsetid;
+ __le16 endgid;
+ __u8 nguid[16];
+ __u8 eui64[8];
+ struct nvme_lbaf lbaf[16];
+ __u8 rsvd192[192];
+ __u8 vs[3712];
+};
+
+static inline int ilog2(uint32_t i)
+{
+ int log = -1;
+
+ while (i) {
+ i >>= 1;
+ log++;
+ }
+ return log;
+}
+
+int nvme_get_info(const char *file)
+{
+ struct nvme_id_ns ns;
+ int fd, err;
+ __u32 lba_size;
+
+ fd = open(file, O_RDONLY);
+ if (fd < 0) {
+ perror("file open");
+ return -errno;
+ }
+
+ nsid = ioctl(fd, NVME_IOCTL_ID);
+ if (nsid < 0) {
+ close(fd);
+ return -errno;
+ }
+
+ struct nvme_passthru_cmd cmd = {
+ .opcode = nvme_admin_identify,
+ .nsid = nsid,
+ .addr = (__u64)(uintptr_t)&ns,
+ .data_len = NVME_IDENTIFY_DATA_SIZE,
+ .cdw10 = NVME_IDENTIFY_CNS_NS,
+ .cdw11 = NVME_CSI_NVM << NVME_IDENTIFY_CSI_SHIFT,
+ .timeout_ms = NVME_DEFAULT_IOCTL_TIMEOUT,
+ };
+
+ err = ioctl(fd, NVME_IOCTL_ADMIN_CMD, &cmd);
+ if (err) {
+ close(fd);
+ return err;
+ }
+
+ lba_size = 1 << ns.lbaf[(ns.flbas & 0x0f)].ds;
+ lba_shift = ilog2(lba_size);
+
+ close(fd);
+ return 0;
+}
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
--
2.17.1
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH liburing v3 4/5] test: add io_uring passthrough test
[not found] ` <CGME20220728093907epcas5p353489ffcc9bed6f3a5c64b4679ad11ee@epcas5p3.samsung.com>
@ 2022-07-28 9:33 ` Ankit Kumar
0 siblings, 0 replies; 10+ messages in thread
From: Ankit Kumar @ 2022-07-28 9:33 UTC (permalink / raw)
To: axboe; +Cc: io-uring, joshi.k, Ankit Kumar
Add a way to test uring passthrough commands, which was added
with 5.19 kernel. This requires nvme-ns character device (/dev/ngXnY)
as filename argument. It runs a combination of read/write tests with
sqthread poll, vectored and non-vectored commands, fixed I/O buffers.
Tested-by: Kanchan Joshi <[email protected]>
Signed-off-by: Ankit Kumar <[email protected]>
---
test/Makefile | 1 +
test/io_uring_passthrough.c | 314 ++++++++++++++++++++++++++++++++++++
2 files changed, 315 insertions(+)
create mode 100644 test/io_uring_passthrough.c
diff --git a/test/Makefile b/test/Makefile
index a36ddb3..418c11c 100644
--- a/test/Makefile
+++ b/test/Makefile
@@ -90,6 +90,7 @@ test_srcs := \
io-cancel.c \
iopoll.c \
io_uring_enter.c \
+ io_uring_passthrough.c \
io_uring_register.c \
io_uring_setup.c \
lfs-openat.c \
diff --git a/test/io_uring_passthrough.c b/test/io_uring_passthrough.c
new file mode 100644
index 0000000..03043b7
--- /dev/null
+++ b/test/io_uring_passthrough.c
@@ -0,0 +1,314 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Description: basic read/write tests for io_uring passthrough commands
+ */
+#include <errno.h>
+#include <stdio.h>
+#include <unistd.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include "helpers.h"
+#include "liburing.h"
+#include "nvme.h"
+
+#define FILE_SIZE (256 * 1024)
+#define BS 8192
+#define BUFFERS (FILE_SIZE / BS)
+
+static struct iovec *vecs;
+
+/*
+ * Each offset in the file has the ((test_case / 2) * FILE_SIZE)
+ * + (offset / sizeof(int)) stored for every
+ * sizeof(int) address.
+ */
+static int verify_buf(int tc, void *buf, off_t off)
+{
+ int i, u_in_buf = BS / sizeof(unsigned int);
+ unsigned int *ptr;
+
+ off /= sizeof(unsigned int);
+ off += (tc / 2) * FILE_SIZE;
+ ptr = buf;
+ for (i = 0; i < u_in_buf; i++) {
+ if (off != *ptr) {
+ fprintf(stderr, "Found %u, wanted %lu\n", *ptr, off);
+ return 1;
+ }
+ ptr++;
+ off++;
+ }
+
+ return 0;
+}
+
+static int fill_pattern(int tc)
+{
+ unsigned int val, *ptr;
+ int i, j;
+ int u_in_buf = BS / sizeof(val);
+
+ val = (tc / 2) * FILE_SIZE;
+ for (i = 0; i < BUFFERS; i++) {
+ ptr = vecs[i].iov_base;
+ for (j = 0; j < u_in_buf; j++) {
+ *ptr = val;
+ val++;
+ ptr++;
+ }
+ }
+
+ return 0;
+}
+
+static int __test_io(const char *file, struct io_uring *ring, int tc, int read,
+ int sqthread, int fixed, int nonvec)
+{
+ struct io_uring_sqe *sqe;
+ struct io_uring_cqe *cqe;
+ struct nvme_uring_cmd *cmd;
+ int open_flags;
+ int do_fixed;
+ int i, ret, fd = -1;
+ off_t offset;
+ __u64 slba;
+ __u32 nlb;
+
+#ifdef VERBOSE
+ fprintf(stdout, "%s: start %d/%d/%d/%d: ", __FUNCTION__, read,
+ sqthread, fixed,
+ nonvec);
+#endif
+ if (read)
+ open_flags = O_RDONLY;
+ else
+ open_flags = O_WRONLY;
+
+ if (fixed) {
+ ret = t_register_buffers(ring, vecs, BUFFERS);
+ if (ret == T_SETUP_SKIP)
+ return 0;
+ if (ret != T_SETUP_OK) {
+ fprintf(stderr, "buffer reg failed: %d\n", ret);
+ goto err;
+ }
+ }
+
+ fd = open(file, open_flags);
+ if (fd < 0) {
+ perror("file open");
+ goto err;
+ }
+
+ if (sqthread) {
+ ret = io_uring_register_files(ring, &fd, 1);
+ if (ret) {
+ fprintf(stderr, "file reg failed: %d\n", ret);
+ goto err;
+ }
+ }
+
+ if (!read)
+ fill_pattern(tc);
+
+ offset = 0;
+ for (i = 0; i < BUFFERS; i++) {
+ sqe = io_uring_get_sqe(ring);
+ if (!sqe) {
+ fprintf(stderr, "sqe get failed\n");
+ goto err;
+ }
+ if (read) {
+ int use_fd = fd;
+
+ do_fixed = fixed;
+
+ if (sqthread)
+ use_fd = 0;
+ if (fixed && (i & 1))
+ do_fixed = 0;
+ if (do_fixed) {
+ io_uring_prep_read_fixed(sqe, use_fd, vecs[i].iov_base,
+ vecs[i].iov_len,
+ offset, i);
+ sqe->cmd_op = NVME_URING_CMD_IO;
+ } else if (nonvec) {
+ io_uring_prep_read(sqe, use_fd, vecs[i].iov_base,
+ vecs[i].iov_len, offset);
+ sqe->cmd_op = NVME_URING_CMD_IO;
+ } else {
+ io_uring_prep_readv(sqe, use_fd, &vecs[i], 1,
+ offset);
+ sqe->cmd_op = NVME_URING_CMD_IO_VEC;
+ }
+ } else {
+ int use_fd = fd;
+
+ do_fixed = fixed;
+
+ if (sqthread)
+ use_fd = 0;
+ if (fixed && (i & 1))
+ do_fixed = 0;
+ if (do_fixed) {
+ io_uring_prep_write_fixed(sqe, use_fd, vecs[i].iov_base,
+ vecs[i].iov_len,
+ offset, i);
+ sqe->cmd_op = NVME_URING_CMD_IO;
+ } else if (nonvec) {
+ io_uring_prep_write(sqe, use_fd, vecs[i].iov_base,
+ vecs[i].iov_len, offset);
+ sqe->cmd_op = NVME_URING_CMD_IO;
+ } else {
+ io_uring_prep_writev(sqe, use_fd, &vecs[i], 1,
+ offset);
+ sqe->cmd_op = NVME_URING_CMD_IO_VEC;
+ }
+ }
+ sqe->opcode = IORING_OP_URING_CMD;
+ sqe->user_data = ((uint64_t)offset << 32) | i;
+ if (sqthread)
+ sqe->flags |= IOSQE_FIXED_FILE;
+
+ cmd = (struct nvme_uring_cmd *)sqe->cmd;
+ memset(cmd, 0, sizeof(struct nvme_uring_cmd));
+
+ cmd->opcode = read ? nvme_cmd_read : nvme_cmd_write;
+
+ slba = offset >> lba_shift;
+ nlb = (BS >> lba_shift) - 1;
+
+ /* cdw10 and cdw11 represent starting lba */
+ cmd->cdw10 = slba & 0xffffffff;
+ cmd->cdw11 = slba >> 32;
+ /* cdw12 represent number of lba's for read/write */
+ cmd->cdw12 = nlb;
+ if (do_fixed || nonvec) {
+ cmd->addr = (__u64)(uintptr_t)vecs[i].iov_base;
+ cmd->data_len = vecs[i].iov_len;
+ } else {
+ cmd->addr = (__u64)(uintptr_t)&vecs[i];
+ cmd->data_len = 1;
+ }
+ cmd->nsid = nsid;
+
+ offset += BS;
+ }
+
+ ret = io_uring_submit(ring);
+ if (ret != BUFFERS) {
+ fprintf(stderr, "submit got %d, wanted %d\n", ret, BUFFERS);
+ goto err;
+ }
+
+ for (i = 0; i < BUFFERS; i++) {
+ ret = io_uring_wait_cqe(ring, &cqe);
+ if (ret) {
+ fprintf(stderr, "wait_cqe=%d\n", ret);
+ goto err;
+ }
+ if (cqe->res != 0) {
+ fprintf(stderr, "cqe res %d, wanted 0\n", cqe->res);
+ goto err;
+ }
+ io_uring_cqe_seen(ring, cqe);
+ if (read) {
+ int index = cqe->user_data & 0xffffffff;
+ void *buf = vecs[index].iov_base;
+ off_t voff = cqe->user_data >> 32;
+
+ if (verify_buf(tc, buf, voff))
+ goto err;
+ }
+ }
+
+ if (fixed) {
+ ret = io_uring_unregister_buffers(ring);
+ if (ret) {
+ fprintf(stderr, "buffer unreg failed: %d\n", ret);
+ goto err;
+ }
+ }
+ if (sqthread) {
+ ret = io_uring_unregister_files(ring);
+ if (ret) {
+ fprintf(stderr, "file unreg failed: %d\n", ret);
+ goto err;
+ }
+ }
+
+ close(fd);
+#ifdef VERBOSE
+ fprintf(stdout, "PASS\n");
+#endif
+ return 0;
+err:
+#ifdef VERBOSE
+ fprintf(stderr, "FAILED\n");
+#endif
+ if (fd != -1)
+ close(fd);
+ return 1;
+}
+
+static int test_io(const char *file, int tc, int read, int sqthread,
+ int fixed, int nonvec)
+{
+ struct io_uring ring;
+ int ret, ring_flags = 0;
+
+ ring_flags |= IORING_SETUP_SQE128;
+ ring_flags |= IORING_SETUP_CQE32;
+
+ if (sqthread)
+ ring_flags |= IORING_SETUP_SQPOLL;
+
+ ret = t_create_ring(64, &ring, ring_flags);
+ if (ret == T_SETUP_SKIP)
+ return 0;
+ if (ret != T_SETUP_OK) {
+ fprintf(stderr, "ring create failed: %d\n", ret);
+ return 1;
+ }
+
+ ret = __test_io(file, &ring, tc, read, sqthread, fixed, nonvec);
+ io_uring_queue_exit(&ring);
+
+ return ret;
+}
+
+int main(int argc, char *argv[])
+{
+ int i, ret;
+ char *fname;
+
+ if (argc < 2)
+ return T_EXIT_SKIP;
+
+ fname = argv[1];
+ ret = nvme_get_info(fname);
+
+ if (ret)
+ return T_EXIT_SKIP;
+
+ vecs = t_create_buffers(BUFFERS, BS);
+
+ for (i = 0; i < 16; i++) {
+ int read = (i & 1) != 0;
+ int sqthread = (i & 2) != 0;
+ int fixed = (i & 4) != 0;
+ int nonvec = (i & 8) != 0;
+
+ ret = test_io(fname, i, read, sqthread, fixed, nonvec);
+ if (ret) {
+ fprintf(stderr, "test_io failed %d/%d/%d/%d\n",
+ read, sqthread, fixed, nonvec);
+ goto err;
+ }
+ }
+
+ return T_EXIT_PASS;
+err:
+ return T_EXIT_FAIL;
+}
--
2.17.1
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH liburing v3 5/5] test/io_uring_passthrough: add test case for poll IO
[not found] ` <CGME20220728093908epcas5p4d6318b0f037f0ccf9e5422b6b63f217c@epcas5p4.samsung.com>
@ 2022-07-28 9:33 ` Ankit Kumar
0 siblings, 0 replies; 10+ messages in thread
From: Ankit Kumar @ 2022-07-28 9:33 UTC (permalink / raw)
To: axboe; +Cc: io-uring, joshi.k, Ankit Kumar
For uring passthrough add test case for poll IO completion.
If poll IO is not supported return success.
Tested-by: Kanchan Joshi <[email protected]>
Signed-off-by: Ankit Kumar <[email protected]>
---
test/io_uring_passthrough.c | 76 +++++++++++++++++++++++++++++++++++++
1 file changed, 76 insertions(+)
diff --git a/test/io_uring_passthrough.c b/test/io_uring_passthrough.c
index 03043b7..d4a2169 100644
--- a/test/io_uring_passthrough.c
+++ b/test/io_uring_passthrough.c
@@ -10,6 +10,7 @@
#include "helpers.h"
#include "liburing.h"
+#include "../src/syscall.h"
#include "nvme.h"
#define FILE_SIZE (256 * 1024)
@@ -278,6 +279,75 @@ static int test_io(const char *file, int tc, int read, int sqthread,
return ret;
}
+extern int __io_uring_flush_sq(struct io_uring *ring);
+
+/*
+ * if we are polling io_uring_submit needs to always enter the
+ * kernel to fetch events
+ */
+static int test_io_uring_submit_enters(const char *file)
+{
+ struct io_uring ring;
+ int fd, i, ret, ring_flags, open_flags;
+ unsigned head;
+ struct io_uring_cqe *cqe;
+
+ ring_flags = IORING_SETUP_IOPOLL;
+ ring_flags |= IORING_SETUP_SQE128;
+ ring_flags |= IORING_SETUP_CQE32;
+
+ ret = io_uring_queue_init(64, &ring, ring_flags);
+ if (ret) {
+ fprintf(stderr, "ring create failed: %d\n", ret);
+ return 1;
+ }
+
+ open_flags = O_WRONLY;
+ fd = open(file, open_flags);
+ if (fd < 0) {
+ perror("file open");
+ goto err;
+ }
+
+ for (i = 0; i < BUFFERS; i++) {
+ struct io_uring_sqe *sqe;
+ off_t offset = BS * (rand() % BUFFERS);
+
+ sqe = io_uring_get_sqe(&ring);
+ io_uring_prep_writev(sqe, fd, &vecs[i], 1, offset);
+ sqe->user_data = 1;
+ }
+
+ /* submit manually to avoid adding IORING_ENTER_GETEVENTS */
+ ret = __sys_io_uring_enter(ring.ring_fd, __io_uring_flush_sq(&ring), 0,
+ 0, NULL);
+ if (ret < 0)
+ goto err;
+
+ for (i = 0; i < 500; i++) {
+ ret = io_uring_submit(&ring);
+ if (ret != 0) {
+ fprintf(stderr, "still had %d sqes to submit, this is unexpected", ret);
+ goto err;
+ }
+
+ io_uring_for_each_cqe(&ring, head, cqe) {
+ if (cqe->res == -EOPNOTSUPP)
+ fprintf(stdout, "Device doesn't support polled IO\n");
+ goto ok;
+ }
+ usleep(10000);
+ }
+err:
+ ret = 1;
+ if (fd != -1)
+ close(fd);
+
+ok:
+ io_uring_queue_exit(&ring);
+ return ret;
+}
+
int main(int argc, char *argv[])
{
int i, ret;
@@ -308,6 +378,12 @@ int main(int argc, char *argv[])
}
}
+ ret = test_io_uring_submit_enters(fname);
+ if (ret) {
+ fprintf(stderr, "test_io_uring_submit_enters failed\n");
+ goto err;
+ }
+
return T_EXIT_PASS;
err:
return T_EXIT_FAIL;
--
2.17.1
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH liburing v3 0/5] Add basic test for nvme uring passthrough commands
2022-07-28 9:33 ` [PATCH liburing v3 0/5] Add basic test for nvme uring passthrough commands Ankit Kumar
` (4 preceding siblings ...)
[not found] ` <CGME20220728093908epcas5p4d6318b0f037f0ccf9e5422b6b63f217c@epcas5p4.samsung.com>
@ 2022-07-28 12:36 ` Jens Axboe
2022-07-28 13:01 ` Ankit Kumar
2022-07-28 13:10 ` Jens Axboe
6 siblings, 1 reply; 10+ messages in thread
From: Jens Axboe @ 2022-07-28 12:36 UTC (permalink / raw)
To: Ankit Kumar; +Cc: io-uring, joshi.k
On 7/28/22 3:33 AM, Ankit Kumar wrote:
> This patchset adds a way to test NVMe uring passthrough commands with
> nvme-ns character device. The uring passthrough was introduced with 5.19
> io_uring.
>
> To send nvme uring passthrough commands we require helpers to fetch NVMe
> char device (/dev/ngXnY) specific fields such as namespace id, lba size etc.
>
> How to run:
> ./test/io_uring_passthrough.t /dev/ng0n1
>
> It requires argument to be NVMe device, if not the test will be skipped.
>
> The test covers write/read with verify for sqthread poll, vectored / nonvectored
> and fixed IO buffers, which can be extended in future. As of now iopoll is not
> supported for passthrough commands, there is a test for such case.
>
> Changes from v2 to v3
> - Skip test if argument is not nvme device and remove prints, as
> suggested by Jens.
> - change nvme helper function name, as pointed by Jens.
> - Remove wrong comment about command size, as per Kanchan's review
I didn't get patch 2/5, and lore didn't either. Can you resend the series?
--
Jens Axboe
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH liburing v3 0/5] Add basic test for nvme uring passthrough commands
2022-07-28 12:36 ` [PATCH liburing v3 0/5] Add basic test for nvme uring passthrough commands Jens Axboe
@ 2022-07-28 13:01 ` Ankit Kumar
2022-07-28 13:10 ` Jens Axboe
0 siblings, 1 reply; 10+ messages in thread
From: Ankit Kumar @ 2022-07-28 13:01 UTC (permalink / raw)
To: Jens Axboe; +Cc: io-uring, joshi.k
[-- Attachment #1: Type: text/plain, Size: 1357 bytes --]
On Thu, Jul 28, 2022 at 06:36:51AM -0600, Jens Axboe wrote:
> On 7/28/22 3:33 AM, Ankit Kumar wrote:
> > This patchset adds a way to test NVMe uring passthrough commands with
> > nvme-ns character device. The uring passthrough was introduced with 5.19
> > io_uring.
> >
> > To send nvme uring passthrough commands we require helpers to fetch NVMe
> > char device (/dev/ngXnY) specific fields such as namespace id, lba size etc.
> >
> > How to run:
> > ./test/io_uring_passthrough.t /dev/ng0n1
> >
> > It requires argument to be NVMe device, if not the test will be skipped.
> >
> > The test covers write/read with verify for sqthread poll, vectored / nonvectored
> > and fixed IO buffers, which can be extended in future. As of now iopoll is not
> > supported for passthrough commands, there is a test for such case.
> >
> > Changes from v2 to v3
> > - Skip test if argument is not nvme device and remove prints, as
> > suggested by Jens.
> > - change nvme helper function name, as pointed by Jens.
> > - Remove wrong comment about command size, as per Kanchan's review
>
> I didn't get patch 2/5, and lore didn't either. Can you resend the series?
>
> --
> Jens Axboe
>
>
Sorry, issue from my side it. You should have 2/5 now and I see its there in
lore as well. Hope its sufficient and doesn't require me to resend the entire series
again.
[-- Attachment #2: Type: text/plain, Size: 0 bytes --]
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH liburing v3 0/5] Add basic test for nvme uring passthrough commands
2022-07-28 13:01 ` Ankit Kumar
@ 2022-07-28 13:10 ` Jens Axboe
0 siblings, 0 replies; 10+ messages in thread
From: Jens Axboe @ 2022-07-28 13:10 UTC (permalink / raw)
To: Ankit Kumar; +Cc: io-uring, joshi.k
On 7/28/22 7:01 AM, Ankit Kumar wrote:
> On Thu, Jul 28, 2022 at 06:36:51AM -0600, Jens Axboe wrote:
>> On 7/28/22 3:33 AM, Ankit Kumar wrote:
>>> This patchset adds a way to test NVMe uring passthrough commands with
>>> nvme-ns character device. The uring passthrough was introduced with 5.19
>>> io_uring.
>>>
>>> To send nvme uring passthrough commands we require helpers to fetch NVMe
>>> char device (/dev/ngXnY) specific fields such as namespace id, lba size etc.
>>>
>>> How to run:
>>> ./test/io_uring_passthrough.t /dev/ng0n1
>>>
>>> It requires argument to be NVMe device, if not the test will be skipped.
>>>
>>> The test covers write/read with verify for sqthread poll, vectored / nonvectored
>>> and fixed IO buffers, which can be extended in future. As of now iopoll is not
>>> supported for passthrough commands, there is a test for such case.
>>>
>>> Changes from v2 to v3
>>> - Skip test if argument is not nvme device and remove prints, as
>>> suggested by Jens.
>>> - change nvme helper function name, as pointed by Jens.
>>> - Remove wrong comment about command size, as per Kanchan's review
>>
>> I didn't get patch 2/5, and lore didn't either. Can you resend the series?
>>
>> --
>> Jens Axboe
>>
>>
> Sorry, issue from my side it. You should have 2/5 now and I see its
> there in lore as well. Hope its sufficient and doesn't require me to
> resend the entire series again.
Yep I got it, and I've now applied it. Did a few cleanups on top, but
nothing major. Thanks!
--
Jens Axboe
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH liburing v3 0/5] Add basic test for nvme uring passthrough commands
2022-07-28 9:33 ` [PATCH liburing v3 0/5] Add basic test for nvme uring passthrough commands Ankit Kumar
` (5 preceding siblings ...)
2022-07-28 12:36 ` [PATCH liburing v3 0/5] Add basic test for nvme uring passthrough commands Jens Axboe
@ 2022-07-28 13:10 ` Jens Axboe
6 siblings, 0 replies; 10+ messages in thread
From: Jens Axboe @ 2022-07-28 13:10 UTC (permalink / raw)
To: ankit.kumar; +Cc: io-uring, joshi.k
On Thu, 28 Jul 2022 15:03:22 +0530, Ankit Kumar wrote:
> This patchset adds a way to test NVMe uring passthrough commands with
> nvme-ns character device. The uring passthrough was introduced with 5.19
> io_uring.
>
> To send nvme uring passthrough commands we require helpers to fetch NVMe
> char device (/dev/ngXnY) specific fields such as namespace id, lba size etc.
>
> [...]
Applied, thanks!
[1/5] configure: check for nvme uring command support
commit: 7fc6c1e89f1b83f2bb80a974a40126d10ab95d46
[2/5] io_uring.h: sync sqe entry with 5.20 io_uring
commit: 893b9d13b7571eb99d124c0804c48e331b4dbe3b
[3/5] nvme: add nvme opcodes, structures and helper functions
commit: 612101cc61063eed06d5bd232b1ab7a43732f227
[4/5] test: add io_uring passthrough test
commit: b593422fd0d624b6d1a59d0cc5a674dfdf22db6e
[5/5] test/io_uring_passthrough: add test case for poll IO
commit: ba10a0e0b3039aab43352f08631845f25aa2b225
Best regards,
--
Jens Axboe
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2022-07-28 13:10 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <CGME20220728093902epcas5p40813f72b828e68e192f98819d29b2863@epcas5p4.samsung.com>
2022-07-28 9:33 ` [PATCH liburing v3 0/5] Add basic test for nvme uring passthrough commands Ankit Kumar
[not found] ` <CGME20220728093904epcas5p4eed789c1eda441c223795dd026dc5d3f@epcas5p4.samsung.com>
2022-07-28 9:33 ` [PATCH liburing v3 1/5] configure: check for nvme uring command support Ankit Kumar
[not found] ` <CGME20220728093905epcas5p22963dd2dadb73bdabfaffc55cb2edef5@epcas5p2.samsung.com>
2022-07-28 9:33 ` [PATCH liburing v3 2/5] io_uring.h: sync sqe entry with 5.20 io_uring Ankit Kumar
[not found] ` <CGME20220728093906epcas5p3fb607ddd16054ca7e6cbf26b53306062@epcas5p3.samsung.com>
2022-07-28 9:33 ` [PATCH liburing v3 3/5] nvme: add nvme opcodes, structures and helper functions Ankit Kumar
[not found] ` <CGME20220728093907epcas5p353489ffcc9bed6f3a5c64b4679ad11ee@epcas5p3.samsung.com>
2022-07-28 9:33 ` [PATCH liburing v3 4/5] test: add io_uring passthrough test Ankit Kumar
[not found] ` <CGME20220728093908epcas5p4d6318b0f037f0ccf9e5422b6b63f217c@epcas5p4.samsung.com>
2022-07-28 9:33 ` [PATCH liburing v3 5/5] test/io_uring_passthrough: add test case for poll IO Ankit Kumar
2022-07-28 12:36 ` [PATCH liburing v3 0/5] Add basic test for nvme uring passthrough commands Jens Axboe
2022-07-28 13:01 ` Ankit Kumar
2022-07-28 13:10 ` Jens Axboe
2022-07-28 13:10 ` Jens Axboe
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox