Hi,
Found a error while  run fio nvme_pi testing,  please have a look.

reproducer
./fio/t/nvmept_pi.py --dut /dev/ng0n1 

kernel info :
    Merge branch 'for-6.8/block' into for-next
    
    * for-6.8/block:
      null_blk: Remove usage of the deprecated ida_simple_xx() API

kernel parameter 
sysctl.kernel.io_uring_disabled=0
error info

[ 1083.373005] INFO: start run cmd ./fio/t/nvmept_pi.py --dut /dev/ng0n1 [ 1093.216195] DMAR: DRHD: handling fault status reg 2 [ 1093.216995] DMAR: [DMA Read NO_PASID] Request device [05:00.0] fault addr 0x5000 [fault reason 0x06] PTE Read access is not set [ 1094.372929] DMAR: DRHD: handling fault status reg 102 [ 1094.373313] DMAR: [DMA Read NO_PASID] Request device [05:00.0] fault addr 0x556f7ea68f000 [fault reason 0x04] Access beyond MGAW [ 1095.109239] DMAR: DRHD: handling fault status reg 202 [ 1095.109638] DMAR: [DMA Write NO_PASID] Request device [05:00.0] fault addr 0x5000 [fault reason 0x05] PTE Write access is not set [ 1095.110699] DMAR: [DMA Write NO_PASID] Request device [05:00.0] fault addr 0x0 [fault reason 0x05] PTE Write access is not set [ 1095.111711] DMAR: [DMA Write NO_PASID] Request device [05:00.0] fault addr 0x0 [fault reason 0x05] PTE Write access is not set [ 1095.581436] nvme nvme0: request 0x100 genctr mismatch (got 0xc expected 0xd) [ 1095.582370] nvme nvme0: invalid id 49408 completed on queue 4 [ 1095.583057] nvme nvme0: request 0x100 genctr mismatch (got 0xc expected 0xd) [ 1095.583809] nvme nvme0: invalid id 49408 completed on queue 4
....
1150.335678] nvme nvme0: invalid id 49408 completed on queue 4 [ 1150.336461] [ 1150.337082] nvme nvme0: request 0x100 genctr mismatch (got 0xc expected 0xd) [ 1150.337820] CPU: 0 PID: 151 Comm: kworker/0:8 Kdump: loaded Not tainted 6.7.0+ #1 [ 1150.338479] nvme nvme0: invalid id 49408 completed on queue 4 [ 1150.339298] Hardware name: HP ProLiant DL360 Gen9/ProLiant DL360 Gen9, BIOS P89 05/17/2022 [ 1150.340107] nvme nvme0: request 0x100 genctr mismatch (got 0xc expected 0xd) [ 1150.341092] Workqueue: events netstamp_clear [ 1150.341550] nvme nvme0: invalid id 49408 completed on queue 4 [ 1150.342298] [ 1150.343008] nvme nvme0: request 0x100 genctr mismatch (got 0xc expected 0xd) [ 1150.343762] RIP: 0010:smp_call_function_many_cond+0x110/0x4c0 [ 1150.344359] nvme nvme0: invalid id 49408 completed on queue 4 [ 1150.345114] Code: 48 8b 7b 08 48 63 d [ 1150.346531] RSP: 0018:ffffb8f8807e7d68 EFLAGS: 00000202 [ 1150.696111] nvme nvme0: invalid id 49408 completed on queue 4 [ 1150.847384] [ 1150.847675] nvme nvme0: request 0x100 genctr mismatch (got 0xc expected 0xd) [ 1150.848350] RAX: 0000000000000003 RBX: ffff9f92ffa349c0 RCX: 0000000000000001 [ 1150.848819] nvme nvme0: invalid id 49408 completed on queue 4 [ 1150.849558] RDX: 0000000000000001 RSI: ffff9f92ffaf91e0 RDI: ffff9f8f80138a40 [ 1150.850303] nvme nvme0: request 0x100 genctr mismatch (got 0xc expected 0xd) [ 1150.850991] RBP: 0000000000000246 R08: 0000000000000003 R09: 000000000000000c [ 1150.851782] nvme nvme0: invalid id 49408 completed on queue 4 [ 1150.852525] R10: ffff9f8f80138170 R11: 0000000000000000 R12: 0000000000000000 [ 1150.853265] nvme nvme0: request 0x100 genctr mismatch (got 0xc expected 0xd) [ 1150.853949] R13: 0000000000000001 R14: ffff9f92ffa349c0 R15: 000000000000000b [ 1150.854739] nvme nvme0: invalid id 49408 completed on queue 4 [ 1150.855476] FS: 0000000000000000(0000) GS:ffff9f92ffa00000(0000) knlGS:0000000000000000 [ 1150.856226] nvme nvme0: requesR2: 00005623bf60a5a0 CR3: 000000047ca20003 CR4: 00000000001706f0 [ 1151.153115] nvme nvme0: request 0x100 genctr mismatch (got 0xc expected 0xd) [ 1151.314325] Call Trace: [ 1151.358440] nvme nvme0: invalid id 49408 completed on queue 4 [ 1151.359177] <IRQ> [ 1151.359308] nvme nvme0: request 0x100 genctr mismatch (got 0xc expected 0xd) [ 1151.360006] ? watchdog_timer_fn+0x1ec/0x270 [ 1151.360485] nvme nvme0: invalid id 49408 completed on queue 4 [ 1151.361252] ? __pfx_watchdog_timer_fn+0x10/0x10 [ 1151.361886] nvme nvme0: request 0x100 genctr mismatch (got 0xc expected 0xd) [ 1151.362565] ? __hrtimer_run_queues+0x112/0x2b0 [ 1151.363170] nvme nvme0: invalid id 49408 completed on queue 4 [ 1151.363950] ? hrtimer_interrupt+0xfc/0x230 [ 1151.364588] nvme nvme0: request 0x100 genctr mismatch (got 0xc expected 0xd) [ 1151.365287] ? __sysvec_apic_timer_interrupt+0x4e/0x140 [ 1151.365498] nvme nvme0: invalid id 49408 completed on queue 4 [ 1151.366262] ? sysvec_apic_timer_interrupt+0x6d/0x90 [ 1151.366544] nvme nvme0: request 0x100 genctr mismatch (got st 0x100 genctr mismatch (got 0xc expected 0xd) [ 1151.717102] ? asm_sysvec_apic_timer_interrupt+0x16/0x20 [ 1151.797740] nvme nvme0: invalid id 49408 completed on queue 4 [ 1151.868693] ? smp_call_function_many_cond+0x110/0x4c0 [ 1151.869023] nvme nvme0: request 0x100 genctr mismatch (got 0xc expected 0xd) [ 1151.869722] ? smp_call_function_many_cond+0xf2/0x4c0 [ 1151.870001] nvme nvme0: invalid id 49408 completed on queue 4 [ 1151.870758] ? __pfx_do_sync_core+0x10/0x10 [ 1151.871038] nvme nvme0: request 0x100 genctr mismatch (got 0xc expected 0xd) [ 1151.871753] on_each_cpu_cond_mask+0x20/0x40 [ 1151.871967] nvme nvme0: invalid id 49408 completed on queue 4 [ 1151.872747] text_poke_bp_batch+0xbf/0x2f0 [ 1151.873373] nvme nvme0: request 0x100 genctr mismatch (got 0xc expected 0xd) [ 1151.874067] ? arch_jump_label_transform_queue+0x55/0x80 [ 1151.874313] nvme nvme0: invalid id 49408 completed on queue 4 [ 1151.875067] text_poke_finish+0x1b/0x30 [ 1151.875389] nvme nvme0: request 0x100 genctr mismatch (got 0xc expected 0xd) [ 1151.876072] arch_jump_label_transform_apply+0x16/0x30 [ 1152.092399] static_key_enable+0x16/0x20 [ 1152.226851] nvme nvme0: invalid id 49408 completed on queue 4 [ 1152.378017] process_one_work+0x177/0x340 [ 1152.378232] nvme nvme0: request 0x100 genctr mismatch (got 0xc expected 0xd) [ 1152.378912] worker_thread+0x27e/0x390 [ 1152.379131] nvme nvme0: invalid id 49408 completed on queue 4 [ 1152.379960] ? __pfx_worker_thread+0x10/0x10 [ 1152.380179] nvme nvme0: request 0x100 genctr mismatch (got 0xc expected 0xd) [ 1152.380883] kthread+0xe2/0x110 [ 1152.381510] nvme nvme0: invalid id 49408 completed on queue 4 [ 1152.382295] ? __pfx_kthread+0x10/0x10 [ 1152.382887] nvme nvme0: request 0x100 genctr mismatch (got 0xc expected 0xd) [ 1152.383561] ret_from_fork+0x30/0x50 [ 1152.383781] nvme nvme0: invalid id 49408 completed on queue 4 [ 1152.384527] ? __pfx_kthread+0x10/0x10 [ 1152.384745] nvme nvme0: request 0x100 genctr mismatch (got 0xc expected 0xd) [ 1152.385420] ret_from_fork_asm+0x1b/0x30 [ 1152.385645] nvme nvme0: invalid id 49408 completed on queue 4 [ 1152.386401] </TASK> [ 1152.386610] nvme nvme0: request 0x100 genctr mismatch (got 0xc expected 0xd) [ 1152.441399]leted on queue 4



--
Guangwu Zhang
Thanks