From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 458D2C433F5 for ; Fri, 1 Apr 2022 02:44:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244293AbiDACqN (ORCPT ); Thu, 31 Mar 2022 22:46:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34908 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244194AbiDACqM (ORCPT ); Thu, 31 Mar 2022 22:46:12 -0400 Received: from mail-pj1-x102c.google.com (mail-pj1-x102c.google.com [IPv6:2607:f8b0:4864:20::102c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3160C15AAC6 for ; Thu, 31 Mar 2022 19:44:06 -0700 (PDT) Received: by mail-pj1-x102c.google.com with SMTP id bx5so1185570pjb.3 for ; Thu, 31 Mar 2022 19:44:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20210112.gappssmtp.com; s=20210112; h=message-id:date:mime-version:user-agent:subject:content-language:to :cc:references:from:in-reply-to:content-transfer-encoding; bh=HWtyih8igr9N3NwexuJxuDSMFSnT6hvHzlu33VdYUJk=; b=CrRSqUR2tTuhZv2izdc3Hwy1yVmoJale25fz+zC/RZ/47NWGIH0+nPvfkWE+2d8q8Q RNtrm4qNKYkIHBJpoj9TNbHjtECePEn2l09fu683uFurKZvIloqxiewaQEykEctwvOzU hlNKZiD5e0OhKMauNuSbUOlw2D0wWLTAtJkCn8fLH4dpoxKoqvk1+L77VL73gCxhKjX4 1rBIIFaMufDL3LIsz/YRl1AxZCywLohlBW0dUNMgPjUXJGRs7m+Pb7eqybOi/FmNKui2 tabf/Ll4ewQqDGYo8LXBtZ3UXJWgtAJE+X/538GpyNSAt8uJBdgB0Po0EzmESHFsnaeG KLiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:cc:references:from:in-reply-to :content-transfer-encoding; bh=HWtyih8igr9N3NwexuJxuDSMFSnT6hvHzlu33VdYUJk=; b=Bm0CCs7nKCfRt9OD9OP1ljJ1cn9Wj/93/V9Xo+E5E5hzLLHCnNSKzedxdoV0fumsYT 18M9HfmDqzmo2PNGKZH4KCbix9L7TPL0Q/8QBIHbMcKo0/FXFvFxfgGL1eudb6Tf8jBB mCm18zMlFnbNzZ7iHfHoLorwapUqpBWzCvlZymWQdobwsIX54NG6RxtDeNU9CdPFKoch m8C6ClPucHFqk2BTA2qQ434/ujAhqgWpSbHnU1RqtfJZ6D+JWlrbeFFkHk1yUoX5zEvK 6WT3RGvTMIzYPq+j34GsXKNzmJ9MgtVsTC6/K7Am5+jsmItoPUJwBXrqLaagliJbZIVN hKMg== X-Gm-Message-State: AOAM5333EoU0s1RE9Ou+ZiYoQTPoWJHiWjWVy4gzBnjmFoYhcBa1tSUN WZuOU0ri01LneCGpboDFCTDomQ== X-Google-Smtp-Source: ABdhPJw6EzVnlWTDGXWHyAme/hHYmcfZjrGv8JJhxPhRcvIW5tOYienAotqz9breF+R4C2Grr37duA== X-Received: by 2002:a17:90a:8d08:b0:1c6:5ada:9920 with SMTP id c8-20020a17090a8d0800b001c65ada9920mr9482799pjo.126.1648781045575; Thu, 31 Mar 2022 19:44:05 -0700 (PDT) Received: from [192.168.1.100] ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id f15-20020a056a0022cf00b004fb32b9e000sm852809pfj.1.2022.03.31.19.44.03 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 31 Mar 2022 19:44:04 -0700 (PDT) Message-ID: Date: Thu, 31 Mar 2022 20:44:02 -0600 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux aarch64; rv:91.0) Gecko/20100101 Thunderbird/91.7.0 Subject: Re: [PATCH 17/17] nvme: enable non-inline passthru commands Content-Language: en-US To: Kanchan Joshi Cc: Christoph Hellwig , Kanchan Joshi , Keith Busch , Pavel Begunkov , io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, sbates@raithlin.com, logang@deltatee.com, Pankaj Raghav , =?UTF-8?Q?Javier_Gonz=c3=a1lez?= , Luis Chamberlain , Adam Manzanares , Anuj Gupta References: <20220308152105.309618-18-joshi.k@samsung.com> <20220310083652.GF26614@lst.de> <20220310141945.GA890@lst.de> <20220311062710.GA17232@lst.de> <20220324063218.GC12660@lst.de> <20220325133921.GA13818@test-zns> <20220330130219.GB1938@lst.de> From: Jens Axboe In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org On 3/31/22 8:33 PM, Kanchan Joshi wrote: > On Fri, Apr 1, 2022 at 6:55 AM Jens Axboe wrote: >> >> On 3/30/22 7:14 AM, Kanchan Joshi wrote: >>> On Wed, Mar 30, 2022 at 6:32 PM Christoph Hellwig wrote: >>>> >>>> On Fri, Mar 25, 2022 at 07:09:21PM +0530, Kanchan Joshi wrote: >>>>> Ok. If you are open to take new opcode/struct route, that is all we >>>>> require to pair with big-sqe and have this sorted. How about this - >>>> >>>> I would much, much, much prefer to support a bigger CQE. Having >>>> a pointer in there just creates a fair amount of overhead and >>>> really does not fit into the model nvme and io_uring use. >>> >>> Sure, will post the code with bigger-cqe first. >> >> I can add the support, should be pretty trivial. And do the liburing >> side as well, so we have a sane base. > > I will post the big-cqe based work today. It works with fio. > It does not deal with liburing (which seems tricky), but hopefully it > can help us move forward anyway . Let's compare then, since I just did the support too :-) Some limitations in what I pushed: 1) Doesn't support the inline completion path. Undecided if this is super important or not, the priority here for me was to not pollute the general completion path. 2) Doesn't support overflow. That can certainly be done, only complication here is that we need 2x64bit in the io_kiocb for that. Perhaps something can get reused for that, not impossible. But figured it wasn't important enough for a first run. I also did the liburing support, but haven't pushed it yet. That's another case where some care has to be taken to avoid makig the general path slower. Oh, it's here, usual branch: https://git.kernel.dk/cgit/linux-block/log/?h=io_uring-big-sqe and based on top of the pending 5.18 bits and the current 5.19 bits. >> Then I'd suggest to collapse a few of the patches in the series, >> the ones that simply modify or fix gaps in previous ones. Order >> the series so we build the support and then add nvme support >> nicely on top of that. > > I think we already did away with patches which were fixing only the > gaps. But yes, patches still add infra for features incrementally. > Do you mean having all io_uring infra (async, plug, poll) squashed > into a single io_uring patch? At least async and plug, I'll double check on the poll bit. > On a related note, I was thinking of deferring fixed-buffer and > bio-cache support for now. Yes, I think that can be done as a round 2. Keep the current one simpler. -- Jens Axboe