From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.5 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4158EC433E0 for ; Sun, 12 Jul 2020 20:32:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 12E982063A for ; Sun, 12 Jul 2020 20:32:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20150623.gappssmtp.com header.i=@kernel-dk.20150623.gappssmtp.com header.b="2MnqoxIB" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729249AbgGLUco (ORCPT ); Sun, 12 Jul 2020 16:32:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60318 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729050AbgGLUco (ORCPT ); Sun, 12 Jul 2020 16:32:44 -0400 Received: from mail-pl1-x643.google.com (mail-pl1-x643.google.com [IPv6:2607:f8b0:4864:20::643]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6D0BBC061794 for ; Sun, 12 Jul 2020 13:32:44 -0700 (PDT) Received: by mail-pl1-x643.google.com with SMTP id o1so4555670plk.1 for ; Sun, 12 Jul 2020 13:32:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-language:content-transfer-encoding; bh=/RUUpDripryhlIQErb5BlWHxredIBHW0RouNiFZpmj8=; b=2MnqoxIBOKd1TPSfnYrDBwCEfJAvBIzLbvciHkwogV09JSq5MZSroehXVZfpID9i/f GcwsnD/ueP2MC53zypStMuEB6eLtN1Uy+i0XZhwxwWmNzG2/YHIC2A1XIbTZU1SctRrh Io8ot9xYEXazo6JSDnHwkBVTwwlPtXqj40KfHRC/NmuzkkD22Uqingq+yDfmrpsdvSI2 o1PfP5cd+fu7SpTzJSfJFUKzHW8nX2yMh2SjrNXDxHcnE4HlRbv66JIldeQCEcCfw7Jk +TrS2M2eWX3XBnppKqS6BiBcgFDZmVshULwtglp1do9fJeeGBaF6WTzS0T6VAYbxJQgJ +Q9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=/RUUpDripryhlIQErb5BlWHxredIBHW0RouNiFZpmj8=; b=oZQ46LMhYNvGElYgIVxn7PuLwvtWBSnH8LvgjbV0b+JHvvVgrbWNTjF0OF4eMPEtOw 72GdiLx2iTFXNZ+FEAH+Rav2uEaoxOkWX5VFF/VQ10HnL3hCNzOqe1KwzRcfnc3gj2C7 1tInYdIf8bmp80PdAOSCV3mozsPgxlNqBDigkmlTJdarGblfG9KQQrxrHjhW/HSyVMay uWzA9+dODNx082xzaWeAFmIuOf4SlFNCOIi4fYb2htnQ4Atbf/ixepgiTleAf8W0mONB jr4GQ74xrxNmIO4jwVa8ytt5KFWZhrJr/fw4xUxCP4vkt7IPUqI71fwRvK5Q/rAnm62c yssg== X-Gm-Message-State: AOAM531DNw8CXt7QGSH5+QNHVkHZIS31EdDR+NUkrdXi/gRT9dhora4t HjZXYgOO9fm8PJP2hQRxAz7BA2uzIdRh1Q== X-Google-Smtp-Source: ABdhPJxKkJ7PBOFFGbSzrVe0mzJMIztZYHTN1PID5VHvoaWPehM52hdRtwxKxidhgST7MogD/+9pPg== X-Received: by 2002:a17:90a:e57:: with SMTP id p23mr16255868pja.164.1594585963556; Sun, 12 Jul 2020 13:32:43 -0700 (PDT) Received: from [192.168.1.182] ([66.219.217.173]) by smtp.gmail.com with ESMTPSA id p12sm11264635pjz.44.2020.07.12.13.32.42 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 12 Jul 2020 13:32:43 -0700 (PDT) Subject: Re: [RFC 0/9] scrap 24 bytes from io_kiocb To: Pavel Begunkov , io-uring@vger.kernel.org References: <618bc9a5-420c-b176-df86-260734270f56@gmail.com> From: Jens Axboe Message-ID: <3b3ee104-ee6b-7147-0677-bd0eb4efe76e@kernel.dk> Date: Sun, 12 Jul 2020 14:32:41 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: <618bc9a5-420c-b176-df86-260734270f56@gmail.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: io-uring-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org On 7/12/20 11:34 AM, Pavel Begunkov wrote: > On 12/07/2020 18:59, Jens Axboe wrote: >> On 7/12/20 3:41 AM, Pavel Begunkov wrote: >>> Make io_kiocb slimmer by 24 bytes mainly by revising lists usage. The >>> drawback is adding extra kmalloc in draining path, but that's a slow >>> path, so meh. It also frees some space for the deferred completion path >>> if would be needed in the future, but the main idea here is to shrink it >>> to 3 cachelines in the end. >>> >>> I'm not happy yet with a few details, so that's not final, but it would >>> be lovely to hear some feedback. >> >> I think it looks pretty good, most of the changes are straight forward. >> Adding a completion entry that shares the submit space is a good idea, >> and really helps bring it together. >> >> From a quick look, the only part I'm not super crazy about is patch #3. > > Thanks! > >> I'd probably rather use a generic list name and not unionize the tw >> lists. > > I don't care much, but without compiler's help always have troubles > finding and distinguishing something as generic as "list". To me, it's easier to verify that we're doing the right thing when they use the same list member. Otherwise you have to cross reference two different names, easier to shoot yourself in the foot that way. So I'd prefer just retaining it as 'list' or something generic. > BTW, I thought out how to bring it down to 3 cache lines, but that would > require taking io_wq_work out of io_kiocb and kmalloc'ing it on demand. > And there should also be a bunch of nice side effects like improving apoll. How would this work with the current use of io_wq_work as storage for whatever bits we're hanging on to? I guess it could work with a prep series first more cleanly separating it, though I do feel like we've been getting closer to that already. Definitely always interested in shrinking io_kiocb, just need to keep an eye out for the fast(er) paths not needing two allocations (and frees) for a single request. -- Jens Axboe