From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.0 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DBCCCC43461 for ; Thu, 3 Sep 2020 02:21:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B2FC820773 for ; Thu, 3 Sep 2020 02:21:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20150623.gappssmtp.com header.i=@kernel-dk.20150623.gappssmtp.com header.b="vqCCBYAf" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727819AbgICCVH (ORCPT ); Wed, 2 Sep 2020 22:21:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37208 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727945AbgICCVE (ORCPT ); Wed, 2 Sep 2020 22:21:04 -0400 Received: from mail-pf1-x441.google.com (mail-pf1-x441.google.com [IPv6:2607:f8b0:4864:20::441]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E64A4C061246 for ; Wed, 2 Sep 2020 19:21:03 -0700 (PDT) Received: by mail-pf1-x441.google.com with SMTP id u20so980886pfn.0 for ; Wed, 02 Sep 2020 19:21:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jsr/m2qCMJZs9lYgA2JIdiGg8dIhRn0MzzQ2WqNorz4=; b=vqCCBYAf2QWJP3SzLyaYG3Mv11uIobnUL0a4gbAAYS0UbRGaZ4GMg0rpbRjC6Xh30s siZ8Z4De5d+YRDj4csaxOEWDNeWMX30EuOfqZa2l0qTfXjTiHNMHsL8bwAhbFppirczo 1GDjciedXRNTbClJ+bh0nF3vdMnhxAD1ZQ+ObBKGls1A4xI21Hk9nL6dYQyuYdqpTfZg FykJP+87yHUrI+lZstB0ki6SFlFsl6o9D0qn33asftGp0I3NdlaQX5Oe78bM9drFPHwJ Hc4MeObwlQBG/UIuA14iPkyhl/+1ix/geKryRm2n4ci6Imepv3Hgjo2luPGl2OXo7dtt izBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jsr/m2qCMJZs9lYgA2JIdiGg8dIhRn0MzzQ2WqNorz4=; b=pCpTN+BiEUxs2uw9O/xMk0h1T7oqJqwEQbiBWDkk1mGLdazI5RCuG7eCZHqGQ7IvB+ F49ZilQIBjYZEfh5qOB8Po2xKIvVfKfCEMPwXFLgkDp5ovJPumf9B70JGmU31mdorjbD I5PXi/DTjnN0jXNxBGAdqkW8qjWLo5RGgveS32tU6OJWaYfdK9/EWCPeIYcPqILHVl8X 5jecbdw7R1d1dY3GuG40E/RX3sn/4wKgc9u+ZWmRNzoma0jngC0TnCQneveYO19hi8gz oiVLKXUFXQ2yZoG0PGqA1qtH4wxzDhgFZDUrCG+3nAWMJhIES8vjSne205qtAXs++Rez pSXg== X-Gm-Message-State: AOAM533WNextQE6O9LIIbU5pmPXMbV71ajj1LOTrw2WRc2wc+j2PgJQK hGP53K2ENK9DDM2XNjN0iEmU8vP2PqP8w1Qi X-Google-Smtp-Source: ABdhPJxYPDl+f1yzaYY5o2JGREnljB0LFDYkWAeunnnIenpK2d2PBVMpJSoIkSIXhjohmAr6P7yang== X-Received: by 2002:a65:610f:: with SMTP id z15mr847801pgu.123.1599099663136; Wed, 02 Sep 2020 19:21:03 -0700 (PDT) Received: from localhost.localdomain ([66.219.217.173]) by smtp.gmail.com with ESMTPSA id ie13sm663102pjb.5.2020.09.02.19.21.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Sep 2020 19:21:02 -0700 (PDT) From: Jens Axboe To: io-uring@vger.kernel.org Cc: Jens Axboe Subject: [PATCH 4/8] io_uring: move SQPOLL post-wakeup ring need wakeup flag into wake handler Date: Wed, 2 Sep 2020 20:20:49 -0600 Message-Id: <20200903022053.912968-5-axboe@kernel.dk> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200903022053.912968-1-axboe@kernel.dk> References: <20200903022053.912968-1-axboe@kernel.dk> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: io-uring-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org We need to decouple the clearing on wakeup from the the inline schedule, as that is going to be required for handling multiple rings in one thread. Wrap our wakeup handler so we can clear it when we get the wakeup, by definition that is when we no longer need the flag set. Signed-off-by: Jens Axboe --- fs/io_uring.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index 76f02db37ffc..95c81e0395d9 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -6613,6 +6613,18 @@ static inline void io_ring_clear_wakeup_flag(struct io_ring_ctx *ctx) spin_unlock_irq(&ctx->completion_lock); } +static int io_sq_wake_function(struct wait_queue_entry *wqe, unsigned mode, + int sync, void *key) +{ + struct io_ring_ctx *ctx = container_of(wqe, struct io_ring_ctx, sqo_wait_entry); + int ret; + + ret = autoremove_wake_function(wqe, mode, sync, key); + if (ret) + io_ring_clear_wakeup_flag(ctx); + return ret; +} + static int io_sq_thread(void *data) { struct io_ring_ctx *ctx = data; @@ -6621,6 +6633,7 @@ static int io_sq_thread(void *data) int ret = 0; init_wait(&ctx->sqo_wait_entry); + ctx->sqo_wait_entry.func = io_sq_wake_function; complete(&ctx->sq_thread_comp); @@ -6707,7 +6720,6 @@ static int io_sq_thread(void *data) schedule(); finish_wait(ctx->sqo_wait, &ctx->sqo_wait_entry); - io_ring_clear_wakeup_flag(ctx); ret = 0; continue; } -- 2.28.0