From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.4 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5AD4DC432C3 for ; Fri, 15 Nov 2019 13:10:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 21CC72072D for ; Fri, 15 Nov 2019 13:10:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="FnBVaXtC" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727314AbfKONKx (ORCPT ); Fri, 15 Nov 2019 08:10:53 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:48096 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727223AbfKONKx (ORCPT ); Fri, 15 Nov 2019 08:10:53 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id xAFCx7Zl175955; Fri, 15 Nov 2019 13:10:50 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc : references : from : message-id : date : mime-version : in-reply-to : content-type : content-transfer-encoding; s=corp-2019-08-05; bh=WIdJy37bQllOd+8nBkBehzSw2PBmaIKl7daCm1PZAJ0=; b=FnBVaXtCcysuuNPCD/4T+n8Q6ctN4i8RHwOCS8oqvjaw5tjyadLUOfaQ6lBZKOd9XXJA wOU/rz4k1qJSGEWrd0MoKM6xrhgdsp4+CPzIWsAr8HhkoJ+KviHrTsM05+wr7EiN/sCA Z+5YKFLyXc29U6aYE3NnzMP6N24dgUPmzLOr1sbm9oM3BZTNBQEZWaKuc/GArvbOl4Bd 74CnUSjUbP6pC903sKaV8rjAHD9OWYzt2YvYb+crUGfkcTxqUWhjzxQ9Qof/1A6l1VZ/ 10HTck/5BFhfzaCc2KVCc/+D8dOYjHaGjikzt4I2uqLMuC/KyJ7af/zbB8o4eii08CeK Sg== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by userp2130.oracle.com with ESMTP id 2w9gxpk6m2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 15 Nov 2019 13:10:49 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.27/8.16.0.27) with SMTP id xAFCwCP0081628; Fri, 15 Nov 2019 13:10:49 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userp3020.oracle.com with ESMTP id 2w9h18071e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 15 Nov 2019 13:10:49 +0000 Received: from abhmp0001.oracle.com (abhmp0001.oracle.com [141.146.116.7]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id xAFDAmWc005399; Fri, 15 Nov 2019 13:10:48 GMT Received: from [192.168.1.14] (/114.88.246.185) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 15 Nov 2019 05:10:48 -0800 Subject: Re: [PATCH] io_uring: fix duplicated increase of cached_cq_overflow To: Pavel Begunkov , axboe@kernel.dk Cc: io-uring@vger.kernel.org References: <20191115093733.18396-1-bob.liu@oracle.com> <9ba5abd2-94c2-585a-b55c-d97dc5f429a6@gmail.com> <7ed5d143-c9ba-7d0a-03fe-57af65e88a54@oracle.com> From: Bob Liu Message-ID: Date: Fri, 15 Nov 2019 21:10:33 +0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.5.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9441 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1911140001 definitions=main-1911150120 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9441 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1911140001 definitions=main-1911150120 Sender: io-uring-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org On 11/15/19 8:41 PM, Pavel Begunkov wrote: > On 11/15/2019 3:17 PM, Bob Liu wrote: >> On 11/15/19 5:49 PM, Pavel Begunkov wrote: >>> On 11/15/2019 12:37 PM, Bob Liu wrote: >>>> cached_cq_overflow already be increased in function >>>> io_cqring_overflow_flush(). >>>> >>>> Signed-off-by: Bob Liu >>>> --- >>>> fs/io_uring.c | 2 +- >>>> 1 file changed, 1 insertion(+), 1 deletion(-) >>>> >>>> diff --git a/fs/io_uring.c b/fs/io_uring.c >>>> index 55f8b1d..eb23451 100644 >>>> --- a/fs/io_uring.c >>>> +++ b/fs/io_uring.c >>>> @@ -701,7 +701,7 @@ static void io_cqring_fill_event(struct io_kiocb *req, long res) >>>> WRITE_ONCE(cqe->flags, 0); >>>> } else if (ctx->cq_overflow_flushed) { >>>> WRITE_ONCE(ctx->rings->cq_overflow, >>>> - atomic_inc_return(&ctx->cached_cq_overflow)); >>>> + atomic_read(&ctx->cached_cq_overflow)); >>> >>> Not really. It won't get into io_cqring_overflow_flush() if this branch >>> is executed. >> >> io_cqring_overflow_flush(force=true) must have been called when this branch is executed, >> since io_cqring_overflow_flush() is the only place can set 'ctx->cq_overflow_flushed' to true. >> > Yes, it should have been called, but it sets this flag for the future > users of io_cqring_fill_event(), so any _new_ requests in > io_cqring_fill_event() will overflow instead of being added to > @overflow_list. > Oh, I see..Thanks for the kindly explanation. > Either a request completes/overflows in io_cqring_fill_event(), > or it would be added to @overflow_list to be processed by > io_cqring_overflow_flush(). > > >> And 'ctx->cached_cq_overflow' may already be increased in io_cqring_overflow_flush() if force is true and cqe==NULL. >> >> static void io_cqring_overflow_flush(struct io_ring_ctx *ctx, bool force) >> { >> ... >> if (force) >> ctx->cq_overflow_flushed = true; >> >> while (!list_empty(&ctx->cq_overflow_list)) { >> cqe = io_get_cqring(ctx); >> if (!cqe && !force) >> break; >> >> req = list_first_entry(&ctx->cq_overflow_list, struct io_kiocb, >> list); >> list_move(&req->list, &list); >> if (cqe) { >> ... >> } else { >> WRITE_ONCE(ctx->rings->cq_overflow, >> atomic_inc_return(&ctx->cached_cq_overflow)); >> ^^^^^^^^^^^ >> ctx->cached_cq_overflow is increased if 'force=true' and 'ceq==NULL'. >> >> >> Did I miss anything? >> >>> See, it's enqueued for overflow in "else" right below. >>> >>> i.e. list_add_tail(&req->list, &ctx->cq_overflow_list); >>> >>>> } else { >>>> refcount_inc(&req->refs); >>>> req->result = res; >>>> >>