From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4ADF0C433F5 for ; Mon, 14 Feb 2022 17:44:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1357289AbiBNRpB (ORCPT ); Mon, 14 Feb 2022 12:45:01 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:35010 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1357236AbiBNRon (ORCPT ); Mon, 14 Feb 2022 12:44:43 -0500 Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 84A9B6548A for ; Mon, 14 Feb 2022 09:44:35 -0800 (PST) Received: from pps.filterd (m0089730.ppops.net [127.0.0.1]) by m0089730.ppops.net (8.16.1.2/8.16.1.2) with ESMTP id 21ECPZYU015470 for ; Mon, 14 Feb 2022 09:44:34 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=WqJGTjlBszlWWKZYtp71MBllUosGp30baKNJ+u9prKQ=; b=eTFgI6MxxCJDd+hMmqvY/GQq8D+Mwsn1gF53kcc87W9Y2R6XfndmXJ0XK1CnPoCyB5KU DNx7xEAtXDVQwkx7NpdCf7biIWxbnWLu1GfyNi++WazJT5TBhANMjdZiFq/q8UG/t7vr FPMIH309p+sCJm+u3hDB3/XKWqdbHGhjYhw= Received: from mail.thefacebook.com ([163.114.132.120]) by m0089730.ppops.net (PPS) with ESMTPS id 3e7q2ea45x-7 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Mon, 14 Feb 2022 09:44:34 -0800 Received: from twshared22811.39.frc1.facebook.com (2620:10d:c085:108::4) by mail.thefacebook.com (2620:10d:c085:21d::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Mon, 14 Feb 2022 09:44:32 -0800 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id C0AFEABBD105; Mon, 14 Feb 2022 09:44:09 -0800 (PST) From: Stefan Roesch To: , , , CC: Subject: [PATCH v1 09/14] io_uring: add support for async buffered writes Date: Mon, 14 Feb 2022 09:43:58 -0800 Message-ID: <20220214174403.4147994-10-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220214174403.4147994-1-shr@fb.com> References: <20220214174403.4147994-1-shr@fb.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-FB-Internal: Safe Content-Type: text/plain X-Proofpoint-GUID: 2xm-Dlc03T3F4NBd1FDscs5Xibv6n8hd X-Proofpoint-ORIG-GUID: 2xm-Dlc03T3F4NBd1FDscs5Xibv6n8hd X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.816,Hydra:6.0.425,FMLib:17.11.62.513 definitions=2022-02-14_07,2022-02-14_03,2021-12-02_01 X-Proofpoint-Spam-Details: rule=fb_outbound_notspam policy=fb_outbound score=0 mlxscore=0 suspectscore=0 impostorscore=0 clxscore=1015 malwarescore=0 priorityscore=1501 bulkscore=0 mlxlogscore=953 phishscore=0 lowpriorityscore=0 adultscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000 definitions=main-2202140105 X-FB-Internal: deliver Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org This enables the async buffered writes for block devices in io_uring. Buffered writes are enabled for blocks that are already in the page cache or can be acquired with noio. It is possible that a write request cannot be completely fullfilled (short write). In that case the request is punted and sent to the io workers to be completed. Before submitting the request to the io workers, the request is updated with how much has already been written. Signed-off-by: Stefan Roesch --- fs/io_uring.c | 29 ++++++++++++++++++++++++----- 1 file changed, 24 insertions(+), 5 deletions(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index 2e04f718319d..76b1ff602470 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -3615,7 +3615,7 @@ static inline int io_iter_do_read(struct io_kiocb *= req, struct iov_iter *iter) return -EINVAL; } =20 -static bool need_read_all(struct io_kiocb *req) +static bool need_complete_io(struct io_kiocb *req) { return req->flags & REQ_F_ISREG || S_ISBLK(file_inode(req->file)->i_mode); @@ -3679,7 +3679,7 @@ static int io_read(struct io_kiocb *req, unsigned i= nt issue_flags) } else if (ret =3D=3D -EIOCBQUEUED) { goto out_free; } else if (ret =3D=3D req->result || ret <=3D 0 || !force_nonblock || - (req->flags & REQ_F_NOWAIT) || !need_read_all(req)) { + (req->flags & REQ_F_NOWAIT) || !need_complete_io(req)) { /* read all, failed, already did sync or don't want to retry */ goto done; } @@ -3777,9 +3777,10 @@ static int io_write(struct io_kiocb *req, unsigned= int issue_flags) if (unlikely(!io_file_supports_nowait(req))) goto copy_iov; =20 - /* file path doesn't support NOWAIT for non-direct_IO */ - if (force_nonblock && !(kiocb->ki_flags & IOCB_DIRECT) && - (req->flags & REQ_F_ISREG)) + /* File path supports NOWAIT for non-direct_IO only for block devices.= */ + if (!(kiocb->ki_flags & IOCB_DIRECT) && + !(kiocb->ki_filp->f_mode & FMODE_BUF_WASYNC) && + (req->flags & REQ_F_ISREG)) goto copy_iov; =20 kiocb->ki_flags |=3D IOCB_NOWAIT; @@ -3831,6 +3832,24 @@ static int io_write(struct io_kiocb *req, unsigned= int issue_flags) /* IOPOLL retry should happen for io-wq threads */ if (ret2 =3D=3D -EAGAIN && (req->ctx->flags & IORING_SETUP_IOPOLL)) goto copy_iov; + + if (ret2 !=3D req->result && ret2 >=3D 0 && need_complete_io(req)) { + struct io_async_rw *rw; + + /* This is a partial write. The file pos has already been + * updated, setup the async struct to complete the request + * in the worker. Also update bytes_done to account for + * the bytes already written. + */ + iov_iter_save_state(&s->iter, &s->iter_state); + ret =3D io_setup_async_rw(req, iovec, s, true); + + rw =3D req->async_data; + if (rw) + rw->bytes_done +=3D ret2; + + return ret ? ret : -EAGAIN; + } done: kiocb_done(req, ret2, issue_flags); } else { --=20 2.30.2