From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from fout8-smtp.messagingengine.com (fout8-smtp.messagingengine.com [103.168.172.151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D3F3384A27; Wed, 4 Sep 2024 19:37:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.151 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725478629; cv=none; b=HME9+N11WnafohKOKOFXhHcW8p+BPXsvZpR+BCIpA71iolCsOYeodnPs4hqReQOUmlBxgyHD+O6LvD85n1+uk0oSDqFGj5z/VjotqF4oILq5GWCpXmpVnH2XldYuT/kmrj/oPwvukr3BzZiOe61yaYtdTOMNxPNAm5UEwlZ6rn4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725478629; c=relaxed/simple; bh=WXLBYws3lOOhhmemXY9s0yF1S/8d/7OC6/5xIPaUf8c=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=EbhRVHprm/CYFQ//TsJ2lgMK1EMzu4k87CjNIcpTh97aUncnWa75UOmeX7jPY7yc2FEbt6gOG2+io61t6hDlbhKqQvjZcS/K/DU/+DsWTufAaLnG6flO+4bLOUulNViTwHkqETgkKH6RRlj9amty6V9Mo7K5VtTqg1/X98CtUqc= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=fastmail.fm; spf=pass smtp.mailfrom=fastmail.fm; dkim=pass (2048-bit key) header.d=fastmail.fm header.i=@fastmail.fm header.b=CP/L8Eso; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=sxNmrbra; arc=none smtp.client-ip=103.168.172.151 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=fastmail.fm Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fastmail.fm Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=fastmail.fm header.i=@fastmail.fm header.b="CP/L8Eso"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="sxNmrbra" Received: from phl-compute-02.internal (phl-compute-02.phl.internal [10.202.2.42]) by mailfout.phl.internal (Postfix) with ESMTP id 095E21380140; Wed, 4 Sep 2024 15:37:07 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-02.internal (MEProxy); Wed, 04 Sep 2024 15:37:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastmail.fm; h= cc:cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm1; t=1725478627; x=1725565027; bh=lRQ0gFMcstNID5luvvk1fu5bLIw733PRkFBZc2IEINE=; b= CP/L8EsoUBZsJntz4Owkd0mNHU0IgZsWoNG1oPTmjJcCtwtiedK67y3Ss9/+bKYv DyWhZly8mN+LJ6K76xjDKh5KRRLHf1jrUCJftTwlq7aZ+ypb4ELfdGvjELGOR1L5 Pkk4t+TR2AsaXtwDxUloqv3pkHf2bA+LZgAhIdtcPUDXWH08zCMtgt7McP0dTDD0 urca9l8q3TxY8+zcTrrJvJV9lAzQwCvAL7IdBi6yfp5qHFWXDyRUKDmNUkqj6cZq XR8DVdmyi3egl61FfloFVqniU5cUZBs9lT4uz0UBDf76IZb6lQvUKVIKSzFrwrgp JHUPZhn3UIwxkt6fZSHZxQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1725478627; x= 1725565027; bh=lRQ0gFMcstNID5luvvk1fu5bLIw733PRkFBZc2IEINE=; b=s xNmrbra4uADkU2fd/DV9oTQ5Mo52fdhpl7n1ih5LRUB1nX0P9tsIxHRhoZaK2Ecb uyXsjnmNuAr6iQnJgbDc6jNnnGRTNthvqa8V+6LtXh/rE4KHNW4tHLWXXGny6zqH MpolddZhmg0p4RLEm6Oo3TN5b7dtjW26S5ZReYz8S30vMd3IzQb0ltPuJpn4kaDQ qohEc4PMixdm/HR0w5jnClXScUM5iCxs1UhgZsEnMBJWfIk+EYKcrhW07Tk4l8Bf 5fT5x4bV4Uwo2oCyCiSGxPFCk1wEJ+2Zo2tPtvf/3WyoNakqipfg2RvT7gq2eMxe AybfHkZqyn6WahRZrxcpA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrudehjedgudefkecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdp uffrtefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivg hnthhsucdlqddutddtmdenucfjughrpefkffggfgfuvfevfhfhjggtgfesthejredttddv jeenucfhrhhomhepuegvrhhnugcuufgthhhusggvrhhtuceosggvrhhnugdrshgthhhusg gvrhhtsehfrghsthhmrghilhdrfhhmqeenucggtffrrghtthgvrhhnpeffjeevfeefjefg hfefhfeiueffffetledtgffhhfdttdefueevledvleetfeevtdenucffohhmrghinhepkh gvrhhnvghlrdhorhhgnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghi lhhfrhhomhepsggvrhhnugdrshgthhhusggvrhhtsehfrghsthhmrghilhdrfhhmpdhnsg gprhgtphhtthhopedutddpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtoheprgigsgho vgeskhgvrhhnvghlrdgukhdprhgtphhtthhopegsshgthhhusggvrhhtseguughnrdgtoh hmpdhrtghpthhtohepmhhikhhlohhssehsiigvrhgvughirdhhuhdprhgtphhtthhopegr shhmlhdrshhilhgvnhgtvgesghhmrghilhdrtghomhdprhgtphhtthhopegsvghrnhguse hfrghsthhmrghilhdrfhhmpdhrtghpthhtoheplhhinhhugidqfhhsuggvvhgvlhesvhhg vghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehiohdquhhrihhnghesvhhgvghrrd hkvghrnhgvlhdrohhrghdprhgtphhtthhopehjohgrnhhnvghlkhhoohhnghesghhmrghi lhdrtghomhdprhgtphhtthhopehjohhsvghfsehtohigihgtphgrnhgurgdrtghomh X-ME-Proxy: Feedback-ID: id8a24192:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 4 Sep 2024 15:37:05 -0400 (EDT) Message-ID: <93127b12-77ae-4e25-bb48-8c8596c7702f@fastmail.fm> Date: Wed, 4 Sep 2024 21:37:04 +0200 Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH RFC v3 00/17] fuse: fuse-over-io-uring To: Jens Axboe , Bernd Schubert , Miklos Szeredi , Pavel Begunkov , bernd@fastmail.fm Cc: linux-fsdevel@vger.kernel.org, io-uring@vger.kernel.org, Joanne Koong , Josef Bacik , Amir Goldstein References: <20240901-b4-fuse-uring-rfcv3-without-mmap-v3-0-9207f7391444@ddn.com> From: Bernd Schubert Content-Language: en-US, de-DE, fr In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit On 9/4/24 18:42, Jens Axboe wrote: > Overall I think this looks pretty reasonable from an io_uring point of > view. Some minor comments in the replies that would need to get > resolved, and we'll need to get Ming's buffer work done to reap the dio > benefits. > > I ran a quick benchmark here, doing 4k buffered random reads from a big > file. I see about 25% improvement for that case, and notably at half the > CPU usage. That is a bit low for my needs, but you will definitely need to wake up on the same core - not applied in this patch version. I also need to re-test with current kernel versions, but I think even that is not perfect. We had a rather long discussion here https://lore.kernel.org/lkml/d9151806-c63a-c1da-12ad-c9c1c7039785@amd.com/T/#r58884ee2c68f9ac5fdb89c4e3a968007ff08468e and there is a seesaw hack, which makes it work perfectly. Then got persistently distracted with other work - so far I didn't track down yet why __wake_up_on_current_cpu didn't work. Back that time it was also only still patch and not in linux yet. I need to retest and possible figure out where the task switch happens. Also, if you are testing with with buffered writes, v2 series had more optimization, like a core+1 hack for async IO. I think in order to get it landed and to agree on the approach with Miklos it is better to first remove all these optimizations and then fix it later... Though for performance testing it is not optimal. Thanks, Bernd