From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93E2BC636D4 for ; Fri, 10 Feb 2023 18:37:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233117AbjBJShZ (ORCPT ); Fri, 10 Feb 2023 13:37:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59898 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233112AbjBJShY (ORCPT ); Fri, 10 Feb 2023 13:37:24 -0500 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 467D72D177 for ; Fri, 10 Feb 2023 10:37:22 -0800 (PST) Received: by mail-wr1-x429.google.com with SMTP id y1so5949714wru.2 for ; Fri, 10 Feb 2023 10:37:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux-foundation.org; s=google; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=45HS3XcRsfRS5VbZ2TiXDHQBLKSDRISNQErTXQPnH+g=; b=fA28mHicQVXU0rc9eizNL3RpJL4UanQ9gfQOBZ57SNqi5RpYK6zoS094B9dEWMLQUJ VDR5+QKCsCoY483JdS84PYO+YawCsU7ivhQXQygIuTrEgqYgXEUqNL9o00YmCoBr4cj0 8loINQ8lMH5qSDzADuKONPbLiCHUOVpr3CWx4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=45HS3XcRsfRS5VbZ2TiXDHQBLKSDRISNQErTXQPnH+g=; b=pL5rNVQmNA5aXBy8gAkq2Qhx0IvBIaHQkj5awF0i/TiE4AbJFUMyYFcSipYWN1a5HP V0Tk7ypKFVCtwV/S76tslCletBN5ZTm68uPzdxftWdL+xfN0NfTnOSdnDvPthRHHgDt8 oqhFuhJhdul0vRXBSHEajOubxrGtZJFIsPc/BEPtAMOmMo5joGqtFdqpkBWl3iW/jq37 IHRRAGVk4+QSPUvl2iSn4DcG6KgC1C+E96AZWg1Og/AGkrR8bXny53/K41lDx4BYl00H 965DFyj5qnVb6kMlTCGghlSkbRFzKUA0lwZ0q1SogJN9sgaE/AygxqQVkopVl/6gCjJ5 N8rw== X-Gm-Message-State: AO0yUKVmaOZMdd+NOd1r5ViyqJl7SSLbCbU7do1xb+8UWVbeJ9oRQyMg RoE3ZUvZu1XjuelfMtP+PU166d3bO3fqonREcOU= X-Google-Smtp-Source: AK7set/NmxgjiEZOFkqY4Re9vqB1kvnA1eiZhYTp/uCbrub/4tePtnRMVUK5AfCGK11hySDDdYgHdQ== X-Received: by 2002:adf:f484:0:b0:2c3:e34d:a331 with SMTP id l4-20020adff484000000b002c3e34da331mr15542124wro.36.1676054240418; Fri, 10 Feb 2023 10:37:20 -0800 (PST) Received: from mail-ed1-f46.google.com (mail-ed1-f46.google.com. [209.85.208.46]) by smtp.gmail.com with ESMTPSA id v5-20020adfe4c5000000b002c549b91c54sm2223899wrm.52.2023.02.10.10.37.19 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 10 Feb 2023 10:37:19 -0800 (PST) Received: by mail-ed1-f46.google.com with SMTP id r3so5386234edq.13 for ; Fri, 10 Feb 2023 10:37:19 -0800 (PST) X-Received: by 2002:a50:f61e:0:b0:4ab:168c:dbd7 with SMTP id c30-20020a50f61e000000b004ab168cdbd7mr1816109edn.5.1676054239147; Fri, 10 Feb 2023 10:37:19 -0800 (PST) MIME-Version: 1.0 References: <0cfd9f02-dea7-90e2-e932-c8129b6013c7@samba.org> <20230210021603.GA2825702@dread.disaster.area> <20230210040626.GB2825702@dread.disaster.area> <20230210065747.GD2825702@dread.disaster.area> In-Reply-To: From: Linus Torvalds Date: Fri, 10 Feb 2023 10:37:01 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: copy on write for splice() from file to pipe? To: Andy Lutomirski Cc: Dave Chinner , Matthew Wilcox , Stefan Metzmacher , Jens Axboe , linux-fsdevel , Linux API Mailing List , io-uring , "linux-kernel@vger.kernel.org" , Al Viro , Samba Technical Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org On Fri, Feb 10, 2023 at 9:57 AM Andy Lutomirski wrote: > > I am saying exactly what I meant. Obviously mutable data exists. I'm > saying that *putting it in a pipe* *while it's still mutable* is not > good. Which implies that I don't think splice() is good. No offense. No offense at all. As mentioned, I have grown to detest splice over the years. That said, in defense of splice(), it really does solve a lot of conceptual problems. And I still think that conceptually it's absolutely lovely in *theory*. And part of it is very much the fact that pipes are useful and have the infrastructure for other things. So you can mix regular read/write calls with splice, and it actually makes sense. One of the design goals was for things like static http, where you don't really send out just file contents, there's a whole header to it as well. So it's not just a specialized "send file contents to network", it's a "you can do a write() call to start filling the pipe buffer with the http header, then a splice() to start filling the file data". And it was also designed to allow other sources, notably things like video capture cards etc. And very much multiple destinations (again, media accelerators). So it all "makes sense" conceptually as a generic pipe (sic) between different sources and sinks. And again, using a pipe as the mechanism then also makes perfect sense in a historical Unix context of "everything is a pipe". But. The above just tries to make sense of the design, and excuses for it. I want to re-iterate that I think it's all lovely and coherent conceptually. But in practice, it's just a huge pain. The same way "everything is a pipeline of processes" is very much historical Unix and very useful for shell scripting, but isn't actually then normally very useful for larger problems, splice() really never lived up to that conceptual issue, and it's just really really nasty in practice. But we're stuck with it. I'm not convinced your suggestion of extending io_uring with new primitives is any better in practice, though. Linus