From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wm2-f12.google.com (mail-wm2-f12.google.com [74.125.225.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 238CD47DD63 for ; Mon, 5 Oct 2026 11:49:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.225.140 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791200962; cv=none; b=IEzHyRAEQLlgPNUDZ0HHmH5WWlS0STnyybt5TE9Mvzy5Bw/VYzXIY7SiGLLfucBckwrxyp6s+yo2BUuwz/eSKGD0qd0x/5s/qvqj9p2DPol9vdPQxd/2kbQXpVo4k5fceYDGN9/TKm8odisR9Owx62W501qlyE20x3XhxC16KKk= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791200962; c=relaxed/simple; bh=C4rWPxAM+gG+nBkU+Lpq5+tCLNPh5lzEfMeHsRf0DPA=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=dtpPpUeh37ze4Zi+t/OIdayKRbyJ8PtDIzQ2j2Qn+BDM4i8jdXdLgiiwhy3I96vcgwkDru4bJMZ7Vvz6sZ2oFq+VmEkJJqg8VjLYDGhQ/yVvZj0VnF3D4CKqQk3C2veP1OoRq/NOtKoSaHUm0kzETeFLtCIGnRRnxani43E8a8A= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=oe2e4wNU; arc=none smtp.client-ip=74.125.225.140 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="oe2e4wNU" Received: by mail-wm2-f12.google.com with SMTP id 5b1f17b1804b1-49d1fb0cf5eso14151105e9.3 for ; Mon, 05 Oct 2026 04:49:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1791200959; x=1791805759; darn=vger.kernel.org; h=content-transfer-encoding:content-type:in-reply-to:from :content-language:references:cc:to:subject:user-agent:mime-version :date:message-id:from:to:cc:subject:date:message-id:reply-to :content-type; bh=tN32JBtkwvrpOHkqPOwCz1peQMff+9nW+9FnKuTc/9Y=; b=oe2e4wNU3lig5ckdIY4pln90NFMoBuzqjjQp9ZzJ8X5L77pYYYqB8QGUGhNn5BK7ag teomHdKgtcHPMlz/WiXvHlG4Mqk0rycrnXJ912uIw6GEYYvPKxEuLdPx15bRUCANE2uv ElhJ5VhfFNTy7CEgJukBL5HhoL0XKvCSIG556r/Nj1BH3XRFpc1MA1+5+ZVLdT3ilJvY W0bKiKQ9tO8NasBmu7RcIKz5cRszVG0qmcnUvvCR2Zv0WtGFQyYVQGXvgo0gFlVlHf4X JKNl84nvM/LvHX0h4AUV4kuiFJiMSkSxDwbWQhuoEtIxtOwK7scS6B0zruBGFZZJE8ZU b51w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1791200959; x=1791805759; h=content-transfer-encoding:content-type:in-reply-to:from :content-language:references:cc:to:subject:user-agent:mime-version :date:message-id:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=tN32JBtkwvrpOHkqPOwCz1peQMff+9nW+9FnKuTc/9Y=; b=LTvmhYEp1gyYVtA4RKSxplzah6fj6hlvfce942keWyi5tkoAARZ8iQy9S5dNtt4NYk u7OYsYEzQrIcarp9XDFLDIwRCnva6utfNjvwi27yIrtSqJsJ08N/jMNsb22Kz2MjAWRD qyIkseAihBaKSDIOHvTZHiYtwbpTlsa49lI3SwSSoLtXpOF/oCJAUevO4uOOfU/Ichm4 s+w+xBFFkidJqTsc3daPJg7Fa9jkDWh8VDaTuKPX7/kLoefd3kkCFHSbKNZMag3pkwAs Opg5UKDmHl779ILIpJdwYz03ovXCQ74BrPGQwNAm8iUauFt12xmV4AwwaaMUFEecL2yO VYEw== X-Forwarded-Encrypted: i=1; AKwUvBzGDT/NXDoSRfTiYCHsvotDWZ9xggXr8GP3L7SYFiH4qjo8APnckC0YxcfSjyt1j9q+Bn7UlwwJ/g==@vger.kernel.org X-Gm-Message-State: AFuF++lXzn5J09ymMVs7lzaXzc9jCFMJY6mUGTCmvarX+fxy1ujn+Ot7 f5+lorHf+c6rjMbNJVDx0kog97Yqk5XK8S1+tZ9GQJP9upthglsd7rRg X-Gm-Gg: AYBFou1XEJl9MgEuPhokAL+3uEkrllJNTdHVhtYfZ/K+7Yz0DnJJkHDdhjLzKqTI5RJ uSkjYQS0JVpuYfzny31UKY0m7NkOa539S+r0WL5CcGnpa4k2SbH++wBa5UKcuL0DPD+NKlXKCRY pHY9FJi6vhCApypjBd3hFYcUOcZn6wwSeCk/O1NnWz68vwv9wCS6Sy0gTMs9mrFDZVPFdTVQ01N enQE2pH+45J+Zcg4LLhckuSA5R7EHI8ULYgDKCmk6CND0Fqmx8GiL1RWs04VWdK5G8d4TbB5K8W p3uQiotxhr9PK3W3KIy6ne/8KXdZFXMusZ+surTiLOjFWdtr3W/8jJANWCCILnlfW+mMl9/JwXz tCqnDrGd0jQYOft3KObpor4Wt7+gfr+Vzn8blCczgeiKCYzxboG5/UJQjnuYNA8ueZUR++/Bbh6 HWa6cVoi/p6FfNYvAgiMxnrMW0gjJBd9L/mu2E+M1rQrDlnXH+taqUVs7Vf6kkxw0J1/lh0gvHI ujuX4onLHapVnuw68rqHz1aNWXdyicCqgObS0IWfcwTsvqp2KBONv/hI7hPi9wrMyxNb9GwsFc= X-Received: by 2002:a05:600c:1c08:b0:4a1:722a:d2a3 with SMTP id 5b1f17b1804b1-4a1722ad405mr52583605e9.23.1791200959268; Mon, 05 Oct 2026 04:49:19 -0700 (PDT) Received: from [10.41.30.1] ([161.12.45.16]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4a027727cecsm346670305e9.9.2026.10.05.04.49.17 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 05 Oct 2026 04:49:18 -0700 (PDT) Message-ID: Date: Mon, 5 Oct 2026 12:49:15 +0100 Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v6 01/13] dma-buf: introduce initial file I/O infrastructure To: Matthew Brost Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org, io-uring@vger.kernel.org, Christoph Hellwig , Sumit Semwal , =?UTF-8?Q?Christian_K=C3=B6nig?= , Keith Busch , Sagi Grimberg , Alexander Viro , Christian Brauner , Jan Kara , Andrew Morton , Jens Axboe , Nitesh Shetty , Kanchan Joshi , Anuj Gupta , Tushar Gohad , William Power , Phil Cayton , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , Benjamin Marzinski , dm-devel@lists.linux.dev References: <5490ee42c4452fd4198b245ead20fcf7438c066e.1789997898.git.asml.silence@gmail.com> <447c04dd-6bfa-4c45-99e7-37d25a7ab539@gmail.com> Content-Language: en-US From: Pavel Begunkov In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 9/30/26 20:43, Matthew Brost wrote: > On Wed, Sep 30, 2026 at 01:48:49PM +0100, Pavel Begunkov wrote: >> On 9/30/26 11:08, Pavel Begunkov wrote: >>> On 9/30/26 04:00, Matthew Brost wrote: >>>> On Mon, Sep 21, 2026 at 02:38:45PM +0100, Pavel Begunkov wrote: >>> ...>> +struct dma_buf_io_map *dma_buf_io_create_map(struct dma_buf_io_ctx *ctx) >>>>> +{ >>>>> +    struct dma_buf *dmabuf = ctx->dmabuf; >>>>> +    struct dma_buf_io_map *map; >>>>> +    long ret; >>>>> + >>>>> +    guard(mutex)(&ctx->map_create_mutex); >>>>> + >>>>> +    scoped_guard(mutex, &ctx->map_mutex) { >>>>> +        if (ctx->maps_killed) >>>>> +            return ERR_PTR(-ENOENT); >>>>> +        /* recheck under the lock in case it has already been re-created */ >>>>> +        map = __dma_buf_io_get_map(ctx); >>>>> +        if (map) >>>>> +            return map; >>>>> +    } >>>>> + >>>>> +    dma_buf_io_wait_active_maps(ctx); >>>>> + >>>>> +    ret = dma_resv_lock_interruptible(dmabuf->resv, NULL); >>>>> +    if (ret) >>>>> +        return ERR_PTR(ret); >>>>> + >>>>> +    ret = dma_resv_wait_timeout(dmabuf->resv, DMA_RESV_USAGE_KERNEL, >>>>> +                    true, MAX_SCHEDULE_TIMEOUT); >>>>> +    if (ret <= 0) { >>>>> +        if (!ret) >>>>> +            ret = -EAGAIN; >>>>> +        dma_resv_unlock(dmabuf->resv); >>>>> +        return ERR_PTR(ret); >>>>> +    } >>>>> + >>>>> +    map = ctx->dev_ops->map(ctx); >>>> >>>> I'm playing around this code now. >>>> >>>> I think you need the dma_resv_wait_timeout after the 'map'? >>>> >>>> If a device doesn't support p2p ->map() will typically trigger an async >>>> migrate to system memory and data will be moving but the map is valid - >>>> Xe 100% does this, I checked AMDGPU and fairly confident it has the same >>>> async behavior. >>> >>> There is a wait right before because I read somewhere in dma-buf >>> comments that I need to do that, sounds a bit odd if I need to >>> wait on fences before and after. I can add it, just curious how >>> come that other dma_buf_map_attachment() callers don't need to do >>> that. Or maybe they wait somewhere else? > > In GPU drivers, when we receive a foreign object and map it in Xe or > AMDGPU, this logic sits deep in the stack. However, the top-level call > is typically ttm_bo_validate(), which in both drivers eventually > resolves to the TTM ->move() vfunc. That path calls > dma_buf_map_attachment(), which in turn invokes the dma-buf's ->map() > callback. > > That ->map() callback can trigger a asynchronous move on a different > device, where the top-level call is again ttm_bo_validate(). > We then end up back in the TTM ->move() vfunc, where kernel fences are > installed. I realize that's a lot of layers, but the call chain can end > up looking like this. > > Now we have a mapping and an object with kernel fences attached. Before > the object can be used by either the exec IOCTL (batch buffer > submission), the VM bind IOCTL (mapping into the GPU address space), or > a CPU page fault (not relevant for dma-bufs since they cannot be > CPU-mapped, but a useful example of a normal BO move), we wait on those > kernel fences. > > In the case of the exec IOCTL or VM bind IOCTL, the kernel fences are > added as dependencies to a drm_sched job, delaying its execution until > the fences signal. That is where the wait occurs. In the case of a CPU > page fault, the kernel waits directly on the fences before installing > the CPU page mappings. > > So the TL;DR is if you want to immediately hand back a valid mapping, > the kernel fences must be waited on before returning after ->map() call. Thanks for the overview! I'll add the wait after ->map in v8 >> I can't find it, so maybe it was the comment below and I mixed >> sth up back then. I'll move it after ->map(). >> >> >> * Note that for non-dynamic exporters the driver must guarantee that >> * that the memory is available for use and cleared of any old data by >> * the time this function returns. Drivers which pipeline their buffer >> * moves internally must wait for all moves and clears to complete. >> * Dynamic exporters do not need to follow this rule: For non-dynamic >> * importers the buffer is already pinned through @pin, which has the >> * same requirements. Dynamic importers otoh are required to obey the >> * dma_resv fences. >> * -- Pavel Begunkov