public inbox for [email protected]
 help / color / mirror / Atom feed
From: Pavel Begunkov <[email protected]>
To: Jens Axboe <[email protected]>, Daurnimator <[email protected]>
Cc: [email protected]
Subject: Re: [PATCHSET 0/4] Add support for shared io-wq backends
Date: Tue, 28 Jan 2020 02:25:29 +0300	[thread overview]
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>


[-- Attachment #1.1: Type: text/plain, Size: 3981 bytes --]

On 28/01/2020 02:23, Jens Axboe wrote:
> On 1/27/20 4:17 PM, Pavel Begunkov wrote:
>> On 28/01/2020 02:00, Jens Axboe wrote:
>>> On 1/27/20 3:40 PM, Jens Axboe wrote:
>>>> On 1/27/20 2:45 PM, Pavel Begunkov wrote:
>>>>> On 27/01/2020 23:33, Jens Axboe wrote:
>>>>>> On 1/27/20 7:07 AM, Pavel Begunkov wrote:
>>>>>>> On 1/27/2020 4:39 PM, Jens Axboe wrote:
>>>>>>>> On 1/27/20 6:29 AM, Pavel Begunkov wrote:
>>>>>>>>> On 1/26/2020 8:00 PM, Jens Axboe wrote:
>>>>>>>>>> On 1/26/20 8:11 AM, Pavel Begunkov wrote:
>>>>>>>>>>> On 1/26/2020 4:51 AM, Daurnimator wrote:
>>>>>>>>>>>> On Fri, 24 Jan 2020 at 10:16, Jens Axboe <[email protected]> wrote:
>>>>>>>>> Ok. I can't promise it'll play handy for sharing. Though, you'll be out
>>>>>>>>> of space in struct io_uring_params soon anyway.
>>>>>>>>
>>>>>>>> I'm going to keep what we have for now, as I'm really not imagining a
>>>>>>>> lot more sharing - what else would we share? So let's not over-design
>>>>>>>> anything.
>>>>>>>>
>>>>>>> Fair enough. I prefer a ptr to an extendable struct, that will take the
>>>>>>> last u64, when needed.
>>>>>>>
>>>>>>> However, it's still better to share through file descriptors. It's just
>>>>>>> not secure enough the way it's now.
>>>>>>
>>>>>> Is the file descriptor value really a good choice? We just had some
>>>>>> confusion on ring sharing across forks. Not sure using an fd value
>>>>>> is a sane "key" to use across processes.
>>>>>>
>>>>> As I see it, the problem with @mm is that uring is dead-bound to it.
>>>>> For example, a process can create and send uring (e.g. via socket),
>>>>> and then be killed. And that basically means
>>>>> 1. @mm of the process is locked just because of the sent uring
>>>>> instance.
>>>>> 2. a process may have an io_uring, which bound to @mm of another
>>>>> process, even though the layouts may be completely different.
>>>>>
>>>>> File descriptors are different here, because io_uring doesn't know
>>>>> about them, They are controlled by the userspace (send, dup, fork,
>>>>> etc), and don't sabotage all isolation work done in the kernel. A dire
>>>>> example here is stealing io-wq from within a container, which is
>>>>> trivial with global self-made id. I would love to hear, if I am
>>>>> mistaken somewhere.
>>>>>
>>>>> Is there some better option?
>>>>
>>>> OK, so how about this:
>>>>
>>>> - We use the 'fd' as the lookup key. This makes it easy since we can
>>>>   just check if it's a io_uring instance or not, we don't need to do any
>>>>   tracking on the side. It also means that the application asking for
>>>>   sharing must already have some relationship to the process that
>>>>   created the ring.
>>
>> Yeah, that's exactly the point.
>>
>>>>
>>>> - mm/creds must be transferred through the work item. Any SQE done on
>>>>   behalf of io_uring_enter() directly already has that, if punted we
>>>>   must pass the creds and mm. This means we break the static setup of
>>>>   io_wq->mm/creds. It also means that we probably have to add that to
>>>>   io_wq_work, which kind of sucks, but...
>>
>> ehh, juggling mm's... But don't have anything nicer myself.
> 
> We already do juggle mm's, this is no different. A worker potentially
> retain the mm across works if they are the same.
> 
>>> It'd fix Stefan's worry too.
>>>
>>>> I think with that we have a decent setup, that's also safe. I've dropped
>>>> the sharing patches for now, from the 5.6 tree.
>>>
>>> So one concern might be SQPOLL, it'll have to use the ctx creds and mm
>>> as usual. I guess that is ok.
>>>
>>
>> OK. I'll send the patches for the first part now, and take a look at
>> the second one a bit latter if isn't done until then.
> 
> Hang on a second, I'm doing the mm and creds bits right now. I'll push
> that to a branch, if you want to do the actual fd stuff on top of that,
> that would be great.
> 
Sure, should be trivially mergeable.

-- 
Pavel Begunkov


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

  reply	other threads:[~2020-01-27 23:26 UTC|newest]

Thread overview: 33+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-01-23 23:16 [PATCHSET 0/4] Add support for shared io-wq backends Jens Axboe
2020-01-23 23:16 ` [PATCH 1/4] io-wq: make the io_wq ref counted Jens Axboe
2020-01-23 23:16 ` [PATCH 2/4] io-wq: add 'id' to io_wq Jens Axboe
2020-01-23 23:16 ` [PATCH 3/4] io-wq: allow lookup of existing io_wq with given id Jens Axboe
2020-01-24  9:54   ` Stefan Metzmacher
2020-01-24 16:41     ` Jens Axboe
2020-01-23 23:16 ` [PATCH 4/4] io_uring: add support for sharing kernel io-wq workqueue Jens Axboe
2020-01-24  9:51 ` [PATCHSET 0/4] Add support for shared io-wq backends Stefan Metzmacher
2020-01-24 16:43   ` Jens Axboe
2020-01-24 19:14     ` Stefan Metzmacher
2020-01-24 21:37       ` Jens Axboe
2020-01-24 20:34 ` Pavel Begunkov
2020-01-24 21:38   ` Jens Axboe
2020-01-26  1:51 ` Daurnimator
2020-01-26 15:11   ` Pavel Begunkov
2020-01-26 17:00     ` Jens Axboe
2020-01-27 13:29       ` Pavel Begunkov
2020-01-27 13:39         ` Jens Axboe
2020-01-27 14:07           ` Pavel Begunkov
2020-01-27 19:39             ` Pavel Begunkov
2020-01-27 19:45               ` Jens Axboe
2020-01-27 20:33             ` Jens Axboe
2020-01-27 21:45               ` Pavel Begunkov
2020-01-27 22:40                 ` Jens Axboe
2020-01-27 23:00                   ` Jens Axboe
2020-01-27 23:17                     ` Pavel Begunkov
2020-01-27 23:23                       ` Jens Axboe
2020-01-27 23:25                         ` Pavel Begunkov [this message]
2020-01-27 23:38                           ` Jens Axboe
2020-01-28 10:01                             ` Stefan Metzmacher
2020-01-28 10:30                               ` Pavel Begunkov
2020-01-28 10:35                                 ` Stefan Metzmacher
2020-01-28 10:51                                   ` Pavel Begunkov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox