public inbox for [email protected]
 help / color / mirror / Atom feed
From: Pavel Begunkov <[email protected]>
To: Bijan Mottahedeh <[email protected]>, [email protected]
Cc: [email protected]
Subject: Re: [PATCH 4/8] io_uring: implement fixed buffers registration similar to fixed files
Date: Mon, 16 Nov 2020 23:09:25 +0000	[thread overview]
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>

On 16/11/2020 21:24, Bijan Mottahedeh wrote:
> On 11/15/2020 5:33 AM, Pavel Begunkov wrote:
>> On 12/11/2020 23:00, Bijan Mottahedeh wrote:
>>> Apply fixed_rsrc functionality for fixed buffers support.
>>
>> I don't get it, requests with fixed files take a ref to a node (see
>> fixed_file_refs) and put it on free, but I don't see anything similar
>> here. Did you work around it somehow?
> 
> No that's my oversight.  I think I wrongfully assumed that io_import_*fixed() would take care of that.
> 
> Should I basically do something similar to io_file_get()/io_put_file()?

If done in a dumb way, that'd mean extra pair of percpu get/put
and +8B in io_kiocb. Frankly, I don't like that idea.

However, if you don't split paths and make fixed_file_ref_node to
supports all types of resources at the same time, it should be
bearable. I.e. register removals of both types to a single node,
and use ->fixed_file_refs for all request's resources.
So you don't grow io_kiocb and do maximum one percpu_ref_get/put()
pair per request.

I'll send a small patch preparing grounds, because there is actually
another nasty thing from past that needs to be reworked.

> 
> io_import_fixed()
> io_import_iovec_fixed()
> -> io_buf_get()
> 
> io_dismantle_io()
> -> io_put_buf()
> 
>>
>> That's not critical for this particular patch as you still do full
>> quisce in __io_uring_register(), but IIRC was essential for
>> update/remove requests.
> 
> That's something I'm not clear about.  Currently we quiesce for the following cases:
> 
>         case IORING_UNREGISTER_FILES:
>         case IORING_REGISTER_FILES_UPDATE:
>         case IORING_REGISTER_BUFFERS_UPDATE:

static bool io_register_op_must_quiesce(int op)
{
	switch (op) {
	case IORING_UNREGISTER_FILES:
	case IORING_REGISTER_FILES_UPDATE:
	case IORING_REGISTER_PROBE:
	case IORING_REGISTER_PERSONALITY:
	case IORING_UNREGISTER_PERSONALITY:
		return false;
	default:
		return true;
	}
}

It returns _false_ for these cases, so _doesn't_ quiesce for them.

> 
> I had assume I have to add IORING_UNREGISTER_BUFFERS as well.  But above, do we in fact the quiesce give the ref counts?
> 
> Are you ok with the rest of the patches or should I address anything else?

io_import_fixed() currently can be called twice, and that would give
you 2 different bvecs. Hence after removing full quisce io_read()
retrying short reads will probably be able to partially read into 2
different buffers. That really have to be fixed.


I haven't looked the patchset properly yet. I'll reply to the
cover-letter + a small comment below


>>>   static ssize_t io_import_fixed(struct io_kiocb *req, int rw,
>>>                      struct iov_iter *iter)
>>>   {
>>> @@ -2959,10 +2982,15 @@ static ssize_t io_import_fixed(struct io_kiocb *req, int rw,
>>>       size_t offset;
>>>       u64 buf_addr;
>>>   +    /* attempt to use fixed buffers without having provided iovecs */
>>> +    if (unlikely(!ctx->buf_data))
>>> +        return -EFAULT;

I removed it for files,
because (ctx->buf_data) IFF (ctx->nr_user_bufs == 0),
so the following ctx->nr_user_bufs check is enough.

>>> +
>>> +    buf_index = req->buf_index;
>>>       if (unlikely(buf_index >= ctx->nr_user_bufs))
>>>           return -EFAULT;
>>>       index = array_index_nospec(buf_index, ctx->nr_user_bufs);
>>> -    imu = &ctx->user_bufs[index];
>>> +    imu = io_buf_from_index(ctx, index);
>>>       buf_addr = req->rw.addr;
>>>         /* overflow */
>>> @@ -8167,28 +8195,73 @@ static unsigned long ring_pages(unsigned sq_entries, unsigned cq_entries)
>>>       return pages;
>>>   }


-- 
Pavel Begunkov

  reply	other threads:[~2020-11-16 23:12 UTC|newest]

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-11-12 23:00 [PATCH 0/8] io_uring: buffer registration enhancements Bijan Mottahedeh
2020-11-12 23:00 ` [PATCH 1/8] io_uring: modularize io_sqe_buffer_register Bijan Mottahedeh
2020-11-12 23:00 ` [PATCH 2/8] io_uring: modularize io_sqe_buffers_register Bijan Mottahedeh
2020-11-12 23:00 ` [PATCH 3/8] io_uring: generalize fixed file functionality Bijan Mottahedeh
2020-11-12 23:00 ` [PATCH 4/8] io_uring: implement fixed buffers registration similar to fixed files Bijan Mottahedeh
2020-11-15 13:33   ` Pavel Begunkov
2020-11-16 21:24     ` Bijan Mottahedeh
2020-11-16 23:09       ` Pavel Begunkov [this message]
2020-11-17  0:41         ` Bijan Mottahedeh
2020-11-12 23:00 ` [PATCH 5/8] io_uring: generalize files_update functionlity to rsrc_update Bijan Mottahedeh
2020-11-12 23:00 ` [PATCH 6/8] io_uring: support buffer registration updates Bijan Mottahedeh
2020-11-18 20:17   ` Pavel Begunkov
2020-12-09  0:42     ` Bijan Mottahedeh
2020-11-12 23:00 ` [PATCH 7/8] io_uring: support readv/writev with fixed buffers Bijan Mottahedeh
2020-11-17 11:04   ` Pavel Begunkov
2020-11-17 22:59     ` Bijan Mottahedeh
2020-11-18  9:14       ` Pavel Begunkov
2020-11-18 20:12       ` Pavel Begunkov
     [not found]         ` <[email protected]>
     [not found]           ` <[email protected]>
2020-11-19 19:27             ` Bijan Mottahedeh
2020-11-12 23:00 ` [PATCH 8/8] io_uring: support buffer registration sharing Bijan Mottahedeh
2020-11-16 23:28 ` [PATCH 0/8] io_uring: buffer registration enhancements Pavel Begunkov
2020-11-17  0:21   ` Bijan Mottahedeh

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    [email protected] \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox