* Re: [PATCH 0/3] readfile(2): a new syscall to make open/read/close faster [not found] ` <20200714065110.GA8047@amd> @ 2020-07-14 8:07 ` Miklos Szeredi 2020-07-14 11:34 ` Pavel Begunkov 0 siblings, 1 reply; 8+ messages in thread From: Miklos Szeredi @ 2020-07-14 8:07 UTC (permalink / raw) To: Pavel Machek Cc: Greg KH, Jan Ziak, Linux API, linux-fsdevel, linux-kernel, linux-kselftest, linux-man, Michael Kerrisk, shuah, Al Viro, io-uring On Tue, Jul 14, 2020 at 8:51 AM Pavel Machek <[email protected]> wrote: > > Hi! > > > > At first, I thought that the proposed system call is capable of > > > reading *multiple* small files using a single system call - which > > > would help increase HDD/SSD queue utilization and increase IOPS (I/O > > > operations per second) - but that isn't the case and the proposed > > > system call can read just a single file. > > > > If you want to do this for multple files, use io_ring, that's what it > > was designed for. I think Jens was going to be adding support for the > > open/read/close pattern to it as well, after some other more pressing > > features/fixes were finished. > > What about... just using io_uring for single file, too? I'm pretty > sure it can be wrapped in a library that is simple to use, avoiding > need for new syscall. Just wondering: is there a plan to add strace support to io_uring? And I don't just mean the syscalls associated with io_uring, but tracing the ring itself. I think that's quite important as io_uring becomes mainstream. Thanks, Miklos ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 0/3] readfile(2): a new syscall to make open/read/close faster 2020-07-14 8:07 ` [PATCH 0/3] readfile(2): a new syscall to make open/read/close faster Miklos Szeredi @ 2020-07-14 11:34 ` Pavel Begunkov 2020-07-14 11:55 ` Miklos Szeredi 0 siblings, 1 reply; 8+ messages in thread From: Pavel Begunkov @ 2020-07-14 11:34 UTC (permalink / raw) To: Miklos Szeredi, Pavel Machek Cc: Greg KH, Jan Ziak, Linux API, linux-fsdevel, linux-kernel, linux-kselftest, linux-man, Michael Kerrisk, shuah, Al Viro, io-uring On 14/07/2020 11:07, Miklos Szeredi wrote: > On Tue, Jul 14, 2020 at 8:51 AM Pavel Machek <[email protected]> wrote: >> >> Hi! >> >>>> At first, I thought that the proposed system call is capable of >>>> reading *multiple* small files using a single system call - which >>>> would help increase HDD/SSD queue utilization and increase IOPS (I/O >>>> operations per second) - but that isn't the case and the proposed >>>> system call can read just a single file. >>> >>> If you want to do this for multple files, use io_ring, that's what it >>> was designed for. I think Jens was going to be adding support for the >>> open/read/close pattern to it as well, after some other more pressing >>> features/fixes were finished. >> >> What about... just using io_uring for single file, too? I'm pretty >> sure it can be wrapped in a library that is simple to use, avoiding >> need for new syscall. > > Just wondering: is there a plan to add strace support to io_uring? > And I don't just mean the syscalls associated with io_uring, but > tracing the ring itself. What kind of support do you mean? io_uring is asynchronous in nature with all intrinsic tracing/debugging/etc. problems of such APIs. And there are a lot of handy trace points, are those not enough? Though, this can be an interesting project to rethink how async APIs are worked with. > > I think that's quite important as io_uring becomes mainstream. -- Pavel Begunkov ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 0/3] readfile(2): a new syscall to make open/read/close faster 2020-07-14 11:34 ` Pavel Begunkov @ 2020-07-14 11:55 ` Miklos Szeredi 2020-07-15 8:31 ` Pavel Begunkov 0 siblings, 1 reply; 8+ messages in thread From: Miklos Szeredi @ 2020-07-14 11:55 UTC (permalink / raw) To: Pavel Begunkov Cc: Pavel Machek, Greg KH, Jan Ziak, Linux API, linux-fsdevel, linux-kernel, linux-kselftest, linux-man, Michael Kerrisk, shuah, Al Viro, io-uring On Tue, Jul 14, 2020 at 1:36 PM Pavel Begunkov <[email protected]> wrote: > > On 14/07/2020 11:07, Miklos Szeredi wrote: > > On Tue, Jul 14, 2020 at 8:51 AM Pavel Machek <[email protected]> wrote: > >> > >> Hi! > >> > >>>> At first, I thought that the proposed system call is capable of > >>>> reading *multiple* small files using a single system call - which > >>>> would help increase HDD/SSD queue utilization and increase IOPS (I/O > >>>> operations per second) - but that isn't the case and the proposed > >>>> system call can read just a single file. > >>> > >>> If you want to do this for multple files, use io_ring, that's what it > >>> was designed for. I think Jens was going to be adding support for the > >>> open/read/close pattern to it as well, after some other more pressing > >>> features/fixes were finished. > >> > >> What about... just using io_uring for single file, too? I'm pretty > >> sure it can be wrapped in a library that is simple to use, avoiding > >> need for new syscall. > > > > Just wondering: is there a plan to add strace support to io_uring? > > And I don't just mean the syscalls associated with io_uring, but > > tracing the ring itself. > > What kind of support do you mean? io_uring is asynchronous in nature > with all intrinsic tracing/debugging/etc. problems of such APIs. > And there are a lot of handy trace points, are those not enough? > > Though, this can be an interesting project to rethink how async > APIs are worked with. Yeah, it's an interesting problem. The uring has the same events, as far as I understand, that are recorded in a multithreaded strace output (syscall entry, syscall exit); nothing more is needed. I do think this needs to be integrated into strace(1), otherwise the usefulness of that tool (which I think is *very* high) would go down drastically as io_uring usage goes up. Thanks, Miklos ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 0/3] readfile(2): a new syscall to make open/read/close faster 2020-07-14 11:55 ` Miklos Szeredi @ 2020-07-15 8:31 ` Pavel Begunkov 2020-07-15 8:41 ` Miklos Szeredi 0 siblings, 1 reply; 8+ messages in thread From: Pavel Begunkov @ 2020-07-15 8:31 UTC (permalink / raw) To: Miklos Szeredi Cc: Pavel Machek, Greg KH, Jan Ziak, Linux API, linux-fsdevel, linux-kernel, linux-kselftest, linux-man, Michael Kerrisk, shuah, Al Viro, io-uring On 14/07/2020 14:55, Miklos Szeredi wrote: > On Tue, Jul 14, 2020 at 1:36 PM Pavel Begunkov <[email protected]> wrote: >> >> On 14/07/2020 11:07, Miklos Szeredi wrote: >>> On Tue, Jul 14, 2020 at 8:51 AM Pavel Machek <[email protected]> wrote: >>>> >>>> Hi! >>>> >>>>>> At first, I thought that the proposed system call is capable of >>>>>> reading *multiple* small files using a single system call - which >>>>>> would help increase HDD/SSD queue utilization and increase IOPS (I/O >>>>>> operations per second) - but that isn't the case and the proposed >>>>>> system call can read just a single file. >>>>> >>>>> If you want to do this for multple files, use io_ring, that's what it >>>>> was designed for. I think Jens was going to be adding support for the >>>>> open/read/close pattern to it as well, after some other more pressing >>>>> features/fixes were finished. >>>> >>>> What about... just using io_uring for single file, too? I'm pretty >>>> sure it can be wrapped in a library that is simple to use, avoiding >>>> need for new syscall. >>> >>> Just wondering: is there a plan to add strace support to io_uring? >>> And I don't just mean the syscalls associated with io_uring, but >>> tracing the ring itself. >> >> What kind of support do you mean? io_uring is asynchronous in nature >> with all intrinsic tracing/debugging/etc. problems of such APIs. >> And there are a lot of handy trace points, are those not enough? >> >> Though, this can be an interesting project to rethink how async >> APIs are worked with. > > Yeah, it's an interesting problem. The uring has the same events, as > far as I understand, that are recorded in a multithreaded strace > output (syscall entry, syscall exit); nothing more is needed> > I do think this needs to be integrated into strace(1), otherwise the > usefulness of that tool (which I think is *very* high) would go down > drastically as io_uring usage goes up. Not touching the topic of usefulness of strace + io_uring, but I'd rather have a tool that solves a problem, than a problem that created and honed for a tool. -- Pavel Begunkov ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 0/3] readfile(2): a new syscall to make open/read/close faster 2020-07-15 8:31 ` Pavel Begunkov @ 2020-07-15 8:41 ` Miklos Szeredi 2020-07-15 8:49 ` Pavel Begunkov 0 siblings, 1 reply; 8+ messages in thread From: Miklos Szeredi @ 2020-07-15 8:41 UTC (permalink / raw) To: Pavel Begunkov Cc: Pavel Machek, Greg KH, Jan Ziak, Linux API, linux-fsdevel, linux-kernel, linux-kselftest, linux-man, Michael Kerrisk, shuah, Al Viro, io-uring On Wed, Jul 15, 2020 at 10:33 AM Pavel Begunkov <[email protected]> wrote: > > On 14/07/2020 14:55, Miklos Szeredi wrote: > > On Tue, Jul 14, 2020 at 1:36 PM Pavel Begunkov <[email protected]> wrote: > >> > >> On 14/07/2020 11:07, Miklos Szeredi wrote: > >>> On Tue, Jul 14, 2020 at 8:51 AM Pavel Machek <[email protected]> wrote: > >>>> > >>>> Hi! > >>>> > >>>>>> At first, I thought that the proposed system call is capable of > >>>>>> reading *multiple* small files using a single system call - which > >>>>>> would help increase HDD/SSD queue utilization and increase IOPS (I/O > >>>>>> operations per second) - but that isn't the case and the proposed > >>>>>> system call can read just a single file. > >>>>> > >>>>> If you want to do this for multple files, use io_ring, that's what it > >>>>> was designed for. I think Jens was going to be adding support for the > >>>>> open/read/close pattern to it as well, after some other more pressing > >>>>> features/fixes were finished. > >>>> > >>>> What about... just using io_uring for single file, too? I'm pretty > >>>> sure it can be wrapped in a library that is simple to use, avoiding > >>>> need for new syscall. > >>> > >>> Just wondering: is there a plan to add strace support to io_uring? > >>> And I don't just mean the syscalls associated with io_uring, but > >>> tracing the ring itself. > >> > >> What kind of support do you mean? io_uring is asynchronous in nature > >> with all intrinsic tracing/debugging/etc. problems of such APIs. > >> And there are a lot of handy trace points, are those not enough? > >> > >> Though, this can be an interesting project to rethink how async > >> APIs are worked with. > > > > Yeah, it's an interesting problem. The uring has the same events, as > > far as I understand, that are recorded in a multithreaded strace > > output (syscall entry, syscall exit); nothing more is needed> > > I do think this needs to be integrated into strace(1), otherwise the > > usefulness of that tool (which I think is *very* high) would go down > > drastically as io_uring usage goes up. > > Not touching the topic of usefulness of strace + io_uring, but I'd rather > have a tool that solves a problem, than a problem that created and honed > for a tool. Sorry, I'm not getting the metaphor. Can you please elaborate? Thanks, Miklos ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 0/3] readfile(2): a new syscall to make open/read/close faster 2020-07-15 8:41 ` Miklos Szeredi @ 2020-07-15 8:49 ` Pavel Begunkov 2020-07-15 9:00 ` Pavel Begunkov 0 siblings, 1 reply; 8+ messages in thread From: Pavel Begunkov @ 2020-07-15 8:49 UTC (permalink / raw) To: Miklos Szeredi Cc: Pavel Machek, Greg KH, Jan Ziak, Linux API, linux-fsdevel, linux-kernel, linux-kselftest, linux-man, Michael Kerrisk, shuah, Al Viro, io-uring On 15/07/2020 11:41, Miklos Szeredi wrote: > On Wed, Jul 15, 2020 at 10:33 AM Pavel Begunkov <[email protected]> wrote: >> >> On 14/07/2020 14:55, Miklos Szeredi wrote: >>> On Tue, Jul 14, 2020 at 1:36 PM Pavel Begunkov <[email protected]> wrote: >>>> >>>> On 14/07/2020 11:07, Miklos Szeredi wrote: >>>>> On Tue, Jul 14, 2020 at 8:51 AM Pavel Machek <[email protected]> wrote: >>>>>> >>>>>> Hi! >>>>>> >>>>>>>> At first, I thought that the proposed system call is capable of >>>>>>>> reading *multiple* small files using a single system call - which >>>>>>>> would help increase HDD/SSD queue utilization and increase IOPS (I/O >>>>>>>> operations per second) - but that isn't the case and the proposed >>>>>>>> system call can read just a single file. >>>>>>> >>>>>>> If you want to do this for multple files, use io_ring, that's what it >>>>>>> was designed for. I think Jens was going to be adding support for the >>>>>>> open/read/close pattern to it as well, after some other more pressing >>>>>>> features/fixes were finished. >>>>>> >>>>>> What about... just using io_uring for single file, too? I'm pretty >>>>>> sure it can be wrapped in a library that is simple to use, avoiding >>>>>> need for new syscall. >>>>> >>>>> Just wondering: is there a plan to add strace support to io_uring? >>>>> And I don't just mean the syscalls associated with io_uring, but >>>>> tracing the ring itself. >>>> >>>> What kind of support do you mean? io_uring is asynchronous in nature >>>> with all intrinsic tracing/debugging/etc. problems of such APIs. >>>> And there are a lot of handy trace points, are those not enough? >>>> >>>> Though, this can be an interesting project to rethink how async >>>> APIs are worked with. >>> >>> Yeah, it's an interesting problem. The uring has the same events, as >>> far as I understand, that are recorded in a multithreaded strace >>> output (syscall entry, syscall exit); nothing more is needed> >>> I do think this needs to be integrated into strace(1), otherwise the >>> usefulness of that tool (which I think is *very* high) would go down >>> drastically as io_uring usage goes up. >> >> Not touching the topic of usefulness of strace + io_uring, but I'd rather >> have a tool that solves a problem, than a problem that created and honed >> for a tool. > > Sorry, I'm not getting the metaphor. Can you please elaborate? Sure, I mean _if_ there are tools that conceptually suit better, I'd prefer to work with them, then trying to shove a new and possibly alien infrastructure into strace. But my knowledge of strace is very limited, so can't tell whether that's the case. E.g. can it utilise static trace points? -- Pavel Begunkov ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 0/3] readfile(2): a new syscall to make open/read/close faster 2020-07-15 8:49 ` Pavel Begunkov @ 2020-07-15 9:00 ` Pavel Begunkov 2020-07-15 11:17 ` Miklos Szeredi 0 siblings, 1 reply; 8+ messages in thread From: Pavel Begunkov @ 2020-07-15 9:00 UTC (permalink / raw) To: Miklos Szeredi Cc: Pavel Machek, Greg KH, Jan Ziak, Linux API, linux-fsdevel, linux-kernel, linux-kselftest, linux-man, Michael Kerrisk, shuah, Al Viro, io-uring On 15/07/2020 11:49, Pavel Begunkov wrote: > On 15/07/2020 11:41, Miklos Szeredi wrote: >> On Wed, Jul 15, 2020 at 10:33 AM Pavel Begunkov <[email protected]> wrote: >>> >>> On 14/07/2020 14:55, Miklos Szeredi wrote: >>>> On Tue, Jul 14, 2020 at 1:36 PM Pavel Begunkov <[email protected]> wrote: >>>>> >>>>> On 14/07/2020 11:07, Miklos Szeredi wrote: >>>>>> On Tue, Jul 14, 2020 at 8:51 AM Pavel Machek <[email protected]> wrote: >>>>>>> >>>>>>> Hi! >>>>>>> >>>>>>>>> At first, I thought that the proposed system call is capable of >>>>>>>>> reading *multiple* small files using a single system call - which >>>>>>>>> would help increase HDD/SSD queue utilization and increase IOPS (I/O >>>>>>>>> operations per second) - but that isn't the case and the proposed >>>>>>>>> system call can read just a single file. >>>>>>>> >>>>>>>> If you want to do this for multple files, use io_ring, that's what it >>>>>>>> was designed for. I think Jens was going to be adding support for the >>>>>>>> open/read/close pattern to it as well, after some other more pressing >>>>>>>> features/fixes were finished. >>>>>>> >>>>>>> What about... just using io_uring for single file, too? I'm pretty >>>>>>> sure it can be wrapped in a library that is simple to use, avoiding >>>>>>> need for new syscall. >>>>>> >>>>>> Just wondering: is there a plan to add strace support to io_uring? >>>>>> And I don't just mean the syscalls associated with io_uring, but >>>>>> tracing the ring itself. >>>>> >>>>> What kind of support do you mean? io_uring is asynchronous in nature >>>>> with all intrinsic tracing/debugging/etc. problems of such APIs. >>>>> And there are a lot of handy trace points, are those not enough? >>>>> >>>>> Though, this can be an interesting project to rethink how async >>>>> APIs are worked with. >>>> >>>> Yeah, it's an interesting problem. The uring has the same events, as >>>> far as I understand, that are recorded in a multithreaded strace >>>> output (syscall entry, syscall exit); nothing more is needed> >>>> I do think this needs to be integrated into strace(1), otherwise the >>>> usefulness of that tool (which I think is *very* high) would go down >>>> drastically as io_uring usage goes up. >>> >>> Not touching the topic of usefulness of strace + io_uring, but I'd rather >>> have a tool that solves a problem, than a problem that created and honed >>> for a tool. >> >> Sorry, I'm not getting the metaphor. Can you please elaborate? > > Sure, I mean _if_ there are tools that conceptually suit better, I'd > prefer to work with them, then trying to shove a new and possibly alien > infrastructure into strace. > > But my knowledge of strace is very limited, so can't tell whether that's > the case. E.g. can it utilise static trace points? I think, if you're going to push this idea, we should start a new thread CC'ing strace devs. -- Pavel Begunkov ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 0/3] readfile(2): a new syscall to make open/read/close faster 2020-07-15 9:00 ` Pavel Begunkov @ 2020-07-15 11:17 ` Miklos Szeredi 0 siblings, 0 replies; 8+ messages in thread From: Miklos Szeredi @ 2020-07-15 11:17 UTC (permalink / raw) To: Pavel Begunkov Cc: Pavel Machek, Greg KH, Jan Ziak, Linux API, linux-fsdevel, linux-kernel, linux-kselftest, linux-man, Michael Kerrisk, shuah, Al Viro, io-uring On Wed, Jul 15, 2020 at 11:02 AM Pavel Begunkov <[email protected]> wrote: > I think, if you're going to push this idea, we should start a new thread > CC'ing strace devs. Makes sense. I've pruned the Cc list, so here's the link for reference: https://lore.kernel.org/linux-fsdevel/CAJfpegu3EwbBFTSJiPhm7eMyTK2MzijLUp1gcboOo3meMF_+Qg@mail.gmail.com/T/#u Thanks, Miklos ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2020-07-15 11:17 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <CAODFU0q6CrUB_LkSdrbp5TQ4Jm6Sw=ZepZwD-B7-aFudsOvsig@mail.gmail.com>
[not found] ` <[email protected]>
[not found] ` <20200714065110.GA8047@amd>
2020-07-14 8:07 ` [PATCH 0/3] readfile(2): a new syscall to make open/read/close faster Miklos Szeredi
2020-07-14 11:34 ` Pavel Begunkov
2020-07-14 11:55 ` Miklos Szeredi
2020-07-15 8:31 ` Pavel Begunkov
2020-07-15 8:41 ` Miklos Szeredi
2020-07-15 8:49 ` Pavel Begunkov
2020-07-15 9:00 ` Pavel Begunkov
2020-07-15 11:17 ` Miklos Szeredi
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox