* Question about sendfile
@ 2021-07-03 10:47 Hao Xu
2021-07-07 14:16 ` Pavel Begunkov
0 siblings, 1 reply; 6+ messages in thread
From: Hao Xu @ 2021-07-03 10:47 UTC (permalink / raw)
To: Pavel Begunkov; +Cc: Jens Axboe, io-uring, Joseph Qi
Hi Pavel,
I found this mail about sendfile in the maillist, may I ask why it's not
good to have one pipe each for a io-wq thread.
https://lore.kernel.org/io-uring/[email protected]/
Thanks,
Hao
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Question about sendfile 2021-07-03 10:47 Question about sendfile Hao Xu @ 2021-07-07 14:16 ` Pavel Begunkov 2021-07-14 3:50 ` Hao Xu 2021-11-26 8:50 ` Hao Xu 0 siblings, 2 replies; 6+ messages in thread From: Pavel Begunkov @ 2021-07-07 14:16 UTC (permalink / raw) To: Hao Xu; +Cc: Jens Axboe, io-uring, Joseph Qi On 7/3/21 11:47 AM, Hao Xu wrote: > Hi Pavel, > I found this mail about sendfile in the maillist, may I ask why it's not > good to have one pipe each for a io-wq thread. > https://lore.kernel.org/io-uring/[email protected]/ IIRC, it's one page allocated for each such task, which is bearable but don't like yet another chunk of uncontrollable implicit state. If there not a bunch of active workers, IFAIK there is no way to force them to drop their pipes. I also don't remember the restrictions on the sendfile and what's with the eternal question of "what to do if the write part of sendfile has failed". Though, workers are now much more alike to user threads, so there should be less of concern. And even though my gut feeling don't like them, it may actually be useful. Do you have a good use case where explicit pipes don't work well? -- Pavel Begunkov ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Question about sendfile 2021-07-07 14:16 ` Pavel Begunkov @ 2021-07-14 3:50 ` Hao Xu 2021-11-26 8:50 ` Hao Xu 1 sibling, 0 replies; 6+ messages in thread From: Hao Xu @ 2021-07-14 3:50 UTC (permalink / raw) To: Pavel Begunkov; +Cc: Jens Axboe, io-uring, Joseph Qi 在 2021/7/7 下午10:16, Pavel Begunkov 写道: > On 7/3/21 11:47 AM, Hao Xu wrote: >> Hi Pavel, >> I found this mail about sendfile in the maillist, may I ask why it's not >> good to have one pipe each for a io-wq thread. >> https://lore.kernel.org/io-uring/[email protected]/ > > IIRC, it's one page allocated for each such task, which is bearable but > don't like yet another chunk of uncontrollable implicit state. If there > not a bunch of active workers, IFAIK there is no way to force them to > drop their pipes. > > I also don't remember the restrictions on the sendfile and what's with > the eternal question of "what to do if the write part of sendfile has > failed". I haven't dig into it deeply, will do some investigation. > > Though, workers are now much more alike to user threads, so there > should be less of concern. And even though my gut feeling don't like > them, it may actually be useful. Do you have a good use case where > explicit pipes don't work well? The thing is two linked splice sqes may be cut off in shared sqthread case. > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Question about sendfile 2021-07-07 14:16 ` Pavel Begunkov 2021-07-14 3:50 ` Hao Xu @ 2021-11-26 8:50 ` Hao Xu 2021-12-03 16:03 ` Pavel Begunkov 1 sibling, 1 reply; 6+ messages in thread From: Hao Xu @ 2021-11-26 8:50 UTC (permalink / raw) To: Pavel Begunkov; +Cc: Jens Axboe, io-uring, Joseph Qi 在 2021/7/7 下午10:16, Pavel Begunkov 写道: > On 7/3/21 11:47 AM, Hao Xu wrote: >> Hi Pavel, >> I found this mail about sendfile in the maillist, may I ask why it's not >> good to have one pipe each for a io-wq thread. >> https://lore.kernel.org/io-uring/[email protected]/ > > IIRC, it's one page allocated for each such task, which is bearable but > don't like yet another chunk of uncontrollable implicit state. If there > not a bunch of active workers, IFAIK there is no way to force them to > drop their pipes. > > I also don't remember the restrictions on the sendfile and what's with > the eternal question of "what to do if the write part of sendfile has > failed". Hi Pavel, Could you explain this question a little bit.., is there any special concern? What I thought is sendfile does what it does,when it fails, it will return -1 and errno is set appropriately. > > Though, workers are now much more alike to user threads, so there > should be less of concern. And even though my gut feeling don't like > them, it may actually be useful. Do you have a good use case where > explicit pipes don't work well? > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Question about sendfile 2021-11-26 8:50 ` Hao Xu @ 2021-12-03 16:03 ` Pavel Begunkov 2021-12-05 15:21 ` Hao Xu 0 siblings, 1 reply; 6+ messages in thread From: Pavel Begunkov @ 2021-12-03 16:03 UTC (permalink / raw) To: Hao Xu; +Cc: Jens Axboe, io-uring, Joseph Qi On 11/26/21 08:50, Hao Xu wrote: > 在 2021/7/7 下午10:16, Pavel Begunkov 写道: >> On 7/3/21 11:47 AM, Hao Xu wrote: >>> Hi Pavel, >>> I found this mail about sendfile in the maillist, may I ask why it's not >>> good to have one pipe each for a io-wq thread. >>> https://lore.kernel.org/io-uring/[email protected]/ >> >> IIRC, it's one page allocated for each such task, which is bearable but >> don't like yet another chunk of uncontrollable implicit state. If there >> not a bunch of active workers, IFAIK there is no way to force them to >> drop their pipes. >> >> I also don't remember the restrictions on the sendfile and what's with >> the eternal question of "what to do if the write part of sendfile has >> failed". > Hi Pavel, > Could you explain this question a little bit.., is there any special > concern? What I thought is sendfile does what it does,when it fails, > it will return -1 and errno is set appropriately. I don't have much concern about this one, though interesting how it was solved and whether you need to know the issuing task to handle errors. I didn't like more having uncontrollable memory, i.e. a pipe per worker that used sendfile (IIRC it keeps 1 page), and no way to reuse the memory or release it. In other words, a sendfile request chooses to which worker it goes randomly. E.g. First sendfile may go to worker 1 leaving 1 page allocated. The second sendfile goes to worker 2, so after we have 2 pages allocated, an so on. At some point you have N pages, where any particular one may likely be rarely used. Please correct me if I forgot how it works and wrong here. >> Though, workers are now much more alike to user threads, so there >> should be less of concern. And even though my gut feeling don't like >> them, it may actually be useful. Do you have a good use case where >> explicit pipes don't work well? -- Pavel Begunkov ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Question about sendfile 2021-12-03 16:03 ` Pavel Begunkov @ 2021-12-05 15:21 ` Hao Xu 0 siblings, 0 replies; 6+ messages in thread From: Hao Xu @ 2021-12-05 15:21 UTC (permalink / raw) To: Pavel Begunkov; +Cc: Jens Axboe, io-uring, Joseph Qi 在 2021/12/4 上午12:03, Pavel Begunkov 写道: > On 11/26/21 08:50, Hao Xu wrote: >> 在 2021/7/7 下午10:16, Pavel Begunkov 写道: >>> On 7/3/21 11:47 AM, Hao Xu wrote: >>>> Hi Pavel, >>>> I found this mail about sendfile in the maillist, may I ask why it's >>>> not >>>> good to have one pipe each for a io-wq thread. >>>> https://lore.kernel.org/io-uring/[email protected]/ >>>> >>> >>> IIRC, it's one page allocated for each such task, which is bearable but >>> don't like yet another chunk of uncontrollable implicit state. If there >>> not a bunch of active workers, IFAIK there is no way to force them to >>> drop their pipes. >>> >>> I also don't remember the restrictions on the sendfile and what's with >>> the eternal question of "what to do if the write part of sendfile has >>> failed". >> Hi Pavel, >> Could you explain this question a little bit.., is there any special >> concern? What I thought is sendfile does what it does,when it fails, >> it will return -1 and errno is set appropriately. > > I don't have much concern about this one, though interesting how > it was solved and whether you need to know the issuing task to > handle errors. > > I didn't like more having uncontrollable memory, i.e. a pipe per > worker that used sendfile (IIRC it keeps 1 page), and no way to > reuse the memory or release it. In other words, a sendfile request > chooses to which worker it goes randomly. E.g. First sendfile may go > to worker 1 leaving 1 page allocated. The second sendfile goes to > worker 2, so after we have 2 pages allocated, an so on. At some > point you have N pages, where any particular one may likely be > rarely used. I'm not sure when the pipe is freed(seems it won't be freed after sendfile call and it is reused). If it won't be freed automatically we can manually free it when a worker completes a sendfile work. I think in normal cases, a user cannot and shouldn't visit the internal pipe after senfile is done no matter it succeeds or fails, which means we can free the pipe at that time. Not 100% sure but probably.. > > Please correct me if I forgot how it works and wrong here. > >>> Though, workers are now much more alike to user threads, so there >>> should be less of concern. And even though my gut feeling don't like >>> them, it may actually be useful. Do you have a good use case where >>> explicit pipes don't work well? > ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2021-12-05 15:23 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2021-07-03 10:47 Question about sendfile Hao Xu 2021-07-07 14:16 ` Pavel Begunkov 2021-07-14 3:50 ` Hao Xu 2021-11-26 8:50 ` Hao Xu 2021-12-03 16:03 ` Pavel Begunkov 2021-12-05 15:21 ` Hao Xu
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox