Re: Phoronix pts fio io_uring test regression report on upstream v6.1 and v5.15

[Saeed Mirzamohammadi <saeed.mirzamohammadi@oracle.com>]

Hi Jens,

> On Jan 26, 2023, at 10:35 AM, Jens Axboe <axboe@kernel.dk> wrote:
> 
> On 1/26/23 11:04 AM, Saeed Mirzamohammadi wrote:
>> Hi Jens,
>> 
>>> On Jan 25, 2023, at 4:28 PM, Jens Axboe <axboe@kernel.dk> wrote:
>>> 
>>> On 1/25/23 5:22?PM, Saeed Mirzamohammadi wrote:
>>>> Hi Jens,
>>>> 
>>>> I applied your patch (with a minor conflict in xfs_file_open() since FMODE_BUF_WASYNC isn't in v5.15) and did the same series of tests on the v5.15 kernel. All the io_uring benchmarks regressed 20-45% after it. I haven't tested on v6.1 yet.
>>> 
>>> It should basically make the behavior the same as before once you apply
>>> the patch, so please pass on the patch that you applied for 5.15 so we
>>> can take a closer look.
>> 
>> Attached the patch.
> 
> I tested the upstream variant, and it does what it's supposed to and
> gets parallel writes on O_DIRECT. Unpatched, any dio write results in:
> 
>             fio-566     [000] .....   131.071108: io_uring_queue_async_work: ring 00000000706cb6c0, request 00000000b21691c4, user_data 0xaaab0e8e4c00, opcode WRITE, flags 0xe0040000, hashed queue, work 000000002c5aeb79
> 
> and after the patch:
> 
>             fio-376     [000] .....    24.590994: io_uring_queue_async_work: ring 000000007bdb650a, request 000000006b5350e0, user_data 0xaaab1b3e3c00, opcode WRITE, flags 0xe0040000, normal queue, work 00000000e3e81955
> 

Thanks for looking into this.

> where the hashed queued is serialized based on the inode, and the normal
> queue is not (eg they run in parallel).
> 
> As mentioned, the fio job being used isn't representative of anything
> that should actually be run, the async flag really only exists for
> experimentation. Do you have a real workload that is seeing a regression?
> If yes, does that real workload change performance with the patch?

I tested without the async flag but didn’t see any change in the performance.

I haven’t tested any real workload yet. I’ll share with you if I noticed anything.

Thanks,
Saeed

p.s. I experienced multipathd issues with the patch that I had to work through. Never without the patch.

> 
> -- 
> Jens Axboe