Re: Postgres, fsync, and OSs (specifically linux)

From: Dmitry Dolgov <9erthalion6(at)gmail(dot)com>
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: Thomas Munro <thomas(dot)munro(at)enterprisedb(dot)com>, Craig Ringer <craig(at)2ndquadrant(dot)com>, Heikki Linnakangas <hlinnaka(at)iki(dot)fi>, Robert Haas <robertmhaas(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Postgres, fsync, and OSs (specifically linux)
Date: 2018-05-22 18:54:46
Message-ID: CA+q6zcV+KKXi8CHci-y+cJm_Bpmy-1LRtXLFpT2+4Y_tjTYikQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> On 22 May 2018 at 18:47, Andres Freund <andres(at)anarazel(dot)de> wrote:
> On 2018-05-22 08:57:18 -0700, Andres Freund wrote:
>> Hi,
>>
>>
>> On 2018-05-22 17:37:28 +0200, Dmitry Dolgov wrote:
>> > Thanks for the patch. Out of curiosity I tried to play with it a bit.
>>
>> Thanks.
>>
>>
>> > `pgbench -i -s 100` actually hang on my machine, because the
>> > copy process ended up with waiting after `pg_uds_send_with_fd`
>> > had
>>
>> Hm, that had worked at some point...
>>
>>
>> > errno == EWOULDBLOCK || errno == EAGAIN
>> >
>> > as well as the checkpointer process.
>>
>> What do you mean with that latest sentence?

To investigate what's happening I attached with gdb to two processes, COPY
process from pgbench and checkpointer (since I assumed it may be involved).
Both were waiting in WaitLatchOrSocket right after SendFsyncRequest.

>> > Looks like with the default
>> > configuration and `max_wal_size=1GB` it writes more than reads to a
>> > socket, and a buffer eventually becomes full.
>>
>> That's intended to then wake up the checkpointer immediately, so it can
>> absorb the requests. So something isn't right yet.
>
> Doesn't hang here, but it's way too slow.

Yep, in my case it was also getting slower, but eventually hang.

> Reason for that is that I've wrongly resolved a merge conflict. Attached is a
> fixup patch - does that address the issue for you?

Hm...is it a correct patch? I see the same committed in
8c3debbbf61892dabd8b6f3f8d55e600a7901f2b, so I can't really apply it.

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2018-05-22 18:55:54 Re: [PATCH] (Windows) psql echoes password when reading from pipe
Previous Message Andres Freund 2018-05-22 18:52:06 Re: future of contrib/xml2 and xslt processing