Skip site navigation (1) Skip section navigation (2)

Re: BUG #7534: walreceiver takes long time to detect n/w breakdown

From: Heikki Linnakangas <hlinnakangas(at)vmware(dot)com>
To: Amit kapila <amit(dot)kapila(at)huawei(dot)com>
Cc: Fujii Masao <masao(dot)fujii(at)gmail(dot)com>, "pgsql-bugs(at)postgresql(dot)org" <pgsql-bugs(at)postgresql(dot)org>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: BUG #7534: walreceiver takes long time to detect n/w breakdown
Date: 2012-10-10 15:44:34
Message-ID: 507597E2.8090701@vmware.com (view raw or flat)
Thread:
Lists: pgsql-bugspgsql-hackers
On 04.10.2012 13:12, Amit kapila wrote:
> Following changes are done to support replication timeout in sender as well as receiver:
>
> 1. One new configuration parameter wal_receiver_timeout is added to detect timeout at receiver task.
> 2. Existing parameter replication_timeout is renamed to wal_sender_timeout.

Ok. The other option would be to have just one GUC, I'm open to 
bikeshedding on this one. On one hand, there's no reason the timeouts 
have to the same, so it would be nice to have separate settings, but on 
the other hand, I can't imagine a case where a single setting wouldn't 
work just as well.

> 3. Now PrimaryKeepaliveMessage structure is modified to add one more field to indicate whether keep-alive is of type 'r' (i.e.
>      reply) or 'h' (i.e. heart-beat).
> 4. Now the keep-alive message from sender will be sent to standby if it was idle for more than or equal to half of wal_sender_timeout.
>      In this case it will send keep-alive of type 'h'.
> 5. Once the standby receiver a keep-alive, it needs to send an immediate reply to primary to indicate connection is alive.
> 6. Now Reply message to send wal offset and Feedback message to send oldest transaction are merged into single Reply message.
>      So now the structure StandbyReplyMessage is changed to add two more fields as xmin and epoch. Also StandbyHSFeedbackMessage
>      structure is changed to remove xmin and epoch fields (as these are moved to StandbyReplyMessage).
> 7. Because of changes as in step-6, once receiver task receives some data from primary then it will only send Reply Message.

Oh I see. That's not what I meant by combining the keep-alive and hs 
feedback messages, I imagined that the hearbeats would *also* use the 
same message type. Ie. there would be only a single message type from 
standby to primary, used for:

1. updating the receive/apply pointer
2. HS feedback
3. for pinging the server when wal_receiver_timeout is approaching
4. to reply to to pings from the server.

Since we didn't quite achieve that, it seems best leave out this merging 
of reply and HS feedback message types, to keep the patch small. We 
might still want to do that, but better do that as a separate patch.

> 8. Same Reply message is sent in step-5 and step-7 but incase of step-5, then reply is sent immediately but incase of step-7, reply is sent
>       if wal_receiver_status_interval has lapsed (this part is same as earlier).
> 9. Similar to sender, if receiver finds itself idle for more than or equal to half of configured wal_receiver_timeout, then it will send the
>       hot-standby heartbeat. This heart-beat has been modified to send only sendTime.
> 10. Once sender task receiver heart-beat message from standby then it sends back the reply immediately. In this keep-alive message is
>         sent of type 'r'.
> 11. If even after wal_sender_timeout no message received from standby then it will be considered as network break at sender task.
> 12. If even after wal_receiver_timeout no message received from primary then it will be considered as network break at receiver task.

Attached is an updated patch. I reverted the merging of message types 
and fixed a bunch of cosmetic issues. There was one bug: in the main 
loop of walreceiver, you send the "ping" message on every wakeup after 
enough time has passed since last reception. That means that if the 
server doesn't reply promptly, you send a new ping message every 100 ms 
(NAPTIME_PER_CYCLE), until it gets a reply. Walsender had the same 
issue, but it was not quite as sever there because the naptime was 
longer. Fixed that.

How does this look now?

- Heikki

Attachment: replication_timeout_patch_v4_heikki.patch
Description: text/x-diff (27.2 KB)

In response to

Responses

pgsql-hackers by date

Next:From: Alvaro HerreraDate: 2012-10-10 15:44:47
Subject: Re: [bugfix] sepgsql didn't follow the latest core API changes
Previous:From: Tom LaneDate: 2012-10-10 15:34:20
Subject: Re: Is there a good reason why PL languages do not support cstring type arguments and return values ?

pgsql-bugs by date

Next:From: demurciaDate: 2012-10-10 15:54:26
Subject: BUG #7593: Unexpected query result combining or and in operators
Previous:From: Noah MischDate: 2012-10-10 11:05:55
Subject: Re: BUG #6510: A simple prompt is displayed using wrong charset

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group