Skip site navigation (1) Skip section navigation (2)

Synchronous replication patch v1

From: "Fujii Masao" <masao(dot)fujii(at)gmail(dot)com>
To: pgsql-hackers(at)postgresql(dot)org
Subject: Synchronous replication patch v1
Date: 2008-10-31 11:36:39
Message-ID: 3f0b79eb0810310436w360f0afdy76ff1499b177ce0d@mail.gmail.com (view raw or flat)
Thread:
Lists: pgsql-hackers
Hi

Attached is a patch for a synchronous log-shipping replication which
was discussed just a month ago. I would like you to review this patch
in Nov commit fest.

The outline of this patch is as follow;

1) Walsender

This is new process to focus on sending xlog through the position
which a backend requests on commit. Walsender calculates the area
of xlog to be replicated by a logic similar to XLogWrite.

At first, walsender is forked as a normal backend by postmater (i.e.
the standby connects to postmaster just like normal frontend).
A backend works as walsender after receiving "mimic-walsender"
message. Then, walsender is handled differently from a backend.

Now, the number of walsenders is restricted to one.

2) Communication between backends and walsender

On commit, a backend tells walsender the position (LSN) to be
replicated via shmem, and wakes it up by signaling if needed.
Then, a backend sleeps until requested replication is completed.
At this time, walsender might signal a backend to wake up.

Synchronous and asynchronous replication mode are supported.
In async case, a backend basically don't need to sleep for replication.

User can tune a backend's max sleep time as a replication timeout.
Now, the timeout closes the connection to the standby, terminates
walsender, but the other postgres process continue to work.

3) Management of the xlog positions for replication

XLog positions are managed consistent. It's necessary to be careful
especially in AdvanceXLInsertBuffer and xlog_switch case.

4) Walreceiver

This is new contrib program to focus on receiving xlog and writing it.
User can specify the xlog location (where walreceiver writes xlog in
just after receiving), and the archive location (where walreceiver
archives a filled xlog file). This options are used to cooperate with
pg_standby (prevents pg_standby from reading the xlog file under
walreceiver writing)


The above is a necessary minimum function, and some requests
which came out in the discussion have not been implemented yet.
If there is other indispensable function, please let me know.

And, there are some problems in this patch;

* This patch is somewhat big, though it should be subdivided for
   review.

* Source code comments and documents are insufficient.

Is it against the rule of commit fest to add such a status patch
into review-queue? If so, I would aim for 8.5. Otherwise,
I will deal with the problems also during commit fest.
What is your opinion?


For compile
----------------
* apply sync_replication_v1.patch to HEAD
* locate walsender.c on src/backend/postmaster
* locate walsender.h on src/include/postmaster
* locate walreceiver on contrib

How to use
---------------
1) Start postgres normally (don't need to configure any parameter)
2) Start walreceiver and connect with postmaster just like psql.
     Wal streaming starts automatically.

Now, there are three configurable parameter in postgresql.conf.

> synchronous_replication = on   # immediate replication at commit
> replication_timeout = 0ms      # 0 is disabled
> wal_sender_delay = 200ms       # 1-10000 milliseconds

The usage of walreceiver is as follow.

> Usage:
>   walreceiver [OPTION]... [XLOGLOCATION] [ARCHIVELOCATION]
>
> Options:
>   -h HOSTNAME   database server host or socket directory (default: local socket)
>   -p PORT       database server port (default: "5432")
>   -U NAME       database user name (default: postgres)
>   -?            show usage

If you want to do replication by using walreceiver and pg_standby,
it's necessary to make the archive location of them the same.

Regards;

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

Attachment: sync_replication_v1.tgz
Description: application/x-gzip (26.6 KB)

Responses

pgsql-hackers by date

Next:From: Zdenek KotalaDate: 2008-10-31 12:14:37
Subject: Re: PG_PAGE_LAYOUT_VERSION 5 - time for change
Previous:From: Zdenek KotalaDate: 2008-10-31 11:25:59
Subject: Re: PG_PAGE_LAYOUT_VERSION 5 - time for change

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group