Re: Using streaming replication as log archiving

From: "Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov>
To: "Magnus Hagander" <magnus(at)hagander(dot)net>
Cc: "PostgreSQL-development" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Using streaming replication as log archiving
Date: 2010-09-30 13:45:00
Message-ID: 4CA44E0C0200002500036184@gw.wicourts.gov
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Magnus Hagander <magnus(at)hagander(dot)net> wrote:

>> If you could keep the development "friendly" to such features, I
>> may get around to adding them to support our needs....
>
> Would it be enough to have kind of an "archive_command" switch
> that says "whenever you've finished a complete wal segment, run
> this command on it"?

That would allow some nice options. I've been thinking what would
be the ideal use of this with our backup scheme, and the best I've
thought up would be that each WAL segment file would be a single
output stream, with the option of calling a executable (which could
be a script) with the target file name and then piping the stream to
it. At 16MB or a forced xlog switch, it would close the stream and
call the executable again with a new file name. You could have a
default executable for the default behavior, or just build in a
default if no executable is specified.

The reason I like this is that I could pipe the stream through
pg_clearxlogtail and gzip pretty much "as is" to the locations on
the database server currently used for rsync to the two targets, and
the rsync commands would send the incremental changes once per
minute to both targets. I haven't thought of another solution which
provides incremental transmission of the WAL to the local backup
location, which would be a nice thing to have, since this is most
crucial when the WAN is down and not only is WAL data not coming
back to our central location, but our application framework based
replication stream isn't making back, either.

-Kevin

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2010-09-30 13:52:46 Re: is sync rep stalled?
Previous Message David Fetter 2010-09-30 13:35:42 Re: is sync rep stalled?