Skip site navigation (1) Skip section navigation (2)

Re: [FEATURE REQUEST] Streaming Onlinebackup(MaybeOFFTOPIC)

From: Alvaro Herrera <alvherre(at)commandprompt(dot)com>
To: Zeugswetter Andreas ADI SD <Andreas(dot)Zeugswetter(at)s-itsolutions(dot)at>
Cc: Kevin Grittner <Kevin(dot)Grittner(at)wicourts(dot)gov>, Heikki Linnakangas <heikki(at)enterprisedb(dot)com>, Simon Riggs <simon(at)2ndquadrant(dot)com>, Jeff Davis <pgsql(at)j-davis(dot)com>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: [FEATURE REQUEST] Streaming Onlinebackup(MaybeOFFTOPIC)
Date: 2007-09-28 13:21:16
Message-ID: 20070928132116.GE18001@alvh.no-ip.org (view raw or flat)
Thread:
Lists: pgsql-hackers
Zeugswetter Andreas ADI SD wrote:
> 
> > > The probably useful next step would be to pass the current length to
> the
> > > archive_command,
> > > so it can write the filled part of the file without the need for a
> > > filter.
> >  
> > I can see that helping a lot, but not by writing onto the file on
> disk.
> > If the file is nearly empty, that would be a lot of disk I/O which
> doesn't
> > need to happen.
> 
> I think you misunderstood what I meant.
> The actual archive command is constructed by expanding certain
> placeholders.
> I am suggesting to add such a placeholder for the size of the filled
> part of the log.
> 
> A hypothetical example (note suggested %b placeholder for size in
> bytes):
> archive_command=dd if=%p of=/backup/WAL/%f bs=1 count=%b
> 
> This allows to avoid unnecessary io for the backup of partially filled
> logs.

A nice improvement on that would be to have a "rearchive_command" to
allow to sync the new bytes written since a previous archive_command (so
it needs a new placeholder "start from this byte").  This allows writing
dd seek=%s skip=%s count=%b bs=1

(I had suggested something like this when PITR was just invented, but it
was disregarded because it was too complex for the first cut or the
feature).

-- 
Alvaro Herrera                 http://www.amazon.com/gp/registry/DXLWNGRJD34J
"Sallah, I said NO camels! That's FIVE camels; can't you count?"
(Indiana Jones)

In response to

Responses

pgsql-hackers by date

Next:From: Zeugswetter Andreas ADI SDDate: 2007-09-28 13:44:36
Subject: Re: [FEATURE REQUEST] Streaming Onlinebackup (MaybeOFFTOPIC)
Previous:From: Pavel StehuleDate: 2007-09-28 12:46:36
Subject: Re: proposal casting from XML[] to int[], numeric[], text[]

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group