Skip site navigation (1) Skip section navigation (2)

Re: Streaming replication, and walsender during recovery

From: Greg Smith <greg(at)2ndquadrant(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Simon Riggs <simon(at)2ndQuadrant(dot)com>, Fujii Masao <masao(dot)fujii(at)gmail(dot)com>, Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Streaming replication, and walsender during recovery
Date: 2010-01-28 18:37:33
Message-ID: 4B61D96D.9070605@2ndquadrant.com (view raw or flat)
Thread:
Lists: pgsql-hackers
Tom Lane wrote:
> (Anyway, the argument that it's important for performance is pure
> speculation AFAIK, untainted by any actual measurements.  Given the lack
> of optimization of WAL replay, it seems entirely possible that the last
> thing you want to burden a slave with is sourcing data to more slaves.)
>   

On any typical production hardware, the work of WAL replay is going to 
leave at least one (and probably more) CPUs idle, and have plenty of 
network resources to spare too because it's just shuffling WAL in/out 
rather than dealing with so many complicated client conversations.  And 
the thing you want to redistribute--the WAL file--is practically 
guaranteed to be sitting in the OS cache at the point where you'd be 
doing it, so no disk use either.  You'll disrupt a little bit of 
memory/CPU cache, sure, but that's about it as far as leeching resources 
from the main replay in order to support the secondary slave.  I'll 
measure it fully the next time I have one setup to give some hard 
numbers, I've never seen it rise to the point where it was worth 
worrying about before to bother.

Anyway, I think what Simon was trying to suggest was that it's possible 
right now to ship partial WAL files over as they advance, if you monitor 
pg_xlogfile_name_offset and are willing to coordinate copying chunks 
over.  That basic idea is even built already--the Skytools walmgr deals 
with partial WALs for example.  Having all that built-into the server 
with a nicer UI is awesome, but it's been possible to build something 
with the same basic feature set since 8.2.  Getting that going with a 
chain of downstreams slaves is not so easy though, so there's something 
that I think would be unique to the 9.0 implementation.

-- 
Greg Smith    2ndQuadrant   Baltimore, MD
PostgreSQL Training, Services and Support
greg(at)2ndQuadrant(dot)com  www.2ndQuadrant.com


In response to

pgsql-hackers by date

Next:From: Heikki LinnakangasDate: 2010-01-28 18:40:36
Subject: Re: Streaming replication, and walsender during recovery
Previous:From: Simon RiggsDate: 2010-01-28 18:29:23
Subject: Re: Streaming replication, and walsender during recovery

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group