Re: draft RFC: concept for partial, wal-based replication

On Monday 30 November 2009 10:32:50 Stefan Kaltenbrunner wrote:
> Andres Freund wrote:
> > On Monday 30 November 2009 03:57:11 Itagaki Takahiro wrote:
> >> Boszormenyi Zoltan <zb(at)cybertec(dot)at> wrote:
> >>> we tried to discuss on a lower level what should be needed
> >>> for a partial replication based on streaming replication.
> >>
> >> We need to discuss a "partial recovery" before the partial replication.
> >
> > If you do the filtering on the sending side you dont actually need
> > partial recover in the sense that you filter in the rmgr or similar.
> >
> > Or do I miss something?
>
> the question is if filtering on the sending side is actually the "right
> thing" to do.
> It increases the overhead and the complexity on the master, especially
> if you think about different (partial) replication agreements for
> different slaves and it might also be hard to integrate with the planned
> sync/async modes.
> On the other hand if you filter on the master you might be able to avoid
> a lot of network traffic du to filtered wal records.
> I think for a first step it might make more sense to look into doing the
> filtering on the receiving side and look into actual integration with SR
> at a later stage.
I think filtering on the receiving side is harder by many degrees because you
don't have an up 2 date copy of the catalog. I cant think of a design that
does not impose severe constraints on catalog and especially replication
settings to implement on the receiving side.

Andres