Re: Re: xReader, double-effort (was: Temporary tables under hot standby)

On Sat, Apr 28, 2012 at 11:06 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Simon Riggs <simon(at)2ndQuadrant(dot)com> writes:
>> Translating WAL is a very hard task.
>
> No kidding.  I would think it's impossible on its face.  Just for
> starters, where will you get table and column names from?  (Looking at
> the system catalogs is cheating, and will not work reliably anyway.)
>
> IMO, if we want non-physical replication, we're going to need to build
> it in at a higher level than after-the-fact processing of WAL.
> I foresee wasting quite a lot of effort on the currently proposed
> approaches before we admit that they're unworkable.

I think the question we should be asking ourselves is not whether WAL
as it currently exists is adequate for logical replication, but rather
or not it could be made adequate. For example, suppose that we were
to arrange things so that, after each checkpoint, the first insert,
update, or delete record for a given relfilenode after each checkpoint
emits a special WAL record that contains the relation name, schema
OID, attribute names, and attribute type OIDs. Well, now we are much
closer to being able to do some meaningful decoding of the tuple data,
and it really doesn't cost us that much. Handling DDL (and manual
system catalog modifications) seems pretty tricky, but I'd be very
reluctant to give up on it without banging my head against the wall
pretty hard. The trouble with giving up on WAL completely and moving
to a separate replication log is that it means a whole lot of
additional I/O, which is bound to have a negative effect on
performance.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company