Re: Catalog/Metadata consistency during changeset extraction from wal

On 21 June 2012 16:13, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> On Thursday, June 21, 2012 05:05:04 PM Simon Riggs wrote:
>> On 21 June 2012 15:53, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
>> >> ISTM we should maintain a lookup table on target system that has the
>> >> minimal required information on it.
>> >
>> > You need just about the whole catalog because the *_out procs might need
>> > to lookup types, operators and such again.
>> > Unless you want to rewrite those functions you need to provide a normal
>> > execution environment.
>>
>> OK, so its more tables than I first thought, but its not all rows and
>> columns of all catalog tables.
> Sure, there are a few you probably can leave out (pg_database, pg_auth*,
> pg_*acl, pg_(sh)?depend, pg_database, pg_tablespace, ...) but its not many.

That's a start. Leaving out the shared catalogs makes me smile already.

>> > I don't see how your idea works because of that? Am I missing something?
>> Why does the number/size of the tables required make that not work?
> The number of tables itself isn't a fundamental problem although it would make
> stuff harder.
> The problem is that the out functions expect a normal operating environment
> and might e.g. do catalog lookups themselves. I don't see how we can do
> anything here without providing a (nearly) full catalog.

I accept that there could be pathological functions in there. We're
not trying to make it work with any conceivable datatype/operator, so
forcing logical replication to follow sensible rules makes sense. Are
there any out functions that anybody uses that do that?

It's too much change to actually version the main catalog. Keeping a
separate copy of a versioned catalog for use by replication sounds
much more likely to fly.

In any case, I think we'll have to go back through the list and do
more work on evaluation. When the options look like that, its typical
to have ruled out the final winner early on, but that doesn't mean it
isn't in there somewhere.

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services