On 5/11/10, Jan Wieck <JanWieck(at)yahoo(dot)com> wrote:
> On 5/11/2010 11:20 AM, Marko Kreen wrote:
> > On 5/11/10, Jan Wieck <JanWieck(at)yahoo(dot)com> wrote:
> > > On 5/11/2010 9:36 AM, Marko Kreen wrote:
> > > > And although PgQ can operate with any N >= 2 segments, it queries
> > > > on 2 at a time, same as Slony. Rest are just there to give admins
> > > > some safety room for "OH F*CK" moments. With short rotation times,
> > > > it starts to seem useful..
> > > >
> > >
> > > Agreed. The rotation time should actually reflect the longest running
> > > transactions experienced on a frequent base from the application. And
> > > needs to be a safeguard against rotating over even longer running
> > > transactions.
> > >
> > Nightly pg_dump.. ;)
> > > The problem with a long running transaction is that it could have
> > > into log segment 1 before we switched to segment 2. We can only TRUNCATE
> > > segment 1 after that transaction committed AND the log has been consumed
> > > everyone interested in it.
> > >
> > > I am not familiar with how PgQ/Londiste do this. Slony specifically
> > > remembers the highest XID in progress at the time of switching, waits
> > > the lowest XID in progress is higher than that (so all log that ever
> > > into that segment is now visible or aborted), then waits for all log in
> > > segment to be confirmed and finally truncates the log. All this time, it
> > > needs to do the UNION query over both log segments.
> > >
> > The "highest XID" means actually "own transaction" here?
> > And it's not committed yet? That's seems to leave transactions
> > that happen before it's own commit into dubious state?
> One needs to tell transactions to switch log, commit, then look at the
> highest running XID after that. Any XID lower/equal to that one could
> possibly have written into the old segment.
Yeah, sounds fine. Except you cannot ignore the newer table
with that. But that makes difference only for consumers that
> > Although you may be fine, if you don't try to minimize
> > reading both tables.
> > PgQ does this:
> > Rotate:
> > 1) If some consumer reads older table, don't rotate.
> > 2) Set table_nr++, switch_step1 = txid_current(), switch_step2 = NULL
> > 3) Commit
> > 4) Set switch_step2 = txid_current() where switch_step2 IS NULL
> > 5) Commit
> Right, exactly like that :)
> > Reader:
> > 1) xmin1 = xmin of lower snapshot of batch
> > 2) xmax2 = xmax of higher snapshot of batch
> > 3) if xmax2 < switch_step1, read older table
> > 4) if xmin1 > switch_step2, read newer table
> > 5) otherwise read both
> Sounds familiar. I still don't know exactly what role the 3rd log segment
> plays in that, but it sure cannot hurt.
It makes sure you have one rotation_period of events always
available. In case you want to do some recovery on them.
But that's it.
In response to
pgsql-cluster-hackers by date
|Next:||From: Devrim GÜNDÜZ||Date: 2010-05-12 12:59:16|
|Subject: Re: Postgres-XC build error on Fedora 12|
|Previous:||From: Jan Wieck||Date: 2010-05-11 16:04:56|
|Subject: Re: GDQ iimplementation|