Skip site navigation (1) Skip section navigation (2)

Re: GDQ iimplementation

From: Jan Wieck <JanWieck(at)Yahoo(dot)com>
To: Marko Kreen <markokr(at)gmail(dot)com>
Cc: Hannu Krosing <hannu(at)2ndquadrant(dot)com>, Josh Berkus <josh(at)agliodbs(dot)com>, pgsql-cluster-hackers(at)postgresql(dot)org
Subject: Re: GDQ iimplementation
Date: 2010-05-11 14:06:35
Message-ID: 4BE9646B.6000709@Yahoo.com (view raw or flat)
Thread:
Lists: pgsql-cluster-hackers
On 5/11/2010 9:36 AM, Marko Kreen wrote:
> Seems we are in agreement.

That's always a good point to start from.

> And although PgQ can operate with any N >= 2 segments, it queries
> on 2 at a time, same as Slony.  Rest are just there to give admins
> some safety room for "OH F*CK" moments.  With short rotation times,
> it starts to seem useful..

Agreed. The rotation time should actually reflect the longest running 
transactions experienced on a frequent base from the application. And 
there needs to be a safeguard against rotating over even longer running 
transactions.

The problem with a long running transaction is that it could have 
written into log segment 1 before we switched to segment 2. We can only 
TRUNCATE segment 1 after that transaction committed AND the log has been 
consumed by everyone interested in it.

I am not familiar with how PgQ/Londiste do this. Slony specifically 
remembers the highest XID in progress at the time of switching, waits 
until the lowest XID in progress is higher than that (so all log that 
ever went into that segment is now visible or aborted), then waits for 
all log in that segment to be confirmed and finally truncates the log. 
All this time, it needs to do the UNION query over both log segments.

 > There does not seem any advantage for querying more than 2 segments.

I didn't experiment with such implementation yet. I'll theorize about 
that in a separate thread later.

>>  No. I don't know how you read 1) into the above and 2) was my
>> misunderstanding reading the Wiki. I don't want either.
> 
> Oh sorry, I got that impression from wiki, not from you.
> 
> As there are some ideas from you on the wiki, I assumed
> you are involved, so used 'you' very liberally.

No problem. I misinterpreted stuff there as "the currently favored idea" 
too.


Jan

-- 
Anyone who trades liberty for security deserves neither
liberty nor security. -- Benjamin Franklin

In response to

Responses

pgsql-cluster-hackers by date

Next:From: Simon RiggsDate: 2010-05-11 14:26:39
Subject: Re: GDQ iimplementation (was: Re: Clustering features for upcoming developer meeting -- please claim yours!)
Previous:From: Marko KreenDate: 2010-05-11 14:03:40
Subject: Re: GDQ iimplementation (was: Re: Clustering features for upcoming developer meeting -- please claim yours!)

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group