Skip site navigation (1) Skip section navigation (2)

Re: Re: Hot Standby query cancellation and Streaming Replication integration

From: Greg Smith <greg(at)2ndquadrant(dot)com>
To: Bruce Momjian <bruce(at)momjian(dot)us>
Cc: Joachim Wieland <joe(at)mcknight(dot)de>, Greg Stark <gsstark(at)mit(dot)edu>, Robert Haas <robertmhaas(at)gmail(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Re: Hot Standby query cancellation and Streaming Replication integration
Date: 2010-03-02 20:11:00
Message-ID: 4B8D70D4.7010004@2ndquadrant.com (view raw or flat)
Thread:
Lists: pgsql-hackers
Bruce Momjian wrote:
>> Right now you can't choose "master bloat", but you can choose the other
>> two.  I think that is acceptable for 9.0, assuming the other two don't
>> have the problems that Tom foresees.
>>     
>
> I was wrong.  You can choose "master bloat" with
> vacuum_defer_cleanup_age, but only crudely because it is measured in
> xids and the master defers no matter what queries are running on the
> slave...

OK with you finding the situation acceptable, so long as it's an 
informed decision.  From how you're writing about this, I'm comfortable 
you (and everybody else still involved here) have absorbed the issues 
enough that we're all talking about the same thing now.  Since there are 
a couple of ugly user-space hacks possible for prioritizing "master 
bloat", and nobody is stepping up to work on resolving this via my 
suggestion involving better SR integration, seems to me heated 
discussion of code changes has come to a resolution of sorts I (and 
Simon, just checked) can live with.  Sounds like we have three action 
paths here:

-Tom already said he was planning a tour through the HS/SR code, I 
wanted that to happen with him aware of this issue.
-Josh will continue doing his testing, also better informed about this 
particular soft spot.
-I'll continue test-case construction for the problems here there are 
still concerns about (pathologic max_standby_delay and b-tree split 
issues being the top two on that list), and keep sharing particularly 
interesting ones here to help everyone else's testing. 

If it turns out any of those paths leads to a must-fix problem that 
doesn't have an acceptable solution, at least the idea of this as a 
"plan B" is both documented and more widely understood then when I 
started ringing this particular bell.

I just updated the Open Items list:  
http://wiki.postgresql.org/wiki/PostgreSQL_9.0_Open_Items to officially 
put myself on the hook for the following HS related documentation items 
that have come up recently, aiming to get them all wrapped up in time 
before or during early beta:

-Update Hot Standby documentation: clearly explain relationships between 
the 3 major setup trade-offs, "buffer cleanup lock", notes on which 
queries are killed once max_standby_delay is reached, measuring XID 
churn on master for setting vacuum_defer_cleanup_age
-Clean up archive_command docs related to recent "/bin/true" addition.  
Given that's where I expect people who run into the pg_stop_backup 
warning message recently added will end up at, noting its value for 
escaping from that particular case might be useful too.

To finish airing my personal 9.0 TODO list now that I've gone this far, 
I'm also still working on completing the following patches that initial 
versions have been submitted of, was close to finishing both before 
getting side-tracked onto this larger issue:

-pgbench > 4000 scale bug fix:  
http://archives.postgresql.org/message-id/4B621BA3.7090306@2ndquadrant.com
-Improving the logging/error reporting/no timestamp issues in pg_standby 
re-raised recently by Selena:  
http://archives.postgresql.org/message-id/2b5e566d1001250945oae17be8n6317f827e3bd7492@mail.gmail.com

If nobody else claims them as something they're working on before, I 
suspect I'll then move onto building some of the archiver UI 
improvements discussed most recently as part of the "pg_stop_backup does 
not complete" thread, despite Heikki having crushed my dreams of a 
simple solution to those by pointing out the shared memory memory 
limitation involved.

-- 
Greg Smith  2ndQuadrant US  Baltimore, MD
PostgreSQL Training, Services and Support
greg(at)2ndQuadrant(dot)com   www.2ndQuadrant.us


In response to

Responses

pgsql-hackers by date

Next:From: Marc MunroDate: 2010-03-02 20:47:30
Subject: Hot Standby query cancellation and Streaming Replication integration
Previous:From: Josh BerkusDate: 2010-03-02 19:03:14
Subject: Re: Re: Hot Standby query cancellation and Streaming Replication integration

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group