Re: Re: Hot Standby query cancellation and Streaming Replication integration

Bruce Momjian wrote:
>> Right now you can't choose "master bloat", but you can choose the other
>> two. I think that is acceptable for 9.0, assuming the other two don't
>> have the problems that Tom foresees.
>>
>
> I was wrong. You can choose "master bloat" with
> vacuum_defer_cleanup_age, but only crudely because it is measured in
> xids and the master defers no matter what queries are running on the
> slave...

OK with you finding the situation acceptable, so long as it's an
informed decision. From how you're writing about this, I'm comfortable
you (and everybody else still involved here) have absorbed the issues
enough that we're all talking about the same thing now. Since there are
a couple of ugly user-space hacks possible for prioritizing "master
bloat", and nobody is stepping up to work on resolving this via my
suggestion involving better SR integration, seems to me heated
discussion of code changes has come to a resolution of sorts I (and
Simon, just checked) can live with. Sounds like we have three action
paths here:

-Tom already said he was planning a tour through the HS/SR code, I
wanted that to happen with him aware of this issue.
-Josh will continue doing his testing, also better informed about this
particular soft spot.
-I'll continue test-case construction for the problems here there are
still concerns about (pathologic max_standby_delay and b-tree split
issues being the top two on that list), and keep sharing particularly
interesting ones here to help everyone else's testing.

If it turns out any of those paths leads to a must-fix problem that
doesn't have an acceptable solution, at least the idea of this as a
"plan B" is both documented and more widely understood then when I
started ringing this particular bell.

I just updated the Open Items list:
http://wiki.postgresql.org/wiki/PostgreSQL_9.0_Open_Items to officially
put myself on the hook for the following HS related documentation items
that have come up recently, aiming to get them all wrapped up in time
before or during early beta:

-Update Hot Standby documentation: clearly explain relationships between
the 3 major setup trade-offs, "buffer cleanup lock", notes on which
queries are killed once max_standby_delay is reached, measuring XID
churn on master for setting vacuum_defer_cleanup_age
-Clean up archive_command docs related to recent "/bin/true" addition.
Given that's where I expect people who run into the pg_stop_backup
warning message recently added will end up at, noting its value for
escaping from that particular case might be useful too.

To finish airing my personal 9.0 TODO list now that I've gone this far,
I'm also still working on completing the following patches that initial
versions have been submitted of, was close to finishing both before
getting side-tracked onto this larger issue:

-pgbench > 4000 scale bug fix:
http://archives.postgresql.org/message-id/4B621BA3.7090306@2ndquadrant.com
-Improving the logging/error reporting/no timestamp issues in pg_standby
re-raised recently by Selena:
http://archives.postgresql.org/message-id/2b5e566d1001250945oae17be8n6317f827e3bd7492@mail.gmail.com

If nobody else claims them as something they're working on before, I
suspect I'll then move onto building some of the archiver UI
improvements discussed most recently as part of the "pg_stop_backup does
not complete" thread, despite Heikki having crushed my dreams of a
simple solution to those by pointing out the shared memory memory
limitation involved.

--
Greg Smith 2ndQuadrant US Baltimore, MD
PostgreSQL Training, Services and Support
greg(at)2ndQuadrant(dot)com www.2ndQuadrant.us