Re: Fixed xloginsert_locks for 9.4

On 10/3/14, 5:23 PM, Andres Freund wrote:
> How are we ever going to be able to tune it further without feedback
> from actual production usage?

With improved targeted synthetic test cases that isolate the bottleneck
to where it's really obvious, way more obvious than it will ever be in
production? I just watched you do a very successful round of chewing
right through this problem that way. And haven't the synthetic tests
already been more useful to guide development than feedback from
production for some time?

An alternate way to state one of your questions along this angle is "how
can we tell if the fixed limit of 8 is still a bottleneck on a 9.4
server unless there's a GUC to increase past that"? In my first message
here I was trying to highlight that we have little idea what that world
looks like yet. This change is already so beneficial and large, the
hotspots on systems impacted by it are probably going to move to
somewhere *very* different than earlier versions.

And when that happens, it's typically not revisiting the thing you just
made way, way faster than is still the problem anymore. If it turns out
a large bottleneck in real-world 9.4 deployments is *still*
xloginsert_locks, and there's no GUC for tuning it beyond 8, then the
people in the support trenches are going to see removing that GUC as a
terrible error. That might happen. I'm not trying to criticize your
work here, but you haven't actually made the case yet this is
likely--you cheated a little with the I/O part to get around where the
bottleneck shifts once you have a decent number of slots (8). That
tweak was part of why you got a larger gain than I did.

So that's a bad path everyone might see in the future, and if we end up
there all of us in the support trenches will suffer for having done the
wrong thing. I get what you're saying there, and I'll owe you an
apology if it plays out that way. In every other potential future I can
imagine, and in every future I consider probable, eliminating the GUC
for now makes life easier. Everyone has already talked a bunch about
why extra GUCs have a support cost too. It's one less thing to
maintain, document, mess with, break in the field, talk about, and
suffer from unintended regressions.

Do you want to put the GUC right back again in the active branch to keep
progress on this easier to make, since it's really clear already there's
still gain to be chased here? I think you've already justified doing
that. I'm still running the commit with the GUC here. I will join you
among the annoyed that it's gone group the minute I do my next git pull.

--
Greg Smith greg(dot)smith(at)crunchydatasolutions(dot)com
Chief PostgreSQL Evangelist - http://crunchydatasolutions.com/