Re: Perf Benchmarking and regression.

On 2016-06-08 23:00:15 -0400, Noah Misch wrote:
> On Sun, May 29, 2016 at 01:26:03AM -0400, Noah Misch wrote:
> > On Thu, May 12, 2016 at 10:49:06AM -0400, Robert Haas wrote:
> > > On Thu, May 12, 2016 at 8:39 AM, Ashutosh Sharma <ashu(dot)coek88(at)gmail(dot)com> wrote:
> > > > Please find the test results for the following set of combinations taken at
> > > > 128 client counts:
> > > >
> > > > 1) Unpatched master, default *_flush_after : TPS = 10925.882396
> > > >
> > > > 2) Unpatched master, *_flush_after=0 : TPS = 18613.343529
> > > >
> > > > 3) That line removed with #if 0, default *_flush_after : TPS = 9856.809278
> > > >
> > > > 4) That line removed with #if 0, *_flush_after=0 : TPS = 18158.648023
> > >
> > > I'm getting increasingly unhappy about the checkpoint flush control.
> > > I saw major regressions on my parallel COPY test, too:
> > >
> > > http://www.postgresql.org/message-id/CA+TgmoYoUQf9cGcpgyGNgZQHcY-gCcKRyAqQtDU8KFE4N6HVkA@mail.gmail.com
> > >
> > > That was a completely different machine (POWER7 instead of Intel,
> > > lousy disks instead of good ones) and a completely different workload.
> > > Considering these results, I think there's now plenty of evidence to
> > > suggest that this feature is going to be horrible for a large number
> > > of users. A 45% regression on pgbench is horrible. (Nobody wants to
> > > take even a 1% hit for snapshot too old, right?) Sure, it might not
> > > be that way for every user on every Linux system, and I'm sure it
> > > performed well on the systems where Andres benchmarked it, or he
> > > wouldn't have committed it. But our goal can't be to run well only on
> > > the newest hardware with the least-buggy kernel...
> >
> > [This is a generic notification.]
> >
> > The above-described topic is currently a PostgreSQL 9.6 open item. Andres,
> > since you committed the patch believed to have created it, you own this open
> > item. If some other commit is more relevant or if this does not belong as a
> > 9.6 open item, please let us know. Otherwise, please observe the policy on
> > open item ownership[1] and send a status update within 72 hours of this
> > message. Include a date for your subsequent status update. Testers may
> > discover new open items at any time, and I want to plan to get them all fixed
> > well in advance of shipping 9.6rc1. Consequently, I will appreciate your
> > efforts toward speedy resolution. Thanks.
> >
> > [1] http://www.postgresql.org/message-id/20160527025039.GA447393@tornado.leadboat.com
>
> This PostgreSQL 9.6 open item is past due for your status update. Kindly send
> a status update within 24 hours, and include a date for your subsequent status
> update. Refer to the policy on open item ownership:
> http://www.postgresql.org/message-id/20160527025039.GA447393@tornado.leadboat.com

I'm writing a patch right now, planning to post it later today, commit
it tomorrow.

Greetings,

Andres Freund