Re: Use sync commit for logical replication apply in TAP tests

Peter Eisentraut <peter(dot)eisentraut(at)2ndquadrant(dot)com> writes:
> Double hmm, I ran a few more tests and different machines and get
> consistent but underwhelming improvements.
> I don't mind applying the patch nonetheless, but maybe we can get a few
> more test results from others.
> (Instructions: apply the patch and time make -C src/test/subscription check)

OK, here's some results. All are typical debug builds (--enable-cassert
in particular), and each number cited is best result of three runs:

UNPATCHED PATCHED
sss1 1m13.828s 1m12.763s (H/W RAID controller + spinning rust)
laptop 1m9.236s 1m7.969s (Macbook Pro, SSD)
gaur 11m25.58s 11m14.04s (1990s-era spinning rust)
prairiedog 8m37.848s 9m26.256s (ATA/66, 5400RPM spinning rust)

(For context, about 2m10s of gaur's cycle time is the temp install
preparation, and 32s of prairiedog's; it's just a few seconds on the
newer machines.)

I have little faith in the numbers from gaur because I saw run-to-run
variations of a couple of minutes; I suspect that the bgworker launching
bug I described yesterday is making itself felt in this test too.
prairiedog also had rather more variance than one would expect, making
me wonder if something similar is happening there.

But based on these results, I would say that as a test-speed enhancement,
this patch is a failure. I'd also question the wisdom of testing only
a non-default code path. There could be an argument for running some
of the subscription tests with async commit and some without, just to
improve code coverage.

regards, tom lane