Re: A note about debugging TAP failures

> On 23 Apr 2017, at 15:05, Craig Ringer <craig(dot)ringer(at)2ndquadrant(dot)com> wrote:
>
> On 23 Apr. 2017 10:32, "Michael Paquier" <michael(dot)paquier(at)gmail(dot)com <mailto:michael(dot)paquier(at)gmail(dot)com>> wrote:
> On Sun, Apr 23, 2017 at 7:48 AM, Daniel Gustafsson <daniel(at)yesql(dot)se <mailto:daniel(at)yesql(dot)se>> wrote:
> > Skipping the tempdir and instead using ${testname}_data_${name} without a
> > random suffix, we can achieve this with something along the lines of the
> > attached PoC. It works as now (retain of failure, remove on success unless
> > overridden) but that logic can easily be turned around if we want that. If
> > it’s of interest I can pursue this after some sleep (tomorrow has become today
> > at this point).
>
> Yes, something like that may make sense as well for readability.
>
> Keeping folders in case of failures is something that I have been
> advocating in favor of for some time, but this never got into the tree
> :(
>
> Huh? We do keep test temp datadirs etc in case of failure. Just not on success.
>
> Our definition of failure there sucks a bit though. We keep the datadirs if any test fails in a script. If the script its self crashes we still blow away the datadirs which is kind of counterintuitive.
>
> I'd like to change the __DIE__ sig handler to only delete on clean script exit code, tap reporting success, and if some env bar like PG_TESTS_NOCLEAN is undefined. The later could also be used in pg_regress etc.

That sounds like a good idea, even though END might be enough when not using
tempdir() for the datadirs since die() should set a non-zero $? afaik (still
doesn’t hurt to capture with __DIE__ though). I extended my previous test with
this, and other comments in this thread: it produces non-random datadirs,
retains them on die|exit|test-failure or if PG_TESTS_NOCLEAN is set. Theres
also a check-clean target for cleaning out retained datadirs.

Given that the datadirs do occupy quite some space, perhaps a PG_TESTS_DOCLEAN
(or similar) would be good as well to always blow away the datadirs regardless
of test status?

I’m np Perl expert though so there might be better/cleaner ways to achieve
this, in testing it seems to work though. rmtree() is supported at least since
Perl 5.6 from what I can see.

cheers ./daniel