From: | Andreas Seltenreich <seltenreich(at)gmx(dot)de> |
---|---|
To: | Simon Riggs <simon(at)2ndQuadrant(dot)com> |
Cc: | pgsql-hackers(at)postgresql(dot)org |
Subject: | Re: [sqlsmith] Failed assertion in BecomeLockGroupLeader |
Date: | 2016-04-30 00:34:35 |
Message-ID: | 87y47wdjdw.fsf@credativ.de |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Simon Riggs writes:
> It's good that the input is fuzzed, but there needs to be a way to re-run
> identical fuzzing or a way to backtrack to find what broke. Not much point
> finding bugs we can't identify later.
sqlsmith is deterministic and allows re-generating a sequence of random
queries with the --seed argument. Finding a testing methodology that
ensures a repeatable server-side is a harder problem though.
One would have to avoid touching any kind of concurrency, disable
autovacuum, autoanalyze and invoke explicit analyzes/vacuums in concert
with query generation. Further, one would have to avoid any kind of
concurrency while testing. Even then, 1% of the queries run into a
statement_timeout due to randomly generated excessive cross joins. If a
timeout just barely happens, it might not do so on the repeated run and
the deterministic state is gone from then on. I'm afraid this list is
not complete yet.
I didn't think the effort of creating this kind of clean-room testing
was worth it. If reports of failed assertions with backtrace without a
recipe to reproduce them are a nuisance, I'll avoid them in the future.
regards,
Andreas
From | Date | Subject | |
---|---|---|---|
Next Message | Andres Freund | 2016-04-30 00:36:45 | Re: [sqlsmith] Failed assertion in BecomeLockGroupLeader |
Previous Message | Andreas Seltenreich | 2016-04-30 00:28:22 | Re: [sqlsmith] Failed assertion in BecomeLockGroupLeader |