Re: Fuzz testing COPY FROM parsing

On 2/5/21 10:54 AM, Stephen Frost wrote:
> Greetings,
>
> * Heikki Linnakangas (hlinnaka(at)iki(dot)fi) wrote:
>> I've been mucking around with COPY FROM lately, and to test it, I wrote some
>> tools to generate input files and load them with COPY FROM:
>>
>> https://github.com/hlinnaka/pgcopyfuzz
> Neat!
>
>> I used a fuzz testing tool called honggfuzz [1] to generate test inputs for
>> COPY FROM. At first I tried to use afl and libfuzzer, but honggfuzz was much
>> easier to use with PostgreSQL. It has a "persistent fuzzing mode", which
>> allows starting the server normally (well, in single-user mode), and calling
>> a function to get the next input. With the other fuzzers I tried, you have
>> to provide a callback function that the fuzzer calls for each test
>> iteration, and that was hard to integrate into the PostgreSQL main
>> processing loop.
> Yeah, that's been one of the challenges with fuzzers I've played with
> too.
>
>> I ran it for about 2 h on my laptop with the patch I was working on [2]. It
>> didn't find any crashes, but it generated about 1300 input files that it
>> considered "interesting" based on code coverage analysis. When I took those
>> generated inputs, and ran them against unpatched and patched server, some
>> inputs produced different results. So that revealed a couple of bugs in the
>> patch. (I'll post a fixed patched version on that thread soon.)
>>
>> I hope others find this useful, too.
> Nice! I wonder if there's a way to have a buildfarm member or other
> system doing this automatically on new commits and perhaps adding
> coverage for other things like the JSON code..

Not easily in the buildfarm as it is today. We can easily create modules
for extensions and other things that don't require modification of core
code, but things that require patching core code are a whole different
story.

That's not to say it couldn't be done, a SMOP. But using something like
Appveyor or Cirrus might be a lot simpler.

cheers

andrew

--

Andrew Dunstan
EDB: https://www.enterprisedb.com