Re: Adding null patch entry to cfbot/CommitFest

On Monday, May 19, 2025, Tatsuo Ishii <ishii(at)postgresql(dot)org> wrote:

> I have observed cases where a cfbot entry fails without clear reason
> [1]. Even if the patch just modifies comments, it has failed in some
> cfbot test. In this case we could easily guess that master branch
> might have problems at the time when the tests performed.

I too made this realization while reviewing the application. I concur it
is something that we should try and mitigate. Sending a canary patch
through once-a-day, or on any fixed time period, doesn’t quite seem
sufficient. We have many commits per day and immediately switch to them as
they are seen.

The system itself needs to be able to simply create a job and test
master/HEAD whenever it wishes. Then use the outcome of the job to decide
whether to wait for a new HEAD before pulling more CF entries. Or,
alternatively, continue using the “previous” commit as the base until
master/HEAD changes again and it can evaluate the new proposed base.

There are some other variations on these to consider as well. Like, the
first patch (or multiple due to concurrency) to fail on a new commit base
waits for a test of the base to complete before being marked failed or
aborted.

That said, adding a “null” CF entry to the queue right now is doable and
virtually cost-free. While it may not provide complete benefit there
likely would be some. Anyone could increment the version number and email
the thread to release a new canary on-demand. It would also provide some
data/feedback while designing and implementing a more integrated solution.

David J.