Re: Do we need to rethink how to parallelize regression tests to speedup CLOBBER_CACHE_ALWAYS?

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: David Rowley <dgrowleyml(at)gmail(dot)com>
Cc: PostgreSQL Developers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Do we need to rethink how to parallelize regression tests to speedup CLOBBER_CACHE_ALWAYS?
Date: 2021-05-12 13:50:10
Message-ID: 1060267.1620827410@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

David Rowley <dgrowleyml(at)gmail(dot)com> writes:
> Right now we start 1 backend for each test in a parallel group then
> wait for the final backend to complete before running the next group.

> Is a particular reason for it to work that way?

There are a whole lot of cases where test Y depends on an earlier test X.
Some of those dependencies are annotated in parallel_schedule, but I fear
most are not.

If we had a full list of such dependencies then we could imagine building
a job scheduler that would dispatch any script that has no remaining
dependencies.

The cases where "script X can't run concurrently with script Y" are
also problematic. It's not as easy to discover those through testing,
since it might happen to work depending on timing.

regards, tom lane

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Julien Rouhaud 2021-05-12 14:40:46 Re: RFC: Logging plan of the running query
Previous Message Amit Langote 2021-05-12 13:19:17 Re: Feedback on table expansion hook (including patch)