Re: pg_dump --load-via-partition-root vs. parallel restore

Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> On Tue, Aug 28, 2018 at 3:53 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> Parallel pg_restore generally assumes that the archive file is telling it
>> the truth about data dependencies; in particular, that a TABLE DATA item
>> naming a particular target table is loading data into exactly that table.
>> --load-via-partition-root creates a significant probability that that
>> assumption is wrong, at least in scenarios where the data really does get
>> redirected into other partitions than the original one. This can result
>> in inefficiencies (e.g., index rebuild started before a table's data is
>> really all loaded) or outright failures (foreign keys or RLS policies
>> applied before the data is all loaded). I suspect that deadlock failures
>> during restore are also possible, since identify_locking_dependencies
>> is not going to be nearly close to the truth about which operations
>> might hold which locks.

> Hmm. I had the idea that this wasn't a problem because I thought we
> had all of pg_dump strictly separated into pre-data, data, and
> post-data; therefore, I thought it would be the case that none of that
> other stuff would happen until all table data was loaded. From what
> you are saying here, I guess that's not the case?

We don't run restore operations in parallel during the pre-data phase,
mainly because we lack faith that the dependencies are really completely
represented for those objects (particularly in older archives). Once we
get to parallel operation, though, everything is schedulable as soon as
its dependencies are satisfied; so it's possible for post-data objects
to run before supposedly-unrelated data objects are done. There's no
explicit representation of the data/post-data boundary in the archive's
dependencies.

regards, tom lane