Re: Warn when parallel restoring a custom dump without data offsets

From: David Gilman <dgilman(at)gilslotd(dot)com>
To: Justin Pryzby <pryzby(at)telsasoft(dot)com>
Cc: David Gilman <davidgilman1(at)gmail(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, pgsql-hackers(at)lists(dot)postgresql(dot)org, thomas(dot)munro(at)gmail(dot)com
Subject: Re: Warn when parallel restoring a custom dump without data offsets
Date: 2020-05-25 21:55:26
Message-ID: 20200525215526.t7dqywfywxzmlpbu@dar.local
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

The earlier patches weren't applying because I had "git config
diff.noprefix true" set globally and that was messing up the git
format-patch output.

On Mon, May 25, 2020 at 01:54:29PM -0500, David Gilman wrote:
> And I misunderstood how bad it was. I thought it was reading little
> header structs off the disk but it's actually reading the entire table
> (see _skipData). So you're quadratically rereading entire tables and
> thrashing your cache. Oops.

I changed _skipData to fseeko() instead of fread() when possible to cut
down on this thrashing further.

David Gilman :DG<

Attachment Content-Type Size
0001-Remove-unused-seek-check-in-tar-dump-format.patch text/x-diff 1.1 KB
0002-Skip-tables-in-pg_restore-by-seeking-instead-of-read.patch text/x-diff 1.5 KB
0003-Scan-all-TOCs-when-restoring-a-custom-dump-file-with.patch text/x-diff 3.7 KB
0004-Add-integration-test-for-out-of-order-TOC-requests-i.patch text/x-diff 11.4 KB

In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message Tomas Vondra 2020-05-25 22:04:07 Re: Trouble with hashagg spill I/O pattern and costing
Previous Message Peter Eisentraut 2020-05-25 20:50:48 Re: factorial function/phase out postfix operators?