From: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
---|---|
To: | Chao Li <li(dot)evan(dot)chao(at)gmail(dot)com> |
Cc: | Dimitrios Apostolou <jimis(at)gmx(dot)net>, Nathan Bossart <nathandbossart(at)gmail(dot)com>, Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, pgsql-hackers(at)lists(dot)postgresql(dot)org |
Subject: | Re: [PING] [PATCH v2] parallel pg_restore: avoid disk seeks when jumping short distance forward |
Date: | 2025-10-15 19:21:50 |
Message-ID: | 1306492.1760556110@sss.pgh.pa.us |
Views: | Whole Thread | Raw Message | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
I wrote:
> 0004 increases the row width in the existing test case that says
> it's trying to push more than DEFAULT_IO_BUFFER_SIZE through
> the compressors. While I agree with the premise, this solution
> is hugely expensive: it adds about 12% to the already-long runtime
> of 002_pg_dump.pl. I'd like to find a better way, but ran out of
> energy for today. (I think the reason this costs so much is that
> it's effectively iterated hundreds of times because of
> 002_pg_dump.pl's more or less cross-product approach to testing
> everything. Maybe we should pull it out of that structure?)
The attached patchset accomplishes that by splitting 002_pg_dump.pl
into two scripts, one that is just concerned with the compression
test cases and one that does everything else. This might not be
the prettiest solution, since it duplicates a lot of perl code.
I thought about refactoring 002_pg_dump.pl so that it could handle
two separate sets of runs-plus-tests, but decided it was overly
complicated already.
Anyway, 0001 attached is the same as in v4, 0002 performs the
test split without intending to change coverage, and then 0003
adds the new test cases I wanted. For me, this ends up with
just about the same runtime as before, or maybe a smidge less.
I'd hoped for possibly more savings than that, but I'm content
with it being a wash.
I think this is more or less committable, and then we could get
back to the original question of whether it's worth tweaking
pg_restore's seek-vs-scan behavior.
regards, tom lane
Attachment | Content-Type | Size |
---|---|---|
v5-0001-Align-the-data-block-sizes-of-pg_dump-s-various-c.patch | text/x-diff | 6.1 KB |
v5-0002-Split-002_pg_dump.pl-into-two-test-files.patch | text/x-diff | 32.5 KB |
v5-0003-Add-more-TAP-test-coverage-for-pg_dump.patch | text/x-diff | 6.9 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Isaac Morland | 2025-10-15 19:35:36 | Re: Thoughts on a "global" client configuration? |
Previous Message | Peter Eisentraut | 2025-10-15 19:20:31 | Re: Thoughts on a "global" client configuration? |