Re: [HACKERS] Custom compression methods

From: Andrew Dunstan <andrew(at)dunslane(dot)net>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Justin Pryzby <pryzby(at)telsasoft(dot)com>
Cc: Dilip Kumar <dilipbalaut(at)gmail(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>, Andres Freund <andres(at)anarazel(dot)de>, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, Alexander Korotkov <a(dot)korotkov(at)postgrespro(dot)ru>, David Steele <david(at)pgmasters(dot)net>, Ildus Kurbangaliev <i(dot)kurbangaliev(at)gmail(dot)com>, Dmitry Dolgov <9erthalion6(at)gmail(dot)com>, pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Re: [HACKERS] Custom compression methods
Date: 2021-03-20 20:37:51
Message-ID: fe462d41-7063-72a1-2e70-0300c53ee2c4@dunslane.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers


On 3/20/21 3:03 PM, Tom Lane wrote:
> I wrote:
>> Yeah, _doSetFixedOutputState is the wrong place: that runs on the
>> pg_restore side of the fence, and would not have access to the
>> necessary info in a separated dump/restore run.
>> It might be necessary to explicitly pass the state through in a TOC item,
>> as we do for things like the standard_conforming_strings setting.
> Ah, now that I read your patch I see that's exactly what you did.
>
> I fixed up some issues in 0008/0009 (mostly cosmetic, except that
> you forgot a server version check in dumpToastCompression) and
> pushed that, so we can see if it makes crake happy.
>
>

It's still produced a significant amount more difference between the
dumps. For now I've increased the fuzz factor a bit like this:

diff --git a/PGBuild/Modules/TestUpgradeXversion.pm
b/PGBuild/Modules/TestUpgradeXversion.pm
index 1d1d313..567d7cb 100644
--- a/PGBuild/Modules/TestUpgradeXversion.pm
+++ b/PGBuild/Modules/TestUpgradeXversion.pm
@@ -621,7 +621,7 @@ sub test_upgrade    ## no critic
(Subroutines::ProhibitManyArgs)
    # generally from reordering of larg object output.
    # If not we heuristically allow up to 2000 lines of diffs
 
-   if (   ($oversion ne $this_branch && $difflines < 2000)
+   if (   ($oversion ne $this_branch && $difflines < 2700)
        || ($oversion eq $this_branch) && $difflines < 50)
    {
        return 1;

I'll try to come up with something better. Maybe just ignore lines like

SET default_toast_compression = 'pglz';

when taking the diff.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2021-03-20 20:39:11 Re: [HACKERS] Custom compression methods
Previous Message Peter Eisentraut 2021-03-20 20:01:25 Re: popcount