Re: ZSTD TOAST compression, and an extensible compression method encoding

From: Hannu Krosing <hannuk(at)google(dot)com>
To: Greg Burd <greg(at)burd(dot)me>
Cc: pgsql-hackers(at)lists(dot)postgresql(dot)org, Manu <manuelreyesbravo(at)gmail(dot)com>, Nikhil Kumar Veldanda <veldanda(dot)nikhilkumar17(at)gmail(dot)com>, Michael Paquier <michael(at)paquier(dot)xyz>, Japin Li <japinli(at)hotmail(dot)com>, Dharin Shah <dharinshah95(at)gmail(dot)com>
Subject: Re: ZSTD TOAST compression, and an extensible compression method encoding
Date: 2026-10-25 16:58:42
Message-ID: CAMT0RQRCq5Y2o5pezyohzeOsi7fqtBt1_Gi=Nj=GMMtqfhqkYA@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Sun, Oct 25, 2026 at 5:34 PM Greg Burd <greg(at)burd(dot)me> wrote:
...
> The auto-training/retraining aspect is likely going to elicit a lot of good
> design feedback and differing opinions, but I'd like those who participate in
> the discussion about that to first acknowledge that the fundamental idea of
> auto-retraining/evolution of dictionaries is an idea that they feel should be
> in or out of this kind of solution explicitly and up front if possible.

At least the underlying storage should allow multiple dictionalries
for the same relation and moving to newer dictionaries or some
heuristic for dictionary selection.

It also would be nice to have an option for "retiring" a dictionary by
- setting it read-only, so no new rows can use it
- having some process re-encode old values (or maybe in some cases
just wait until partitions with such values are dropped)
- then have a verify and drop function

> I do think it is a good idea and should be in, but there should be functions to
> manage dictionaries manually and a way to enable/disable the automatic piece.

+1

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message David Steele 2026-10-25 18:01:09 Re: Avoid truncated or zero-padded copies of replaced files in backups.
Previous Message Hannu Krosing 2026-10-25 16:52:14 Re: ZSTD TOAST compression, and an extensible compression method encoding