Re: Optimize external TOAST storage

From: Robert Haas <robertmhaas(at)gmail(dot)com>
To: Nathan Bossart <nathandbossart(at)gmail(dot)com>
Cc: davinder singh <davindersingh2692(at)gmail(dot)com>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Optimize external TOAST storage
Date: 2022-03-17 17:04:17
Message-ID: CA+TgmoYQXhEiGU8ud3EojeBA9KnRy7HW_QUp3-HoDEqCc0DnEg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Wed, Mar 16, 2022 at 2:36 PM Nathan Bossart <nathandbossart(at)gmail(dot)com> wrote:
> Thinking further, is simply reducing the number of TOAST chunks the right
> thing to look at? If I want to add a TOAST attribute that requires 100,000
> chunks, and you told me that I could save 10% in the read path for an extra
> 250 chunks of disk space, I would probably choose read performance. If I
> wanted to add 100,000 attributes that were each 3 chunks, and you told me
> that I could save 10% in the read path for an extra 75,000 chunks of disk
> space, I might choose the extra disk space. These are admittedly extreme
> (and maybe even impossible) examples, but my point is that the amount of
> disk space you are willing to give up may be related to the size of the
> attribute. And maybe one way to extract additional read performance with
> this optimization is to use a variable threshold so that we are more likely
> to use it for large attributes.

Right, so perhaps the ultimate thing here would be a more fine-grained
knob than SET STORAGE EXTERNAL -- something that allows you to specify
that you want to compress only when it really helps. While some people
might find that useful, I think the current patch is less ambitious,
and I think that's OK. It just wants to save something in the cases
where it's basically free. Unfortunately we've learned that it's never
*entirely* free because making the last TOAST chunk longer can always
cost you something, even if it gets longer by only 1 byte. But for
larger values it's hard for that to be significant.

--
Robert Haas
EDB: http://www.enterprisedb.com

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Bharath Rupireddy 2022-03-17 17:05:50 Re: Allow async standbys wait for sync replication
Previous Message Robert Haas 2022-03-17 16:56:45 Re: Unhyphenation of crash-recovery