Re: Pluggable toaster

Hi,

Aleksander, thanks for the discussion! It seems to me that I have to add
some parts
of it to API documentation, to clarify the details on API purpose and
use-cases.

On Mon, Oct 24, 2022 at 6:37 PM Aleksander Alekseev <
aleksander(at)timescale(dot)com> wrote:

> Hi Nikita,
>
> > > > TOAST implementation is not necessary for Table AM.
> >
> > >What other use cases for TOAST do you have in mind?
> >
> > The main use case is the same as for the TOAST mechanism - storing and
> retrieving
> > oversized data. But we expanded this case with some details -
> > - update TOASTed data (yes, current TOAST implementation cannot update
> stored
> > data - is marks whole TOASTED object as dead and stores new one);
> > - retrieve part of the stored data chunks without fully de-TOASTing
> stored data (even
> > with existing TOAST this will be painful if you have to get just a small
> part of the several
> > hundreds Mb sized object);
> > - be able to store objects of size larger than 1 Gb;
> > - store more than 4 Tb of TOASTed data for one table;
> > - optimize storage for fast search and retrieval of parts of TOASTed
> object - this is
> > must-have for effectively using JSON, PostgreSQL already is in
> catching-up position
> > in JSON performance field.
>
> I see. Actually most of this is what TableAM does. We just happened to
> give this part of TableAM a separate name. The only exception is the
> last case, when you create custom TOASTers for particular types and
> potentially specify them for the given column.
>
> All in all, this makes sense.
>
> > Right. that's why we propose a validate method (may be, it's a wrong
> > name, but I don't known better one) which accepts several arguments, one
> > of which is table AM oid. If that method returns false then toaster
> > isn't useful with current TAM, storage or/and compression kinds, etc.
>
> OK, I missed this message. So there was some misunderstanding after
> all, sorry for this.
>
> That's exactly what I wanted to know. It's much better than allowing
> any TOASTer to run on top of any TableAM.
>
> --
> Best regards,
> Aleksander Alekseev
>

--
Regards,
Nikita Malakhov
Postgres Professional
https://postgrespro.ru/