Re: Variable length varlena headers redux

Tom Lane wrote:
> Gregory Stark <stark(at)enterprisedb(dot)com> writes:
> > That seems like an awful lot of copying and pallocs that aren't there
> > currently though. And it'll make us reluctant to change over frequently used
> > data types like text -- which are precisely the ones that would gain us the
> > most.
>
> > It seems to me that it might be better to change to storing varlena lengths in
> > network byte order instead. That way we can dedicate the leading bits to toast
> > flags and read more bytes as necessary.
>
> This'll add its own overhead ... but probably less than pallocs and
> data-copying would. And I agree we can find (pretty much) all the
> places that need changing by the expedient of deliberately renaming
> the macros and struct fields.

I think we should go with the pallocs and see how it performs. That is
certainly going to be easier to do, and we can test it pretty easily.

One palloc optimization idea would be to split out the representation so
the length is stored seprately from the data in memory, and we could use
an int32 for the length, and point to the shared buffer for the data.
However I don't think our macros can handle that so it might be a
non-starter.

However, I think we should find out of the palloc is a problem before
avoiding it.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +