Re: PQgetlength vs. octet_length()

From: Greg Stark <gsstark(at)mit(dot)edu>
To: Michael Clark <codingninja(at)gmail(dot)com>
Cc: pgsql-general(at)postgresql(dot)org
Subject: Re: PQgetlength vs. octet_length()
Date: 2009-08-18 17:48:57
Message-ID: 407d949e0908181048p530a3684j8c6ba3058b3f6888@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

On Tue, Aug 18, 2009 at 6:39 PM, Michael Clark<codingninja(at)gmail(dot)com> wrote:
> But it seems pretty crazy that a 140meg bit of data goes to 1.3 gigs.  Does
> that seem a bit excessive?

From what you posted earlier it looked like it was turning into about
500M which sounds about right. Presumably either libpq or your code is
holding two copies of it in ram at some point in the process.

8.5 will have an option to use a denser hex encoding but it will still
be 2x as large as the raw data.

> I avoided the binary mode because that seemed to be rather confusing when
> having to deal with non-bytea data types.  The docs make it sound like
> binary mode should be avoided because what you get back for a datetime
> varies per platform.

There are definitely disadvantages. Generally it requires you to know
what the binary representation of your data types is and they're not
all well documented or guaranteed not to change in the future. I
wouldn't recommend someone try to decode a numeric or a postgres array
for example. And floating point numbers are platform dependent. But
bytea is a case where it seems more natural to use binary than text
representation.

--
greg
http://mit.edu/~gsstark/resume.pdf

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Alban Hertroys 2009-08-18 17:59:35 Re: Unit conversion database (was: multiple paramters in aggregate function)
Previous Message Michael Clark 2009-08-18 17:39:30 PQgetlength vs. octet_length()

Browse pgsql-hackers by date

  From Date Subject
Next Message Josh Berkus 2009-08-18 17:50:57 Re: "make install" now tries to build the documentation
Previous Message Tom Lane 2009-08-18 17:43:52 Re: "make install" now tries to build the documentation