Re: Atomic access to large arrays

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: "Victor de Buen (Bayes)" <vdebuen(at)bayesinf(dot)com>
Cc: pgsql-performance(at)postgresql(dot)org
Subject: Re: Atomic access to large arrays
Date: 2009-07-22 13:54:37
Message-ID: 19329.1248270877@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

"Victor de Buen (Bayes)" <vdebuen(at)bayesinf(dot)com> writes:
> I'm storing historical meteorological gridded data from GFS (
> http://www.nco.ncep.noaa.gov/pmb/products/gfs/) into an array field in a
> table like this:

> vl_grid smallint[361][720],

> - It's posible to tune some TOAST parameters to get faster atomic access
> to large arrays?

It might save a little bit to make the toast chunk size larger, but I'm
not sure you could gain much from that.

> - Using "EXTERNAL" strategy for storing TOAST-able columns could solve
> the problem?

Nope, wouldn't help --- AFAIR array access is not optimized for slice
access. In any case, doing that would give up the compression savings
that you were so happy about.

If your normal access patterns involve "vertical" rather than
"horizontal" scans of the data, maybe you should rethink the choice
of table layout. Or maybe the compression is enough to allow you
to consider storing the data twice, once in the current layout and
once in a "vertical" format.

regards, tom lane

In response to

Responses

Browse pgsql-performance by date

  From Date Subject
Next Message Victor de Buen (Bayes) 2009-07-22 14:27:34 Re: Atomic access to large arrays
Previous Message Scott Marlowe 2009-07-22 11:54:16 Re: Master/Slave, DB separation or just spend $$$?