Skip site navigation (1) Skip section navigation (2)

Re: Best way to construct PostgreSQL ArrayType (_int4) from C int array

From: Adrian Schreyer <ams214(at)cam(dot)ac(dot)uk>
To: Merlin Moncure <mmoncure(at)gmail(dot)com>
Cc: pgsql-general(at)postgresql(dot)org
Subject: Re: Best way to construct PostgreSQL ArrayType (_int4) from C int array
Date: 2011-04-27 17:00:47
Message-ID: BANLkTimsm5eyzoAF0KSf4cFPkb+uFZNutQ@mail.gmail.com (view raw or flat)
Thread:
Lists: pgsql-general
The largest arrays I expect at the moment are more or less sparse
vectors of around 4.8k elements and I have noticed that the
input/output (C/C++ extension) does not scale well with the number of
elements in the array.

Using a function that sums all elements in the array, this is the time
it takes for ~150k arrays of various sizes (including ordering desc
and limit 10):

128: 61ms
256: 80ms
512: 681ms
1024 1065ms
2048 7682ms
4096 21332ms

That's why I thought that the construction of the PostgreSQL array was
not optimal.

On Wed, Apr 27, 2011 at 14:49, Merlin Moncure <mmoncure(at)gmail(dot)com> wrote:
> On Wed, Apr 27, 2011 at 6:02 AM, Adrian Schreyer <ams214(at)cam(dot)ac(dot)uk> wrote:
>> At the moment I am using the following code to construct a PostgreSQL
>> array from a C array in my C extension but I am not so sure if this is
>> really the best solution:
>>
>> const int *data = array.data(); // C array
>> Datum *d = (Datum *) palloc(sizeof(Datum) * size);
>>
>> for (int i = 0; i < size; i++) d[i] = Int32GetDatum(data[i]);
>>
>> ArrayType *a = construct_array(d, size, INT4OID, sizeof(int4), true, 'i');
>>
>> Is this okay or is there a better solution (existing function in the
>> PostgreSQL source for example)?
>
> That's pretty much the best way AFAIK. Int32GetDatum doesn't do
> anything fancy -- it's just a 32 bit mask/assignment.  constructing
> the array at once is going to be a lot better than incrementally
> creating it.  Do you expect the arrays to be large, say bigger than
> 10k elements?
>
> merlin
>

In response to

Responses

pgsql-general by date

Next:From: Thomas HaroldDate: 2011-04-27 17:01:16
Subject: Re: PHP and PostgreSQL 9.0, pg_connect fails to connect
Previous:From: Karsten HilbertDate: 2011-04-27 16:52:42
Subject: Re: problem with parent/child table and FKs

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group