Skip site navigation (1) Skip section navigation (2)

Re: Terrible performance on wide selects

From: Hannu Krosing <hannu(at)tm(dot)ee>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Dann Corbit <DCorbit(at)connx(dot)com>,Steve Crawford <scrawford(at)pinpointresearch(dot)com>,pgsql-performance(at)postgreSQL(dot)org, pgsql-hackers(at)postgreSQL(dot)org
Subject: Re: Terrible performance on wide selects
Date: 2003-01-23 10:11:08
Message-ID: 1043316668.2348.15.camel@localhost.localdomain (view raw or flat)
Thread:
Lists: pgsql-hackerspgsql-performance
Tom Lane kirjutas N, 23.01.2003 kell 02:18:
> "Dann Corbit" <DCorbit(at)connx(dot)com> writes:
> > Why not waste a bit of memory and make the row buffer the maximum
> > possible length?
> > E.g. for varchar(2000) allocate 2000 characters + size element and point
> > to the start of that thing.
> 
> Surely you're not proposing that we store data on disk that way.
> 
> The real issue here is avoiding overhead while extracting columns out of
> a stored tuple.  We could perhaps use a different, less space-efficient
> format for temporary tuples in memory than we do on disk, but I don't
> think that will help a lot.  The nature of O(N^2) bottlenecks is you
> have to kill them all --- for example, if we fix printtup and don't do
> anything with ExecEvalVar, we can't do more than double the speed of
> Steve's example, so it'll still be slow.  So we must have a solution for
> the case where we are disassembling a stored tuple, anyway.
> 
> I have been sitting here toying with a related idea, which is to use the
> heap_deformtuple code I suggested before to form an array of pointers to
> Datums in a specific tuple (we could probably use the TupleTableSlot
> mechanisms to manage the memory for these).  Then subsequent accesses to
> individual columns would just need an array-index operation, not a
> nocachegetattr call.  The trick with that would be that if only a few
> columns are needed out of a row, it might be a net loss to compute the
> Datum values for all columns.  How could we avoid slowing that case down
> while making the wide-tuple case faster?

make the pointer array incrementally for O(N) performance:

i.e. for tuple with 100 cols, allocate an array of 100 pointers, plus
keep count of how many are actually valid,

so the first call to get col[5] will fill first 5 positions in the array
save said nr 5 and then access tuple[ptrarray[5]]

next call to get col[75] will start form col[5] and fill up to col[75]

next call to col[76] will start form col[75] and fill up to col[76]

next call to col[60] will just get tuple[ptrarray[60]]

the above description assumes 1-based non-C arrays ;)

-- 
Hannu Krosing <hannu(at)tm(dot)ee>

In response to

Responses

pgsql-performance by date

Next:From: Hannu KrosingDate: 2003-01-23 10:28:21
Subject: Re: [PERFORM] Terrible performance on wide selects
Previous:From: Chantal AckermannDate: 2003-01-23 09:16:01
Subject: Re: optimizing query

pgsql-hackers by date

Next:From: Hannu KrosingDate: 2003-01-23 10:28:21
Subject: Re: [PERFORM] Terrible performance on wide selects
Previous:From: Antonio ScottiDate: 2003-01-23 09:42:31
Subject: On Commit Triggers

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group