Skip site navigation (1) Skip section navigation (2)

Re: Terrible performance on wide selects

From: "Dann Corbit" <DCorbit(at)connx(dot)com>
To: "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>,"Steve Crawford" <scrawford(at)pinpointresearch(dot)com>
Cc: <pgsql-performance(at)postgreSQL(dot)org>,<pgsql-hackers(at)postgreSQL(dot)org>
Subject: Re: Terrible performance on wide selects
Date: 2003-01-22 23:56:55
Message-ID: D90A5A6C612A39408103E6ECDD77B8294CD85F@voyager.corporate.connx.com (view raw or flat)
Thread:
Lists: pgsql-hackerspgsql-performance
> -----Original Message-----
> From: Tom Lane [mailto:tgl(at)sss(dot)pgh(dot)pa(dot)us] 
> Sent: Wednesday, January 22, 2003 3:15 PM
> To: Steve Crawford
> Cc: pgsql-performance(at)postgreSQL(dot)org; pgsql-hackers(at)postgreSQL(dot)org
> Subject: Re: [HACKERS] Terrible performance on wide selects 
> 
> 
> Steve Crawford sent me some profiling results for queries 
> involving wide tuples (hundreds of columns).
> 
> > Done, results attached. nocachegetattr seems to be the 
> likely suspect.
> 
> Yipes, you can say that again.
> 
>   %   cumulative   self              self     total           
>  time   seconds   seconds    calls  ms/call  ms/call  name    
>  93.38     26.81    26.81   885688     0.03     0.03  nocachegetattr
> 
>                 0.00    0.00       1/885688      heapgettup [159]
>                 0.00    0.00       1/885688      
> CatalogCacheComputeTupleHashValue [248]
>                 0.00    0.00       5/885688      SearchCatCache [22]
>                13.40    0.00  442840/885688      ExecEvalVar [20]
>                13.40    0.00  442841/885688      printtup [12]
> [11]    93.4   26.81    0.00  885688         nocachegetattr [11]
> 
> 
> Half of the calls are coming from printtup(), which seems 
> relatively easy to fix.
> 
> 	/*
> 	 * send the attributes of this tuple
> 	 */
> 	for (i = 0; i < natts; ++i)
> 	{
> 		...
> 		origattr = heap_getattr(tuple, i + 1, typeinfo, 
> &isnull);
> 		...
> 	}
> 
> The trouble here is that in the presence of variable-width 
> fields, heap_getattr requires a linear scan over the tuple 
> --- and so the total time spent in it is O(N^2) in the number 
> of fields.
> 
> What we could do is reinstitute heap_deformtuple() as the inverse of
> heap_formtuple() --- but make it extract Datums for all the 
> columns in a single pass over the tuple.  This would reduce 
> the time in printtup() from O(N^2) to O(N), which would 
> pretty much wipe out that part of the problem.
> 
> The other half of the calls are coming from ExecEvalVar, 
> which is a harder problem to solve, since those calls are 
> scattered all over the place.  It's harder to see how to get 
> them to share work.  Any ideas out there?

Is it possible that the needed information could be retrieved by
querying the system metadata to collect the column information?

Once the required tuple attributes are described, it could form a
binding list that allocates a buffer of sufficient size with pointers to
the required column start points.

Maybe I don't really understand the problem, but it seems simple enough
to do it once for the whole query.

If this is utter stupidity, please disregard and have a hearty laugh at
my expense.
;-)

Responses

pgsql-performance by date

Next:From: Sean ChittendenDate: 2003-01-22 23:59:31
Subject: Re: [PERFORM] Proposal: relaxing link between explicit JOINs and execution order
Previous:From: Tom LaneDate: 2003-01-22 23:14:30
Subject: Re: Terrible performance on wide selects

pgsql-hackers by date

Next:From: Sean ChittendenDate: 2003-01-22 23:59:31
Subject: Re: [PERFORM] Proposal: relaxing link between explicit JOINs and execution order
Previous:From: Tom LaneDate: 2003-01-22 23:14:30
Subject: Re: Terrible performance on wide selects

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group