Skip site navigation (1) Skip section navigation (2)

Re: Large arrays give long lag on server side before command executes

From: David Helgason <david(at)uti(dot)is>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Steve Crawford <scrawford(at)pinpointresearch(dot)com>,postgresql General <pgsql-general(at)postgresql(dot)org>
Subject: Re: Large arrays give long lag on server side before command executes
Date: 2004-09-26 11:11:10
Message-ID: C55D411F-0FAC-11D9-A386-000A9566DA8A@uti.is (view raw or flat)
Thread:
Lists: pgsql-general
On 25. sep 2004, at 01:42, Tom Lane wrote:

> Steve Crawford <scrawford(at)pinpointresearch(dot)com> writes:
>> On Friday 24 September 2004 7:32 am, Tom Lane wrote:
>>> I'm betting on some O(N^2) behavior in the array code, but it'll be
>>> difficult to pinpoint without profile results.
>
>> Possibly the bug we discussed early last year (IIRC added to todo but
>> not fixed):
>> http://archives.postgresql.org/pgsql-performance/2003-01/msg00235.php
>
> No, it's not the same --- David sent me some results off-list and it
> seems ReadArrayStr is the culprit.  I'm not completely sure why yet.

After playing around, I found the culprit. As always, I might just as 
well have gone to the code. Too bad C always reads to me like an old 
manuscript. I know the language, but I have to stutter through it. 
Anyway, these lines should trigger anyones big-O alarm:

/* adt/arrayfuncs.c, line 575ff in v. 7.4.1. In the big switch 
statement inside ReadArrayStr() */

				case '\\':
					{
						char	   *cptr;

						/* Crunch the string on top of the backslash. */
						for (cptr = ptr; *cptr != '\0'; cptr++) /* BAD :) */
							*cptr = *(cptr + 1);
						if (*ptr == '\0')
							ereport(ERROR,
							(errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
							 errmsg("malformed array literal: \"%s\"", arrayStr)));
						break;
					}
				case '\"':
					{
						char	   *cptr;

						scanning_string = !scanning_string;
						/* Crunch the string on top of the quote. */
						for (cptr = ptr; *cptr != '\0'; cptr++) /* BAD :) */
							*cptr = *(cptr + 1);
						/* Back up to not miss following character. */
						ptr--;
						break;
					}


I'm toying with a reimplementation of this algorithm, but it might be 
some days before I have the time. Didn't look at the 8.0 beta yet, so 
of course someone might have been there already.

I'll be back when I have more.

d.


> FWIW, the printtup part of your gripe is fixed in CVS tip.
>
> 			regards, tom lane
>
> ---------------------------(end of 
> broadcast)---------------------------
> TIP 6: Have you searched our list archives?
>
>                http://archives.postgresql.org
>


-- 
David Helgason,
Business Development et al.,
Over the Edge I/S (http://otee.dk)
Direct line +45 2620 0663
Main line +45 3264 5049


-- 
David Helgason,
Business Development et al.,
Over the Edge I/S (http://otee.dk)
Direct line +45 2620 0663
Main line +45 3264 5049


In response to

Responses

pgsql-general by date

Next:From: Eddy MacnaghtenDate: 2004-09-26 12:26:57
Subject: Re: PSQLFS - PostgreSQL File System
Previous:From: Gaetano MendolaDate: 2004-09-26 09:55:24
Subject: Re: Use of 'interval'

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group