DBI/DBD::Pg mem. use goes exponential

From: "Bob Parkinson" <rwp(at)biome(dot)ac(dot)uk>
To: <pgsql-general(at)postgresql(dot)org>
Subject: DBI/DBD::Pg mem. use goes exponential
Date: 2005-09-29 12:38:51
Message-ID: s33bee80.047@ccw0m1.nottingham.ac.uk
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hi,

I'd really like to aviod another list (ie. interfaces) if at all possible, so...

Got a script (trimmed version below) that starts to have exponential memory use after a number of iterations around the while((....$sth->fetch())) { } struct.

Table is quite small (520 odd rows) and the row size is v. small.

Mem. use starts off at 64M (using top SIZE), after 20 iterations it's bloated to 74M then it goes on each fetch to 82, 99, 129, 193, 257, 321M, ...kerbang... err. msg below.

Out of memory during "large" request for 134221824 bytes, total sbrk() is 419323904 bytes at export.pl line 205

I know this is the script trying to (m|p|?)alloc more mem. for the process and failing.

Upgraded PG to 8.0.3, DBI to 1.48 and DBD::Pg to 1.43 this morning, but made no difference. FreeBSD 5.4 (they'll get it right RSN :- )), perl 5.8.2

Any ideas please? This seems well weird to me, but...

Cheers,

Bob

##!/usr/local/bin/perl

### SNIP set up %gwc ###

use DBI;

$dbh=DBI->connect('dbi:Pg:dbname=......................................') || die;

$stmt=qq(create temp table tt as select key,acronym,start_date,end_date,duration,charging_policy from template_event);

$dbh->prepare($stmt)->execute () || die;

foreach my $key (keys %gwc) {

my $gatewaycode=$gwc{$key};

# TEMPLATE_EVENT

if($gatewaycode eq 'B' || gatewaycode eq 'Q'){

my $stmt=qq(select acronym from tt where key=$key order by start_date asc);

my $sth=$dbh->prepare($stmt) || die;

my $rv=$sth->execute() || die;

while((my $rref=$sth->fetch())){

my ($acro,$start_date,$end_date,$duration,$charging)=(at)$rref;

sleep(4);

### eyeball top here and watch it grow ###

}

}

}

Out of memory during "large" request for 134221824 bytes, total sbrk() is 419323904 bytes at export.pl line 205

This message has been checked for viruses but the contents of an attachment
may still contain software viruses, which could damage your computer system:
you are advised to perform your own checks. Email communications with the
University of Nottingham may be monitored as permitted by UK legislation.

This message has been checked for viruses but the contents of an attachment
may still contain software viruses, which could damage your computer system:
you are advised to perform your own checks. Email communications with the
University of Nottingham may be monitored as permitted by UK legislation.

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Sean Davis 2005-09-29 13:14:39 Re: Perl regular expressions
Previous Message Douglas McNaught 2005-09-29 12:15:07 Re: Setting up a fine-grained permission system