Re: INSERT performance deteriorates quickly during a large import

From: Julio Cesar Sánchez González <jsg(at)sistemasyconectividad(dot)com(dot)mx>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: "Krasimir Hristozov (InterMedia Ltd)" <krasi(at)imedia-dev(dot)com>, pgsql-general(at)postgresql(dot)org
Subject: Re: INSERT performance deteriorates quickly during a large import
Date: 2007-11-10 22:20:23
Message-ID: 1194733224.6764.3.camel@andromeda
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general


El jue, 08-11-2007 a las 13:01 -0500, Tom Lane escribió:
> "Krasimir Hristozov \(InterMedia Ltd\)" <krasi(at)imedia-dev(dot)com> writes:
> > We need to import data from a relatively large MySQL database into an
> > existing PostgreSQL database, using a PHP5 script that SELECTs from MySQL
> > and INSERTs in PostgreSQL. A part of the import involves moving about
> > 1,300,000 records from one MySQL table to one of our PostgreSQL tables. The
> > problem is that the insert performance inevitably deteriorates as the number
> > of inserts increases.
>
> Are you *certain* you've gotten rid of all the indexes and foreign keys?
> A simple insert ought to be pretty much constant-time in Postgres, so it
> seems to me that you've missed something.
>
> It also seems possible that you are wrong to disregard PHP as a possible
> source of the problem. Have you tried watching the PHP and PG backend
> processes with "top" (or similar tool) to see who's consuming CPU time
> and/or memory space?
>
> regards, tom lane
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: don't forget to increase your free space map settings

Hi for all,

Why better try use PERL DBI, may be better than.

--
Regards,

Julio Cesar Sánchez González.

--
Ahora me he convertido en la muerte, destructora de mundos.
Soy la Muerte que se lleva todo, la fuente de las cosas que vendran.

www.sistemasyconectividad.com.mx http://darkavngr.blogspot.com/

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Trevor Talbot 2007-11-10 23:17:13 Re: 8.2.3: Server crashes on Windows using Eclipse/Junit
Previous Message Tony Caduto 2007-11-10 21:34:02 Verison 8.3 PL/pgSQL debugger Question