Re: Slow query needs a kick in the pants.

From: Dennis Gearon <gearond(at)cvc(dot)net>
To: Dann Corbit <DCorbit(at)connx(dot)com>
Cc: pgsql-general(at)postgresql(dot)org
Subject: Re: Slow query needs a kick in the pants.
Date: 2003-03-28 02:19:59
Message-ID: 3E83B14F.6050005@cvc.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Do it in two stages, find the primary keys first, then do a search on on IN such
and such query, (BUT 'IN' is slow, some others can recommend a faster alternative. )

Just an idea.

Dann Corbit wrote:
> The other query attempts given to me do not produce the desired
> results. Thanks for taking a look anyway.
>
> Let me explain the problem set, and what I want to accomplish...
>
> I have two tables in all cases. Each table pair consists of the
> following columns:
> 1. A primary key of one or more columns {with a unique index}
> 2. An Oid column {with a unique index}
> 3. A 64 bit CRC
>
> For both tables, the primary key information will "mostly" match. I
> need to know which primary keys are found in the first table but not in
> the second. Also, which primary keys are found in the second table but
> not in the first.
>
> One possibility would be to add a tag column and set it where the two
> tables agree on primary key. Unfortunately, we are likely to have many
> millions of rows (and only a few thousand disagreements) and so I would
> end up rewriting the entire table for both tables (less the tiny
> difference set) and hence that would be inefficient. My outer join
> also stinks up the place. (Several minutes for a million rows --
> unacceptable).
>
> Probably, I am going to have to write my own piece of custom software
> that manipulates the data outside of any database. The database query
> approach is just too slow.
>
>

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Jean-Christian Imbeault 2003-03-28 02:29:43 missing FROM-clause notice but nothing is missing ...
Previous Message Jeff Davis 2003-03-28 02:13:18 Re: Python Error