Skip site navigation (1) Skip section navigation (2)

Re: Help with extracting large volumes of records across related tables

From: Pierre-Frédéric Caillaud <lists(at)boutiquenumerique(dot)com>
To: pgsql-performance(at)postgresql(dot)org
Subject: Re: Help with extracting large volumes of records across related tables
Date: 2004-09-13 14:00:35
Message-ID: opsd9qi9udcq72hf@musicbox (view raw or flat)
Thread:
Lists: pgsql-performance
	Thanks for the thanks !

	Generally, when grouping stuff together, it is a good idea to have two  
sorted lists, and to scan them simultaneously. I have already used this  
solution several times outside of Postgres, and it worked very well (it  
was with Berkeley DB and there were 3 lists to scan in order). The fact  
that Python can very easily virtualize these lists using generators makes  
it possible to do it without consuming too much memory.

> Pierre-Frederic, Paul,
>
> Thanks for your fast response (especially for the python code and
> performance figure) - I'll chase this up as a solution - looks most
> promising!
>
> Cheers,
>
> Damien
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: Have you searched our list archives?
>
>                http://archives.postgresql.org
>



In response to

pgsql-performance by date

Next:From: Bruce MomjianDate: 2004-09-13 14:38:08
Subject: Re: Determine optimal fdatasync/fsync, O_SYNC/O_DSYNC options
Previous:From: Damien DouganDate: 2004-09-13 13:44:45
Subject: Re: Help with extracting large volumes of records across related tables

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group