Re: COPY Performance

On Mon, May 5, 2008 at 9:03 AM, Scott Marlowe <scott(dot)marlowe(at)gmail(dot)com> wrote:
> On Sun, May 4, 2008 at 5:11 PM, Hans Zaunere <lists(at)zaunere(dot)com> wrote:
> > Hello,
> >
> > We're using a statement like this to dump between 500K and >5 million rows.
> >
> > COPY(SELECT SomeID FROM SomeTable WHERE SomeColumn > '0')
> > TO '/dev/shm/SomeFile.csv'
>
> Wait, are you really creating a .csv file in shared memory? Can such
> a thing even work?
>
> If you're not creating a file in /dev/shm, where is it being built?
> On the main OS drive? the battery backed cached RAID array?

OK, looked it up. you're making your .csv file in a ramdisk? If it
gets big it's gonna make the machine start swapping. i'd suggest
storing only small things in a ram disk really. Other than that, I'm
not sure what the problem is.