Skip site navigation (1) Skip section navigation (2)

Re: extending relations more efficiently

From: Stephen Frost <sfrost(at)snowman(dot)net>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: Andres Freund <andres(at)anarazel(dot)de>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: extending relations more efficiently
Date: 2012-05-01 19:48:04
Message-ID: 20120501194804.GS1267@tamriel.snowman.net (view raw or flat)
Thread:
Lists: pgsql-hackers
* Robert Haas (robertmhaas(at)gmail(dot)com) wrote:
> I think the first thing we need here is a good test case, so we're
> clear on what we're trying to solve.  I was just hoping to make file
> extension *faster* and what you and Simon are talking about is making
> it scale better in the face of heavy parallelism; obviously it would
> be nice to do both things, but they are different problems.  Any old
> bulk-loading test will benefit from a raw performance improvement, but
> to test a scalability improvement we would need some kind of test case
> involving parallel bulk loads, or some other kind of parallel activity
> that causes rapid table growth.  That's not something I've frequently
> run into, but I'd be willing to put a bit of time into it if we can
> nail down what we're talking about.

Try loading the TIGER 2011 data into a single table, where you load each
county (or perhaps state) in a separate, parallel, process.  That's what
I was doing when I hit this lock full-force and bitched about it on this
list.

I'd be happy to help construct that case, as well as test any changes in
this area which might help address it (on a 10G SSD-backed SAN..).

	Thanks,

		Stephen

In response to

Responses

pgsql-hackers by date

Next:From: Alvaro HerreraDate: 2012-05-01 19:48:56
Subject: Re: port _srv.o makefile rules don't observe dependency tracking
Previous:From: Noah MischDate: 2012-05-01 19:36:22
Subject: Re: proposal: additional error fields

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group