Skip site navigation (1) Skip section navigation (2)

Re: extending relations more efficiently

From: Stephen Frost <sfrost(at)snowman(dot)net>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: Andres Freund <andres(at)anarazel(dot)de>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: extending relations more efficiently
Date: 2012-05-01 19:48:04
Message-ID: (view raw or whole thread)
Lists: pgsql-hackers
* Robert Haas (robertmhaas(at)gmail(dot)com) wrote:
> I think the first thing we need here is a good test case, so we're
> clear on what we're trying to solve.  I was just hoping to make file
> extension *faster* and what you and Simon are talking about is making
> it scale better in the face of heavy parallelism; obviously it would
> be nice to do both things, but they are different problems.  Any old
> bulk-loading test will benefit from a raw performance improvement, but
> to test a scalability improvement we would need some kind of test case
> involving parallel bulk loads, or some other kind of parallel activity
> that causes rapid table growth.  That's not something I've frequently
> run into, but I'd be willing to put a bit of time into it if we can
> nail down what we're talking about.

Try loading the TIGER 2011 data into a single table, where you load each
county (or perhaps state) in a separate, parallel, process.  That's what
I was doing when I hit this lock full-force and bitched about it on this

I'd be happy to help construct that case, as well as test any changes in
this area which might help address it (on a 10G SSD-backed SAN..).



In response to


pgsql-hackers by date

Next:From: Alvaro HerreraDate: 2012-05-01 19:48:56
Subject: Re: port _srv.o makefile rules don't observe dependency tracking
Previous:From: Noah MischDate: 2012-05-01 19:36:22
Subject: Re: proposal: additional error fields

Privacy Policy | About PostgreSQL
Copyright © 1996-2015 The PostgreSQL Global Development Group