Skip site navigation (1) Skip section navigation (2)

Re: Bulk Insert tuning

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Simon Riggs <simon(at)2ndquadrant(dot)com>
Cc: pgsql-patches(at)postgresql(dot)org
Subject: Re: Bulk Insert tuning
Date: 2008-02-26 20:12:58
Message-ID: 7279.1204056778@sss.pgh.pa.us (view raw or flat)
Thread:
Lists: pgsql-patches
Simon Riggs <simon(at)2ndquadrant(dot)com> writes:
> Following patch implements a simple mechanism to keep a buffer pinned
> while we are bulk loading.

This will fail to clean up nicely after a subtransaction abort, no?
(For that matter I don't think it's right even for a top-level abort.)
And I'm pretty sure it will trash your table entirely if someone
inserts into another relation while a bulk insert is happening.
(Not at all impossible, think of triggers for instance.)

From a code structural point of view, we are already well past the
number of distinct options that heap_insert ought to have.  I was
thinking the other day that bulk inserts ought to use a ring-buffer
strategy to avoid having COPY IN trash the whole buffer arena, just
as we've taught COPY OUT not to.  So maybe a better idea is to
generalize BufferAccessStrategy to be able to handle write as well
as read concerns; or have two versions of it, one for writing and one
for reading.  In any case the point being to encapsulate all these
random little options in a struct, which could also carry along
state that needs to be saved across a series of inserts, such as
the last pinned buffer.

			regards, tom lane

In response to

Responses

pgsql-patches by date

Next:From: Simon RiggsDate: 2008-02-26 21:36:36
Subject: Re: Bulk Insert tuning
Previous:From: Neil ConwayDate: 2008-02-26 20:09:48
Subject: Re: SRF memory leaks

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group