Skip site navigation (1) Skip section navigation (2)

Re: Bulk Insert tuning

From: Simon Riggs <simon(at)2ndquadrant(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: pgsql-patches(at)postgresql(dot)org
Subject: Re: Bulk Insert tuning
Date: 2008-03-20 14:04:30
Message-ID: 1206021870.4285.550.camel@ebony.site (view raw or flat)
Thread:
Lists: pgsql-patches
On Tue, 2008-02-26 at 21:36 +0000, Simon Riggs wrote:
> On Tue, 2008-02-26 at 15:12 -0500, Tom Lane wrote:
> > Simon Riggs <simon(at)2ndquadrant(dot)com> writes:
> > > Following patch implements a simple mechanism to keep a buffer pinned
> > > while we are bulk loading.
> > 
> > This will fail to clean up nicely after a subtransaction abort, no?
> 
> Yes, will fix.

Additional line in AbortSubTransaction handles this.

> > (For that matter I don't think it's right even for a top-level abort.)
> > And I'm pretty sure it will trash your table entirely if someone
> > inserts into another relation while a bulk insert is happening.
> > (Not at all impossible, think of triggers for instance.)
> 
> The pinned buffer is separate from the preferred block for each
> relation; BulkInsertBuffer isn't used for determining the block to
> insert into. If you try to insert into a block that differs from the
> pinned one it unpins it and re-pins the new one. So it is always safe
> with respect to the data in the table.
> 
> It can run into recursive bulk insert ops but that just destroys the
> performance advantage, its not actually dangerous.

I'm about to start refactoring code as suggested, so wanted to drop off
another version to allow everybody to examine the safety/not of this
approach. (So this patch is WIP)

-- 
  Simon Riggs
  2ndQuadrant  http://www.2ndQuadrant.com 

  PostgreSQL UK 2008 Conference: http://www.postgresql.org.uk

Attachment: fast_copy.v4.patch
Description: text/x-patch (16.0 KB)

In response to

pgsql-patches by date

Next:From: Simon RiggsDate: 2008-03-20 14:07:43
Subject: Re: Moving snapshot code around
Previous:From: Alvaro HerreraDate: 2008-03-20 12:36:51
Subject: Re: Text <-> C string

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group