Skip site navigation (1) Skip section navigation (2)

Re: In pg_upgrade, copy fsm, vm, and extent files by checking for fi

From: Bruce Momjian <bruce(at)momjian(dot)us>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: PostgreSQL-development <pgsql-hackers(at)postgreSQL(dot)org>
Subject: Re: In pg_upgrade, copy fsm, vm, and extent files by checking for fi
Date: 2012-11-24 03:31:13
Message-ID: 20121124033113.GC9382@momjian.us (view raw or flat)
Thread:
Lists: pgsql-committerspgsql-hackers
On Wed, Nov 14, 2012 at 06:28:41PM -0500, Bruce Momjian wrote:
> On Wed, Nov 14, 2012 at 06:15:30PM -0500, Tom Lane wrote:
> > Bruce Momjian <bruce(at)momjian(dot)us> writes:
> > > On Wed, Nov 14, 2012 at 05:39:29PM -0500, Tom Lane wrote:
> > >> You would have been better off keeping the array and sorting it so you
> > >> could use binary search, instead of passing the problem off to the
> > >> filesystem.
> > 
> > > Well, testing showed using open() was a big win.
> > 
> > ... on the filesystem you tested on.  I'm concerned that it might not
> > look so good on other platforms.
> 
> True. I am on ext3.  So I need to generate a proof-of-concept patch and
> have others test it?

OK, test patch attached.  The patch will only work if your database uses
a single tablespace.  Doing multiple tablespaces seemed too complex for
the test patch.

Here are my results:

        # tables      git    bsearch patch
	    1        11.16     10.99
	 1000        19.13     19.26
	 2000        26.78     27.11
	 4000        43.81     42.15
	 8000        79.96     77.38
	16000       165.26    162.24
	32000       378.18    368.49
	64000      1083.35   1086.77
 
As you can see, the bsearch code doesn't see to make much difference. 
Sometimes it is faster, other times, slower ---  seem to be just
measurement noise.  This code uses the same method of file lookup as git
head, meaning it looks for specific files rather than patterns ---  this
simplified the bsearch code.

Can anyone see a consistent improvement with the bsearch patch? 
Attached is my test shell script.  There is no reason we can't use
bsearch(), except not using it allows us to remove the pg_upgrade
directory listing function.
 
-- 
  Bruce Momjian  <bruce(at)momjian(dot)us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + It's impossible for everything to be true. +

Attachment: test.sh
Description: application/x-sh (2.3 KB)
Attachment: pg_upgrade.diff
Description: text/x-diff (8.3 KB)

In response to

pgsql-hackers by date

Next:From: Amit kapilaDate: 2012-11-24 05:10:45
Subject: Re: Proposal for Allow postgresql.conf values to be changed via SQL
Previous:From: Bruce MomjianDate: 2012-11-24 03:22:00
Subject: Use of fsync; was Re: Pg_upgrade speed for many tables

pgsql-committers by date

Next:From: Bruce MomjianDate: 2012-11-25 03:12:44
Subject: pgsql: In pg_upgrade, fix a few place that used maloc/free rather than
Previous:From: Peter EisentrautDate: 2012-11-23 21:25:34
Subject: pgsql: Remove -Wlogical-op from standard compiler flags

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group