Re: Re: best practice for moving millions of rows to child table when setting up partitioning?

From: Scott Marlowe <scott(dot)marlowe(at)gmail(dot)com>
To: Mark Stosberg <mark(at)summersault(dot)com>
Cc: pgsql-admin(at)postgresql(dot)org
Subject: Re: Re: best practice for moving millions of rows to child table when setting up partitioning?
Date: 2011-04-28 00:26:14
Message-ID: BANLkTimihm0e4DewUxkSZ_YNs4wkK8VpZA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin

I had a similar problem about a year ago, The parent table had about
1.5B rows each with a unique ID from a bigserial. My approach was to
create all the child tables needed for the past and the next month or
so. Then, I simple did something like:

begin;
insert into table select * from only table where id between 1 and 10000000;
delete from only table where id between 1 and 10000000;
-- first few times check to make sure it's working of course
commit;
begin;
insert into table select * from only table where id between 10000001
and 20000000;
delete from only table where id between 10000001 and 20000000;
commit;

and so on. New entries were already going into the child tables as
they showed up, old entries were migrating 10M rows at a time. This
kept the moves small enough so as not to run the machine out of any
resource involved in moving 1.5B rows at once.

In response to

Responses

Browse pgsql-admin by date

  From Date Subject
Next Message Scott Marlowe 2011-04-28 00:26:58 Re: Re: best practice for moving millions of rows to child table when setting up partitioning?
Previous Message ktm@rice.edu 2011-04-27 20:50:56 Re: Re: best practice for moving millions of rows to child table when setting up partitioning?