Skip site navigation (1) Skip section navigation (2)

Re: Largeobject Access Controls (r2460)

From: "Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov>
To: "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: "Greg Smith" <greg(at)2ndquadrant(dot)com>, "KaiGai Kohei" <kaigai(at)ak(dot)jp(dot)nec(dot)com>, "Robert Haas" <robertmhaas(at)gmail(dot)com>, "Takahiro Itagaki" <itagaki(dot)takahiro(at)oss(dot)ntt(dot)co(dot)jp>, <pgsql-hackers(at)postgresql(dot)org>, "Jaime Casanova" <jcasanov(at)systemguards(dot)com(dot)ec>
Subject: Re: Largeobject Access Controls (r2460)
Date: 2010-01-23 14:48:03
Message-ID: 4B5AB7C3020000250002EA9F@gw.wicourts.gov (view raw or flat)
Thread:
Lists: pgsql-hackers
Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote: 
> "Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov> writes:
>> Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>> Do you have the opportunity to try an experiment on hardware
>>> similar to what you're running that on?  Create a database with
>>> 7 million tables and see what the dump/restore times are like,
>>> and whether pg_dump/pg_restore appear to be CPU-bound or
>>> memory-limited when doing it.
>  
>> If these can be empty (or nearly empty) tables, I can probably
>> swing it as a background task.  You didn't need to match the
>> current 1.3 TB database size I assume?
> 
> Empty is fine.
 
After about 15 hours of run time it was around 5.5 million tables;
the rate of creation had slowed rather dramatically.  I did create
them with primary keys (out of habit) which was probably the wrong
thing.  I canceled the table creation process and started a VACUUM
ANALYZE, figuring that we didn't want any hint-bit writing or bad
statistics confusing the results.  That has been running for 30
minutes with 65 MB to 140 MB per second disk activity, mixed read
and write.  After a few minutes that left me curious just how big
the database was, so I tried:
 
select pg_size_pretty(pg_database_size('test'));
 
I did a Ctrl+C after about five minutes and got:
 
Cancel request sent
 
but it didn't return for 15 or 20 minutes.  Any attempt to query
pg_locks stalls.  Tab completion stalls.  (By the way, this is not
related to the false alarm on that yesterday, which was a result of
my attempting tab completion from within a failed transaction, which
just found nothing rather than stalling.)
 
So I'm not sure whether I can get to a state suitable for starting
the desired test, but I'll stay with a for a while.
 
-Kevin

In response to

Responses

pgsql-hackers by date

Next:From: Bernd HelmleDate: 2010-01-23 15:00:34
Subject: Re: commit fests (was Re: primary key error message)
Previous:From: Hitoshi HaradaDate: 2010-01-23 14:14:32
Subject: Re: commit fests

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group