Re: pg_upgrade on high number tables database issues

On Mon, Mar 10, 2014 at 09:54:36AM -0700, Jeff Janes wrote:
> On Mon, Mar 10, 2014 at 6:58 AM, Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com> wrote:
>
> Hello
>
> I had to migrate our databases from 9.1 to 9.2. We have high number of
> databases per cluster (more than 1000) and high number of tables (indexes)
> per database (sometimes more than 10K, exceptionally more than 100K).
>
> I seen two problems:
>
> a) too long files pg_upgrade_dump_db.sql, pg_upgrade_dump_all.sql in
> postgres HOME directory. Is not possible to change a directory for these
> files.
>
>
> Those files should go into whatever your current directory is when you execute
> pg_upgrade. Why not just cd into whatever directory you want them to be in?
>
>
>
> b) very slow first stage of upgrade - schema export is very slow without
> high IO or CPU utilization.
>
>
> Just the pg_upgrade executable has low IO and CPU utilization, or the entire
> server does?
>
> There were several bottlenecks in this area removed in 9.2 and 9.3.
> Unfortunately the worst of those bottlenecks were in the server, so they depend
> on what database you are upgrading from, and so won't help you much upgrading
> from 9.1.

Yes, I assume 9.3 will be much better, though Jeff is right that if it
is pg_dump locking that is hurting you, you might not see a win even in
9.3.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +