Re: [HACKERS] pg_dump and thousands of schemas

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Denis <socsam(at)gmail(dot)com>
Cc: pgsql-performance(at)postgresql(dot)org
Subject: Re: [HACKERS] pg_dump and thousands of schemas
Date: 2012-11-06 17:51:04
Message-ID: 16061.1352224264@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers pgsql-performance

Denis <socsam(at)gmail(dot)com> writes:
> Here is the output of EXPLAIN ANALYZE. It took 5 seconds but usually it
> takes from 10 to 15 seconds when I am doing backup.

> Sort (cost=853562.04..854020.73 rows=183478 width=219) (actual
> time=5340.477..5405.604 rows=183924 loops=1)

Hmmm ... so the problem here isn't that you've got 2600 schemas, it's
that you've got 183924 tables. That's going to take some time no matter
what.

It does seem like we could make some small changes to optimize that
query a little bit, but they're not going to result in any amazing
improvement overall, because pg_dump still has to deal with all the
tables it's getting back. Fundamentally, I would ask whether you really
need so many tables. It seems pretty likely that you have lots and lots
of basically-identical tables. Usually it would be better to redesign
such a structure into fewer tables with more index columns.

> Here is the output of "pg_dump -s" test.dump
> <http://postgresql.1045698.n5.nabble.com/file/n5730877/test.dump>

This dump contains only 1 schema and 43 tables, so I don't think it's
for the database you're having trouble with ...

regards, tom lane

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2012-11-06 17:57:37 Re: Arguments to foreign tables?
Previous Message Andrew Dunstan 2012-11-06 17:38:07 Re: Doc patch, distinguish sections with an empty row in error code table

Browse pgsql-performance by date

  From Date Subject
Next Message ktm@rice.edu 2012-11-06 18:08:27 Re: Query completed in < 1s in PG 9.1 and ~ 700s in PG 9.2
Previous Message Petr Praus 2012-11-06 17:38:41 Re: Re: Increasing work_mem and shared_buffers on Postgres 9.2 significantly slows down queries