From: | Bolek Ziobrowski <boleslaw(dot)ziobrowski(at)yahoo(dot)pl> |
---|---|
To: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
Cc: | pgsql-bugs(at)postgresql(dot)org |
Subject: | Re: BUG #14384: pg_dump uses excessive amounts of memory for LOBs |
Date: | 2016-10-20 21:22:32 |
Message-ID: | 969e577e-27da-f174-dd1e-f731abfc428a@yahoo.pl |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-bugs |
Tom Lane wrote:
>
> TL;DR: blobs are not exactly lightweight objects. If you want something
> with less overhead, maybe you should just store the data in a plain
> bytea column.
>
Yes, I tested this case with bytea/TOAST in place of LOB column and
pg_dump used just a few MB .
Normally (when dumping a database with hundreds of tables) it uses
insignificant amount of memory so I was surprised when it allocated
about 26GB while processing a 100GB database with 30*10^6 LOBs (caused
swapping and almost triggered OOM killer).
The 'obvious' (for database this size) and easy solution was to use
pg_basebackup - and it sped up the process almost 40 times .
I found a similar case here
(https://www.postgresql.org/message-id/524C3163.1050502%40iqbuzz.ru) so
I think it would be good to warn users about this behavior in pg_dump
documentation (to save a few hours of the third person who triggers this
issue in ... October 2019).
Thank you,
Bolek Ziobrowski
From | Date | Subject | |
---|---|---|---|
Next Message | mahmoudhakh | 2016-10-21 09:32:00 | BUG #14389: Duplicate rows on tables with unique index constrain |
Previous Message | Kevin Grittner | 2016-10-20 13:31:27 | Re: BUG #14387: Trying to send mail to general@postgresql.org |