Re: BUG #12910: Memory leak with logical decoding

From: Andres Freund <andres(at)anarazel(dot)de>
To: pet(dot)slavov(at)gmail(dot)com
Cc: pgsql-bugs(at)postgresql(dot)org
Subject: Re: BUG #12910: Memory leak with logical decoding
Date: 2015-03-31 14:40:48
Message-ID: 20150331144048.GL4878@alap3.anarazel.de
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

Hi,

On 2015-03-27 09:47:57 +0000, pet(dot)slavov(at)gmail(dot)com wrote:
> I am trying to use logical decoding to replay the data modifying queries on
> a different server, to synchronize some of the tables. The primary server
> has some load, but not so much. I am getting the changes with
> pg_logical_slot_get_changes limiting the changes to 50 at a time.

That's generally not a very good idea. It's much better to use the
streaming interface. Repeatedly starting/stopping a logical slot (as the
SQL interface has to do every time) isn't all that efficient.

> At some point pg_logical_slot_get_changes queries become slow and starts to
> eat all the ram and swap, which eventually kills the primary database with
> this error:
> FATAL: the database system is in recovery mode.

That indicates a crash. Please check the server log for any details? If
it crashes, could you perhaps get a backtrace?

What output plugin are you using?

> The nature of changes on the primary can have a lot of data in one
> transaction. Which I guess is the reason of the slow
> pg_logical_slot_get_changes.

In that case changes should just be spooled to disk.

Greetings,

Andres Freund

In response to

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message David G. Johnston 2015-03-31 15:01:14 Re: BUG #12908: tstzrange constructor fails when used in WHERE clause
Previous Message Bruce Momjian 2015-03-31 14:27:18 Re: minor: contrib/btree_gin/btree_gin.c uses DirectFunctionCall3(inet_in,..)