Re: Restricting maximum keep segments by repslots

From: Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>
To: andres(at)anarazel(dot)de
Cc: peter(dot)eisentraut(at)2ndquadrant(dot)com, pgsql-hackers(at)postgresql(dot)org
Subject: Re: Restricting maximum keep segments by repslots
Date: 2017-11-08 04:14:31
Message-ID: 20171108.131431.170534842.horiguchi.kyotaro@lab.ntt.co.jp
Views: Raw Message | Whole Thread | Download mbox
Thread:
Lists: pgsql-hackers

Hello,

At Mon, 6 Nov 2017 05:20:50 -0800, Andres Freund <andres(at)anarazel(dot)de> wrote in <20171106132050(dot)6apzynxrqrzghb4r(at)alap3(dot)anarazel(dot)de>
> Hi,
>
> On 2017-10-31 18:43:10 +0900, Kyotaro HORIGUCHI wrote:
> > - distance:
> > how many bytes LSN can advance before the margin defined by
> > max_slot_wal_keep_size (and wal_keep_segments) is exhasuted,
> > or how many bytes this slot have lost xlog from restart_lsn.
>
> I don't think 'distance' is a good metric - that's going to continually
> change. Why not store the LSN that's available and provide a function
> that computes this? Or just rely on the lsn - lsn operator?

It seems reasonable.,The 'secured minimum LSN' is common among
all slots so showing it in the view may look a bit stupid but I
don't find another suitable place for it. distance = 0 meant the
state that the slot is living but insecured in the previous patch
and that information is lost by changing 'distance' to
'min_secure_lsn'.

Thus I changed the 'live' column to 'status' and show that staus
in text representation.

status: secured | insecured | broken

So this looks like the following (max_slot_wal_keep_size = 8MB,
which is a half of the default segment size)

-- slots that required WAL is surely available
select restart_lsn, status, min_secure_lsn, pg_current_wal_lsn() from pg_replication_slots;
restart_lsn | status | min_recure_lsn | pg_current_wal_lsn
------------+---------+----------------+--------------------
0/1A000060 | secured | 0/1A000000 | 0/1B42BC78

-- slots that required WAL is still available but insecured
restart_lsn | status | min_recure_lsn | pg_current_wal_lsn
------------+-----------+----------------+--------------------
0/1A000060 | insecured | 0/1C000000 | 0/1D76C948

-- slots that required WAL is lost
# We should have seen the log 'Some replication slots have lost...'

restart_lsn | status | min_recure_lsn | pg_current_wal_lsn
------------+--------+----------------+--------------------
0/1A000060 | broken | 0/1C000000 | 0/1D76C9F0

I noticed that I abandoned the segment fragment of
max_slot_wal_keep_size in calculating in the routines. The
current patch honors the frament part of max_slot_wal_keep_size.

regards,

--
Kyotaro Horiguchi
NTT Open Source Software Center

Attachment Content-Type Size
0001-Add-WAL-releaf-vent-for-replication-slots.patch text/x-patch 4.4 KB
0002-Add-monitoring-aid-for-max_replication_slots.patch text/x-patch 9.5 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message David Rowley 2017-11-08 04:44:59 Re: path toward faster partition pruning
Previous Message Andres Freund 2017-11-08 03:40:32 Re: Parallel Hash take II