Re: WIP: long transactions on hot standby feedback replica / proof of concept

From: i(dot)kartyshov(at)postgrespro(dot)ru
To: pgsql-hackers(at)postgresql(dot)org
Subject: Re: WIP: long transactions on hot standby feedback replica / proof of concept
Date: 2017-09-15 14:39:45
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Alexander Korotkov писал 2017-09-05 00:33:
> I've following comments on this patch:
> 1) You shouldn't use ">=" to compare xids. You should use
> TransactionIdFollowsOrEquals() function which handles transaction id
> wraparound correctly.
I fixed it and as an additional I add GUC parameter that could turn off
this behavior. These parameter can be set in the postgresql.conf
configuration file, or with the help of the ALTER SYSTEM command. For
the changes to take effect, call the pg_reload_conf() function or reload
the database server.
Changes are in vacuum_lazy_truncate_v3.patch

> 2) As I understood, this patch makes heap truncate only when no
> concurrent transactions are running on both master and replicas with
> hot_standby_feedback enabled. For busy system, it would be literally
> "never do heap truncate after vacuum".
Yes, the difficulty is that if we have a lot of replicas and requests
for them will overlap each other, then truncation will not occur on the
loaded system

> Perhaps, it's certainly bad idea to disable replaying
> AccessExclusiveLock *every time*, we should skip taking
> AccessExclusiveLock only while writing XLOG_SMGR_TRUNCATE record.
> Besides that, this code violates out coding style.
In patch (standby_truncate_skip_v2.patch) fixed this behaviour and now
it skip writing XLOG_STANDBY_LOCK records (with in AccessExclusiveLock)
only while autovacuum truncations the table.

Amit Kapila писал 2017-09-05 12:52:
> I think one deficiency of this solution is that it will keep on
> generating extra WAL even if standby doesn't need it (as standby has
> successfully truncated the relation). I don't know if we can just get
> away by saying that an additional WAL record per vacuum cycle is
> harmless especially when that doesn't serve any purpose (like for the
> cases when standby is always able to truncate the heap on first WAL
> record). Am I missing some part of solution which avoids it?
You are right, and this implementation generating extra WALs and it
could have some cases. If you have any solutions to avoid it, I`ll be
glade to hear them.

Simon Riggs писал 2017-09-05 14:44:
> If we defer an action on standby replay, when and who will we apply
> it? What happens if the standby is shutdown or crashes while an action
> is pending.
> Perhaps altering the way truncation requires an AccessExclusiveLock
> would help workloads on both master and standby? If a Relation knew it
> had a pending truncation then scans wouldn't need to go past newsize.
> Once older lockers have gone we could simply truncate without the
> lock. Would need a few pushups but seems less scary then trying to
> make pending standby actions work well enough to commit.
Yes, it is the first idea that come to my mind and the most difficult
one. I think that in a few days I'll finish the patch with such ideas of

Ivan Kartyshov
Postgres Professional:
The Russian Postgres Company

Attachment Content-Type Size
standby_truncate_skip_v2.patch text/x-diff 4.7 KB
vacuum_lazy_truncate_v3.patch text/x-diff 3.1 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2017-09-15 14:55:17 Re: pgsql: Add support for coordinating record typmods among parallel worke
Previous Message Bossart, Nathan 2017-09-15 14:36:33 Re: Support to COMMENT ON DATABASE CURRENT_DATABASE