Re: PG 14 release notes, first draft

On Tue, May 11, 2021 at 09:26:36AM +0530, Amit Kapila wrote:
> On Mon, May 10, 2021 at 7:30 PM Justin Pryzby <pryzby(at)telsasoft(dot)com> wrote:
> >
> > Can these be merged:
> > Allow logical replication to stream long transactions to standbys (Dilip Kumar, Tomas Vondra, Amit Kapila, Nikhil Sontakke)
> > Improve the logical replication API to allow streaming large in-progress transactions (Tomas Vondra, Dilip Kumar, Amit Kapila)
> >
>
> I think we should merge the below two:
> Add support for streaming to built-in logical replication (Vondra,
> Dilip Kumar, Amit Kapila, Ajin Cherian, Nikhil Sontakke, Stas Kelvich)
>
> Allow logical replication to stream long transactions to standbys
> (Dilip Kumar, Tomas Vondra, Amit Kapila, Nikhil Sontakke)
>
> and say something like:
> Allow logical replication to stream long in-progress transactions to standbys
> Previously transactions that exceeded logical_decoding_work_mem were
> written to disk until the transaction completed.

OK, updated as you suggested.

> Also, the commits that lead to the above feature is:
> 2020-09-03 [464824323] Add support for streaming to built-in logical replicatio
> 2020-08-08 [7259736a6] Implement streaming mode in ReorderBuffer.

Yes, these were already listed above that item.

> * Improve the logical replication API to allow streaming large
> in-progress transactions (Tomas Vondra, Dilip Kumar, Amit Kapila)
> The output functions begin with "stream". test_decoding also supports these.
>
> In the above, I think it is better to say Enhance instead of Improve.

OK, word changed.

> * Add system view pg_stat_replication_slots to report replication slot
> activity (Sawada Masahiko, Amit Kapila)
>
> I think for this item, we can give Vignesh C as author credit because
> he helped a lot enhance/stabilize this feature after the initial
> commit. See
> https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=f5fc2f5b23d1b1dff60f8ca5dc211161df47eda4
> https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=cca57c1d9bf7eeba5b81115e0b82651cf3d8e4ea
> https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=592f00f8dec68038301467a904ac514eddabf6cd

OK, I have added him:

Add system view pg_stat_replication_slots to report replication slot
activity (Sawada Masahiko, Amit Kapila, Vignesh C)

> * Allow multiple xacts during table sync in logical replication (Peter
> Smith, Amit Kapila, and Takamichi Osumi)
>
> IMPORTANT?
>
> Yes, this is a big enough improvement in the initial table sync
> mechanism of logical replication that it is worth mentioning in
> Release Notes. One of the main user-visible effects this commit can
> have is that it will reduce the time for which we need to hold WAL on
> the server during the initial table sync phase. Then it can avoid
> large copies in case there is any error during the time when
> apply-worker and tablesync-worker synchronizes the apply location.

OK, thanks.

> * Enable logical replication to handle two phase commits (Ajin Cherian)
> This is controlled via pg_create_logical_replication_slot().
>
> We still haven't supported two-phase commits for built-in logical
> replication but we have exposed APIs for decoding plugins to achieve
> that. Using these APIs, we are planning to support it for built-in
> logical replication but that will be probably done in PG-15. So, I
> suggest we can modify it to something like:
>
> * Enhance logical decoding APIs to handle two-phase commits
> This is controlled via pg_create_logical_replication_slot().

OK, updated as you suggested.

> Most of the work for this feature is done via commits:
> https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=a271a1b50e9bec07e2ef3a05e38e7285113e4ce6
> https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=0aa8a01d04c8fe200b7a106878eebc3d0af9105c
>
> So, I think the author credit should be given to Ajin Cherian, Amit
> Kapila, Nikhil Sontakke, and Stas Kelvich

OK, the new item text is:

<listitem>
<!--
Author: Amit Kapila <akapila(at)postgresql(dot)org>
2020-12-30 [0aa8a01d0] Extend the output plugin API to allow decoding of prepar
Author: Amit Kapila <akapila(at)postgresql(dot)org>
2021-01-04 [a271a1b50] Allow decoding at prepare time in ReorderBuffer.
Author: Amit Kapila <akapila(at)postgresql(dot)org>
2021-03-03 [19890a064] Add option to enable two_phase commits via pg_create_log
-->

<para>
Enhance logical decoding APIs to handle two-phase commits (Ajin
Cherian, Amit Kapila, Nikhil Sontakke, Stas Kelvich)
</para>

<para>
This is controlled via pg_create_logical_replication_slot().
</para>
</listitem>

> * I think we should mention about commit:
> https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=d7eb52d7181d83cf2363570f7a205b8eb1008dbc
>
> This allows Logical decoding to work efficiently in presence of a
> large amount of DDLs. This was a complaint from many users for long
> time. Apart from the discussion thread, see some of the earlier
> complaints:
> https://www.postgresql.org/message-id/flat/20f3de7675f83176253f607b5e199b228406c21c(dot)camel(at)cybertec(dot)at
> https://www.postgresql.org/message-id/flat/CAHoiPjzea6N0zuCi=+f9v_j94nfsy6y8SU7-=bp4=7qw6_i=Rg(at)mail(dot)gmail(dot)com
>
> and most recently at:
> https://www.postgresql.org/message-id/DB8PR10MB36126BC0DC3570637FDFED12EB4F9%40DB8PR10MB3612.EURPRD10.PROD.OUTLOOK.COM

OK, I added this:

<listitem>
<!--
Author: Amit Kapila <akapila(at)postgresql(dot)org>
2020-10-15 [d7eb52d71] Execute invalidation messages for each XLOG_XACT_INVALID
-->

<para>
Allow logical decoding to more efficently process cache invalidation messages
</para>

<para>
This allows Logical decoding to work efficiently in presence of a large amount of DDL.
</para>
</listitem>

> * Speed truncation of small tables on large shared buffer servers (Kirk Jamison)
>
> This optimization applies during recovery. Shouldn't we mention that
> in some way?

Yes, new text:

Speed truncation of small tables during recovery on clusters with a
large number of shared buffers (Kirk Jamison)

--
Bruce Momjian <bruce(at)momjian(dot)us> https://momjian.us
EDB https://enterprisedb.com

If only the physical world exists, free will is an illusion.