Re: Freeze avoidance of very large table.

On Wed, Feb 17, 2016 at 4:44 PM, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
> On Wed, Feb 17, 2016 at 4:29 PM, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
>> On Wed, Feb 17, 2016 at 4:08 AM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>>> On Tue, Feb 16, 2016 at 03:57:01PM -0300, Alvaro Herrera wrote:
>>>> Masahiko Sawada wrote:
>>>> > On Wed, Feb 17, 2016 at 12:02 AM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>>>> > > On Tue, Feb 16, 2016 at 11:56:25PM +0900, Masahiko Sawada wrote:
>>>> > >> > I agreed on ripping out the converter plugin ability of pg_upgrade.
>>>> > >> > Remember pg_upgrade was originally written by EnterpriseDB staff, and I
>>>> > >> > think they expected their closed-source fork of Postgres might need a
>>>> > >> > custom page converter someday, but it never needed one, and at this
>>>> > >> > point I think having the code in there is just making things more
>>>> > >> > complex. I see _no_ reason for community Postgres to use a plugin
>>>> > >> > converter because we are going to need that code for every upgrade from
>>>> > >> > pre-9.6 to 9.6+, so why not just hard-code in the functions we need. We
>>>> > >> > can remove it once 9.5 is end-of-life.
>>>> > >> >
>>>> > >>
>>>> > >> Hm, we should rather remove the source code around PAGE_CONVERSION and
>>>> > >> page.c at 9.6?
>>>> > >
>>>> > > Yes. I can do it if you wish.
>>>> >
>>>> > I see. I understand that page-converter code would be useful for some
>>>> > future cases, but makes thing more complex.
>>>>
>>>> If we're not going to use it, let's get rid of it right away. There's
>>>> no point in having a feature that adds complexity just because we might
>>>> find some hypothetical use of it in a not-yet-imagined future.
>>>
>>> Agreed. We can always add it later if we need it.
>>>
>>
>> Attached patch gets rid of page conversion code.
>>
>

Attached updated 5 patches.
I would like to explain these patch shortly again here to make
reviewing more easier.

We can divided these patches into 2 purposes.

1. Freeze map
000_ patch adds additional frozen bit into visibility map, but doesn't
include the logic for improve freezing performance.
001_ patch gets rid of page-conversion code from pg_upgrade. (This
patch doesn't related to this feature essentially, but is required by
002_ patch.)
002_ patch adds upgrading mechanism from 9.6- to 9.6+ and its regression test.

2. Improve freezing logic
003_ patch changes the VACUUM to optimize scans based on freeze map
(i.g., 000_ patch), and its regression test.
004_ patch enhances debug messages in src/backend/access/heap/visibilitymap.c

Please review them.

Regards,

--
Masahiko Sawada