Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

From: Anastasia Lubennikova <a(dot)lubennikova(at)postgrespro(dot)ru>
To: Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>
Cc: Tatsuo Ishii <ishii(at)sraoss(dot)co(dot)jp>, lubennikovaav(at)gmail(dot)com, pgsql-hackers(at)lists(dot)postgresql(dot)org, pavan(dot)deolasee(at)gmail(dot)com, ibrar(dot)ahmad(at)gmail(dot)com
Subject: Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits
Date: 2021-01-11 21:00:39
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 11.01.2021 01:35, Tomas Vondra wrote:
> Hi,
> I started looking at this patch again, hoping to get it committed in
> this CF, but I think there's a regression in handling TOAST tables
> (compared to the v3 patch submitted by Pavan in February 2019).
> The test I'm running a very simple test (see test.sql):
> 1) start a transaction
> 2) create a table with a text column
> 3) copy freeze data into it
> 4) use pg_visibility to see how many blocks are all_visible both in the
>    main table and it's TOAST table
> For v3 patch (applied on top of 278584b526 and
> s/hi_options/ti_options) I get this:
>            pages           NOT all_visible
>   ------------------------------------------
>   main       637                         0
>   toast    50001                         3
> There was some discussion about relcache invalidations causing a
> couple TOAST pages not be marked as all_visible, etc.
> However, for this patch on master I get this
>            pages           NOT all_visible
>   ------------------------------------------
>   main       637                         0
>   toast    50001                     50001
> So no pages in TOAST are marked as all_visible. I haven't investigated
> what's causing this, but IMO that needs fixing to make ths patch RFC.
> Attached is the test script I'm using, and a v5 of the patch - rebased
> on current master, with some minor tweaks to comments etc.

Thank you for attaching the test script. I reproduced the problem. This
regression occurs because TOAST internally uses heap_insert().
You have asked upthread about adding this optimization to heap_insert().

I wrote a quick fix, see the attached patch 0002. The TOAST test passes
now, but I haven't tested performance or any other use-cases yet.
I'm going to test it properly in a couple of days and share results.

With this change a lot of new code is repeated in heap_insert() and
heap_multi_insert(). I think it's fine, because these functions already
have a lot in common.

Anastasia Lubennikova
Postgres Professional:
The Russian Postgres Company

Attachment Content-Type Size
0001-Set-PD_ALL_VISIBLE-and-visibility-map-bits-in-COP-v5.patch text/x-patch 12.8 KB
0002_handle_HEAP_INSERT_FROZEN_in_heap_insert.patch text/x-patch 5.0 KB

In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message Peter Geoghegan 2021-01-11 21:09:48 Re: Deleting older versions in unique indexes to avoid page splits
Previous Message Stephen Frost 2021-01-11 20:52:28 Re: Key management with tests