Re: Freeze avoidance of very large table.

From: Sawada Masahiko <sawada(dot)mshk(at)gmail(dot)com>
To: Jim Nasby <Jim(dot)Nasby(at)bluetreble(dot)com>
Cc: Petr Jelinek <petr(at)2ndquadrant(dot)com>, Bruce Momjian <bruce(at)momjian(dot)us>, Robert Haas <robertmhaas(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, Greg Stark <stark(at)mit(dot)edu>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, Jeff Janes <jeff(dot)janes(at)gmail(dot)com>
Subject: Re: Freeze avoidance of very large table.
Date: 2015-05-28 02:34:28
Views: Raw Message | Whole Thread | Download mbox
Lists: pgsql-hackers

On Thu, Apr 30, 2015 at 8:07 PM, Sawada Masahiko <sawada(dot)mshk(at)gmail(dot)com> wrote:
> On Fri, Apr 24, 2015 at 11:21 AM, Sawada Masahiko <sawada(dot)mshk(at)gmail(dot)com> wrote:
>> On Fri, Apr 24, 2015 at 1:31 AM, Jim Nasby <Jim(dot)Nasby(at)bluetreble(dot)com> wrote:
>>> On 4/23/15 11:06 AM, Petr Jelinek wrote:
>>>> On 23/04/15 17:45, Bruce Momjian wrote:
>>>>> On Thu, Apr 23, 2015 at 09:45:38AM -0400, Robert Haas wrote:
>>>>> Agreed, no extra file, and the same write volume as currently. It would
>>>>> also match pg_clog, which uses two bits per transaction --- maybe we can
>>>>> reuse some of that code.
>>>> Yeah, this approach seems promising. We probably can't reuse code from
>>>> clog because the usage pattern is different (key for clog is xid, while
>>>> for visibility/freeze map ctid is used). But visibility map storage
>>>> layer is pretty simple so it should be easy to extend it for this use.
>>> Actually, there may be some bit manipulation functions we could reuse;
>>> things like efficiently counting how many things in a byte are set. Probably
>>> doesn't make sense to fully refactor it, but at least CLOG is a good source
>>> for cut/paste/whack.
>> I agree with adding a bit that indicates corresponding page is
>> all-frozen into VM, just like CLOG.
>> I'll change the patch as second version patch.
> The second patch is attached.
> In second patch, I added a bit that indicates all tuples in page are
> completely frozen into visibility map.
> The visibility map became a bitmap with two bit per heap page:
> all-visible and all-frozen.
> The logics around vacuum, insert/update/delete heap are almost same as
> previous version.
> This patch lack some point: documentation, comment in source code,
> etc, so it's WIP patch yet,
> but I think that it's enough to discuss about this.

The previous patch is no longer applied cleanly to HEAD.
The attached v2 patch is latest version.

Please review it.


Sawada Masahiko

Attachment Content-Type Size
000_add_frozen_bit_into_visibilitymap_v2.patch text/x-diff 57.6 KB

In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message Pavel Stehule 2015-05-28 02:46:02 Re: [COMMITTERS] pgsql: Add pg_audit, an auditing extension
Previous Message Alvaro Herrera 2015-05-28 02:14:40 Re: [HACKERS] Re: 9.4.1 -> 9.4.2 problem: could not access status of transaction 1