Skip site navigation (1) Skip section navigation (2)

getting rid of freezing

From: Andres Freund <andres(at)2ndquadrant(dot)com>
To: pgsql-hackers(at)postgresql(dot)org
Subject: getting rid of freezing
Date: 2013-05-23 17:51:48
Message-ID: 20130523175148.GA29374@alap2.anarazel.de (view raw or flat)
Thread:
Lists: pgsql-hackers
Hi,

after having discussed $subject shortly over dinner yesterday, while I
should have been preparing the slides for my talk I noticed that there
might be a rather easy way to get rid of freezing.

I think that the existence of hint bits and the crash safe visibility
maps should provide sufficient tooling to make freezing unneccessary
without loosing much information for debugging if we modify the way
vacuum works a bit.

Currently, aside from recovery, we only set all visible in vacuum.

vacuumlazy.c's lazy_scan_heap currently works like:

for (blkno = 0; blkno < nblocks; blkno++)
{
    if (!scan_all && invisible)
       continue;

    /* cannot lock buffer immediately */
    if (!ConditionalLockBufferForCleanup(buf))
    {
        if (!scan_all)
            continue;

        /* don't block if we don't need freezing */
        if (!lazy_check_needs_freeze(buf))
           continue;

        /* now wait for cleanup lock */
        LockBufferForCleanup(buf);
    }

    for (tuple in all_tuples)
    {
        cleanup_tuple();
    }

    if (nfrozen > 0)
       log_heap_freeze()

    if (all_visible)
    {
        PageSetAllVisible(page);
        visibilitymap_set(page);
    }
}

In other words, if we don't need to make sure there aren't any old
tuples, we only scan visible parts of the relation. If we are making a
freeze vacuum we scan the whole relation, waiting for a cleanup lock on
the relation if necessary.

We currently need to make sure we scanned the whole relation and have
frozen everything to have a sensible relfrozenxid for a relation.

So, what I propose instead is basically:
1) only vacuum non-all-visible pages, even when doing it for
   anti-wraparound
2) When we can set all-visible guarantee that all tuples on the page are
   fully hinted. During recovery do the same, so we don't need to log
   all hint bits.
   We can do this with only an exclusive lock on the buffer, we don't
   need a cleanup lock.
3) When we cannot mark a page all-visible or we cannot get the cleanup
   lock, remember the oldest xmin on that page. We could set all visible
   in the former case, but we want the page to be cleaned up sometime
   soonish.
4) If we can get the cleanup lock, purge dead tuples from the page and
   the indexes, just as today. Set the page as all-visible.

That way we know that any page that is all-visible doesn't ever need to
look at xmin/xmax since we are sure to have set all relevant hint
bits.

We don't even necessarily need to log the hint bits for all items since
the redo for all_visible could make sure all items are hinted. The only
problem is knowing up to where we can truncate pg_clog...

Makes sense?

Greetings,

Andres Freund

--
 Andres Freund	                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services


Responses

pgsql-hackers by date

Next:From: Pavan DeolaseeDate: 2013-05-23 17:51:56
Subject: Re: pg_rewind, a tool for resynchronizing an old master after failover
Previous:From: Heikki LinnakangasDate: 2013-05-23 17:48:24
Subject: Re: pg_rewind, a tool for resynchronizing an old master after failover

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group