Skip site navigation (1) Skip section navigation (2)

brin autosummarization -- autovacuum "work items"

From: Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>
To: Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: brin autosummarization -- autovacuum "work items"
Date: 2017-03-01 04:58:23
Message-ID: 20170301045823.vneqdqkmsd4as4ds@alvherre.pgsql (view raw, whole thread or download thread mbox)
Thread:
Lists: pgsql-hackers
I think one of the most serious issues with BRIN indexes is how they
don't get updated automatically as the table is filled.  This patch
attempts to improve on that.  During brininsert() time, we check whether
we're inserting the first item on the first page in a range.  If we are,
request autovacuum to do a summarization run on that table.  This is
dependent on a new reloption for BRIN called "autosummarize", default
off.

The way the request works is that autovacuum maintains a DSA which can
be filled by backends with "work items".  Currently, work items can
specify a BRIN summarization of some specific index; in the future we
could use this framework to request other kinds of things that do not
fit in the "dead tuples / recently inserted tuples" logic that autovac
currently uses to decide to vacuum/analyze tables.

However, it seems I have not quite gotten the hang of DSA just yet,
because after a couple of iterations, crashes occur.  I think the reason
has to do with either a resource owner clearing the DSA at an unwelcome
time, or perhaps there's a mistake in my handling of DSA "relative
pointers" stuff.

This patch was initially written by Simon Riggs, who envisioned that
brininsert itself would invoke the summarization.  However, this doesn't
work because summarization requires having ShareUpdateExclusive lock,
which brininsert doesn't have.  So I modified things to instead use the
DSA stuff.  (He also set things up so that brininsert would only
summarize the just-filled range, but I didn't preserve that idea in the
autovacuum-based implementation; some changed lines there can probably
be removed.)

-- 
Álvaro Herrera                PostgreSQL Expert, https://www.2ndQuadrant.com/

Attachment: brin-autosummarize.patch
Description: text/plain (24.1 KB)

Responses

pgsql-hackers by date

Next:From: Venkata B NagothiDate: 2017-03-01 04:59:16
Subject: Re: patch proposal
Previous:From: Peter EisentrautDate: 2017-03-01 04:51:17
Subject: Re: rename pg_log directory?

Privacy Policy | About PostgreSQL
Copyright © 1996-2017 The PostgreSQL Global Development Group