Re: Changing the autovacuum launcher scheduling; oldest table first algorithm

From: Grigory Smolkin <g(dot)smolkin(at)postgrespro(dot)ru>
To: pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Re: Changing the autovacuum launcher scheduling; oldest table first algorithm
Date: 2018-02-28 17:25:38
Message-ID: fb3e8a5d-660d-8efa-718e-58d519fbabd3@postgrespro.ru
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 02/28/2018 12:04 PM, Masahiko Sawada wrote:
> Hi,
>
> I've created the new thread for the changing AV launcher scheduling.
> The problem of AV launcher scheduling is described on [1] but I
> summarize it here.
>
> If there is even one database that is at risk of wraparound, currently
> AV launcher selects the database having the oldest datfrozenxid until
> all tables in that database have been vacuumed. This leads that tables
> on other database can bloat and not be frozen because other database
> are not selected by AV launcher. It makes things worse if the database
> has a large table that takes a long time to be vacuumed.
>
> Attached patch changes the AV launcher scheduling algorithm based on
> the proposed idea by Robert so that it selects a database first that
> has the oldest table when the database is at risk of wraparound. For
> detail of the algorithm please refer to [2].
>
> In this algorithm, the running AV workers advertise the next
> datfrozenxid on the shared memory that we will have. That way, AV
> launcher can select a database that has the oldest table in the
> database cluster. However, this algorithm doesn't support the case
> where the age of next datfrozenxid gets greater than
> autovacum_vacuum_max_age. This case can happen if users set
> autovacvuum_vacuum_max_age to small value and vacuuming the whole
> database takes a long time. But since it's not a common case and it
> doesn't degrade than current behavior even if this happened, I think
> it's not a big problem.
>
> Feedback is very welcome.
>
> [1] https://www.postgresql.org/message-id/0A3221C70F24FB45833433255569204D1F8A4DC6%40G01JPEXMBYT05
> [2] https://www.postgresql.org/message-id/CA%2BTgmobT3m%3D%2BdU5HF3VGVqiZ2O%2Bv6P5wN1Gj%2BPrq%2Bhj7dAm9AQ%40mail.gmail.com
>
> Regards,
>
> --
> Masahiko Sawada
> NIPPON TELEGRAPH AND TELEPHONE CORPORATION
> NTT Open Source Software Center

Hello!
I`ve tried to compile your patch on Fedora24 with gcc 6.3.1 20161221:

gcc -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -g3 -O0 -I../../../src/include
-D_GNU_SOURCE -c -o autovacuum.o autovacuum.c
In file included from ../../../src/include/postgres.h:46:0,
from autovacuum.c:62:
autovacuum.c: In function ‘get_next_xid_bounds’:
autovacuum.c:1979:9: warning: implicit declaration of function
‘TransactionIdIsVaild’ [-Wimplicit-function-declaration]
Assert(TransactionIdIsVaild(frozenxid));
^
../../../src/include/c.h:713:7: note: in definition of macro ‘Trap’
if (condition) \
^~~~~~~~~
autovacuum.c:1979:2: note: in expansion of macro ‘Assert’
Assert(TransactionIdIsVaild(frozenxid));
^~~~~~
autovacuum.c:1980:28: error: ‘minmutli’ undeclared (first use in this
function)
Assert(MultiXactIdIsValid(minmutli));
^
../../../src/include/c.h:713:7: note: in definition of macro ‘Trap’
if (condition) \
^~~~~~~~~
autovacuum.c:1980:2: note: in expansion of macro ‘Assert’
Assert(MultiXactIdIsValid(minmutli));
^~~~~~
autovacuum.c:1980:9: note: in expansion of macro ‘MultiXactIdIsValid’
Assert(MultiXactIdIsValid(minmutli));
^~~~~~~~~~~~~~~~~~
autovacuum.c:1980:28: note: each undeclared identifier is reported only
once for each function it appears in
Assert(MultiXactIdIsValid(minmutli));
^
../../../src/include/c.h:713:7: note: in definition of macro ‘Trap’
if (condition) \
^~~~~~~~~
autovacuum.c:1980:2: note: in expansion of macro ‘Assert’
Assert(MultiXactIdIsValid(minmutli));
^~~~~~
autovacuum.c:1980:9: note: in expansion of macro ‘MultiXactIdIsValid’
Assert(MultiXactIdIsValid(minmutli));
^~~~~~~~~~~~~~~~~~
<builtin>: recipe for target 'autovacuum.o' failed
make[3]: *** [autovacuum.o] Error 1

--
Grigory Smolkin
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Andres Freund 2018-02-28 17:44:52 Re: Parallel index creation & pg_stat_activity
Previous Message Robert Haas 2018-02-28 17:23:28 Re: [HACKERS] path toward faster partition pruning