Re: Infinite Autovacuum loop caused by failing virtual generated column expression

Hi,

On Mon, Apr 13, 2026 at 11:24 PM Yugo Nagata <nagata(at)sraoss(dot)co(dot)jp> wrote:
>
> Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com> wrote:
>
> > > PG19 added support for stats on virtual generated columns [1].
Creating extended statistics on a virtual generated column whose expression
can raise an error leads to ANALYZE failing repeatedly, and autovacuum
retrying indefinitely. This floods the server logs and also wastes
resources. Vacuum analyze on that column (without extended stats) succeeds.
>
> > True, though this is nothing new. The same thing can happen with
> > expression statistics on an expression that raises an error, which has
> > been possible since PG14.
>
> Yes, this issue is not new, and I’m not aware of a way to prevent it a
priori.
>
> > > In order to avoid retry storms, I think we have two options. (1)
skipping the offending row from the sample, (2) skipping the extended stats
computation for that table with a warning message. At least this avoid
autovacuum infinite retry. Attached a draft patch for the option (2).
Thoughts?
>
> > I'm not sure. The default retry interval is 1 minute, so it won't
> > exactly be a flood of messages. Also, if the error only occurs for a
> > small subset of rows, it's possible that retrying might succeed.
>
> I think it would be good to skip ANALYZE for the extended statistics that
cause
> errors and just emit a warning, rather than aborting ANALYZE for the
entire table.

I reviewed the patches and the discussion upthread. Here's my take on this.

When autovacuum has N tables to vacuum and any one of them errors out, the
remaining ones still get vacuumed (see the PG try-catch block around
autovacuum_do_vac_analyze() in do_autovacuum()). That is unlike a
database-wide vacuum or a vacuum with a table list. Those error out and do
not continue with the others in the list. By vacuum, I mean vacuum analyze
here.

That said, given this is expected behaviour, why do we need to fix it just
for generated columns?

There are basically three options, and the patch as-is is the weakest one:

1/ Accept it as expected behaviour. It's not new, it's not a flood, it may
succeed on a later retry, and the user can drop or fix the offending stats
object in this specific case.

2/ Fix it in general, in autovacuum, so that no error during analyze loops
forever, whatever the source is (extended stats, index stats, per-column
expression, or any other error). This is more work but it fixes the real
cause.

3/ Fix only extended-stats expressions, as the patch proposed upthread does
so far. This is the narrow one; it is not simple to reason about, and it
makes the behaviour inconsistent with the existing autovacuum behaviour.

I would like to understand why we are going with (3) and not (1) or (2). My
preference is (1).

FWIW, there is also an issue with the v2 patch. It catches the expression
error with PG try-catch and then continues in the same transaction without
re-throwing or aborting a subtransaction. Per elog.h, the recovery code has
to either re-throw or abort a (sub)transaction, otherwise the transaction
can be left in an inconsistent state. This is easy to hit if the statistics
expression acquires a resource before it errors, for instance a plpgsql
function that opens an internal query and then raises an error [1]:

WARNING: skipping statistics object "public.t_s" for relation "public.t"
DETAIL: division by zero
WARNING: transaction left non-empty SPI stack
HINT: Check for missing "SPI_finish" calls.
ANALYZE

> It seems reasonable to treat this as the user’s responsibility to notice
the warning
> and address the underlying issue.

Wouldn't an error draw more user attention than a WARNING that only goes to
the server log?

[1]
CREATE FUNCTION f(v int) RETURNS int LANGUAGE plpgsql
VOLATILE AS $$ DECLARE n int; BEGIN SELECT 1 INTO n; RETURN n / 0; END
$$;
CREATE TABLE t (a int);
INSERT INTO t VALUES (1), (2), (3);
CREATE STATISTICS t_s ON (f(a)) FROM t;
ANALYZE t;

--
Bharath Rupireddy
Amazon Web Services: https://aws.amazon.com