Re: Add index scan progress to pg_stat_progress_vacuum

From: "Imseih (AWS), Sami" <simseih(at)amazon(dot)com>
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, "Drouvot, Bertrand" <bertranddrouvot(dot)pg(at)gmail(dot)com>, vignesh C <vignesh21(at)gmail(dot)com>, "Nathan Bossart" <nathandbossart(at)gmail(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, "Bossart, Nathan" <bossartn(at)amazon(dot)com>, Peter Geoghegan <pg(at)bowt(dot)ie>, Justin Pryzby <pryzby(at)telsasoft(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Add index scan progress to pg_stat_progress_vacuum
Date: 2023-01-12 14:02:33
Message-ID: 72CD33F6-C2B5-45E4-A78F-85EC923DCF0F@amazon.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Thanks for the feedback and I apologize for the delay in response.

> I think the problem here is that you're basically trying to work around the
> lack of an asynchronous state update mechanism between leader and workers. The
> workaround is to add a lot of different places that poll whether there has
> been any progress. And you're not doing that by integrating with the existing
> machinery for interrupt processing (i.e. CHECK_FOR_INTERRUPTS()), but by
> developing a new mechanism.

> I think your best bet would be to integrate with HandleParallelMessages().

You are correct. I have been trying to work around the async nature
of parallel workers performing the index vacuum. As you have pointed out,
integrating with HandleParallelMessages does appear to be the proper way.
Doing so will also avoid having to do any progress updates in the index AMs.

In the attached patch, the parallel workers send a new type of protocol message
type to the leader called 'P' which signals the leader that it should handle a
progress update. The leader then performs the progress update by
invoking a callback set in the ParallelContext. This is done inside HandleParallelMessages.
In the index vacuum case, the callback is parallel_vacuum_update_progress.

The new message does not contain a payload, and it's merely used to
signal the leader that it can invoke a progress update.

Also, If the leader performs the index vacuum, it can call parallel_vacuum_update_progress
directly inside vacuumparallel.c.

Regards,

Sami Imseih
Amazon Web Services (AWS)

Attachment Content-Type Size
v22-0001-Add-2-new-columns-to-pg_stat_progress_vacuum.-Th.patch application/octet-stream 16.6 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Andrew Dunstan 2023-01-12 14:09:43 Re: on placeholder entries in view rule action query's range table
Previous Message Matthias van de Meent 2023-01-12 13:55:11 Re: Named Operators