| From: | Ron Johnson <ronljohnsonjr(at)gmail(dot)com> |
|---|---|
| To: | pgsql-general <pgsql-general(at)postgresql(dot)org> |
| Subject: | Re: Dealing with SeqScans when Time-based Partitions Cut Over |
| Date: | 2025-12-18 19:32:07 |
| Message-ID: | CANzqJaAXhYwR6vHzm_DqnxmWk8QCYxEmhs_JeZ+w3F+rx09xSw@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-general |
On Thu, Dec 18, 2025 at 1:48 PM Matthew Planchard <msplanchard(at)gmail(dot)com>
wrote:
>
> In a table with high insert frequency (~1.5k rows/s) and high query
> frequency (~1k queries/s), partitioned by record creation time, we have
> observed the following behavior:
>
> * When the current time crosses a partition boundary, all new records
> are written to the new partition, which was previously empty, as
> expected
>
> * Because the planner's latest knowledge of the partition was based on
> its state prior to the cutover, it assumes the partition is empty and
> creates plans that use sequential scans
>
> * The table accumulates tens to hundreds of thousands of rows, and the
> sequentail scans start to use nearly 100% of available database CPU
>
> * Eventually the planner updates thee stats and all is well, but the
> cycle repeats the next time the partitions cut over.
>
> We have tried setting up a cron job that runs ANALYZE on the most recent
> partition of the table every 15 seconds at the start of the hour, and
> while this does help in reducing the magnitude and duration of the
> problem, it is insufficient to fully resolve it (our engineers are still
> getting daily pages for high DB CPU utilization).
>
What's autovacuum_analyze_scale_factor set to? The default 20% is pretty
high.
autovacuum_naptime might need to be dropped, too.
And maybe have the shell script that the cron job runs sleep only 5 seconds
in the ANALY loop.
> We have considered maintaining a separate connection pool with
> connections that have `enable_seqscan` set to `off`, and updating the
> application to use that pool for these queries, but I was hoping the
> community might have some better suggestions.
>
How about just force seqscan off when the table is created?
ALTER TABLE <table_partition> SET (enable_seqscan = off);
--
Death to <Redacted>, and butter sauce.
Don't boil me, I'm still alive.
<Redacted> lobster!
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Matthew Planchard | 2025-12-18 19:55:12 | Re: Dealing with SeqScans when Time-based Partitions Cut Over |
| Previous Message | Matthew Planchard | 2025-12-18 18:52:22 | Dealing with SeqScans when Time-based Partitions Cut Over |