| From: | 河田達也 <kawatatatsuya0913(at)gmail(dot)com> |
|---|---|
| To: | Chao Li <li(dot)evan(dot)chao(at)gmail(dot)com> |
| Cc: | Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, pgsql-hackers(at)postgresql(dot)org |
| Subject: | Re: [PATCH] Add memory usage reporting to VACUUM VERBOSE |
| Date: | 2025-11-20 14:15:52 |
| Message-ID: | CAHza6qcJPFdPR3QoS8=QcLugKH9Sp3vgkiDOOB0ASOe8RdCbVg@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hi Sawada-san, Chao-san,
Thank you very much for your helpful feedback and testing.
> We can use vacrel->dead_items_info->max_bytes instead of calculating it
again.
I fixed it as you mentioned.
> I meant (1) actually. I don't think the number of indexes attached to
> the target relation is not relevant with the memory usage.
>
> We could add a new counter to track how many times we had to reset the
> dead items storage because it was full. The existing num_index_scans
> counter only shows how many times we performed index vacuuming. This
> means that when index vacuuming is disabled, we still collect dead
> items but num_index_scans shows 0. Adding this new counter would help
> users understand how often the dead items storage reaches capacity,
> even when index vacuuming is turned off.
Thank you for your detailed explanation; I finally understand.
I added a new counter to track the number of dead items storage resets
caused by reaching capacity.
> How about updating total bytes in dead_items_reset()?
Thank you. I fixed it as you mentioned.
> Another case we need to consider is where the table doesn't have an
index...
I fixed it as you mentioned.
> Where “max_bytes” is "/* the maximum bytes TidStore can use */“, but I
think we
> actually want to show max memories ever consumed during vacuuming, right?
Thank you for pointing out the confusing “max” wording.
The word "max” is not the peak memory *actually used* during VACUUM.
It represents the upper limit of memory that the TidStore is allowed to use
at once
(derived from maintenance_work_mem or autovacuum_work_mem).
So the current output may look misleading, especially when total usage is 0
but the limit is non-zero.
To make this more explicit, I updated the message wording to avoid implying
that
this is the maximum consumption, and instead describe it as the configured
memory limit.
The updated style is:
```
memory usage: total 1.02 MB used across 15 index scan(s) (max mem space is
limited 0.06 MB at once)
```
I believe this wording better reflects the actual meaning of max_bytes and
avoids
the interpretation that it is a peak usage value.
Regarding pgindent, running pgindent produced a large number of unrelated
changes across the file,
not only around the lines modified. Since it seemed inappropriate to
include those unrelated diffs,
I picked and applied only the necessary indentation changes for this patch.
I apologize for not mentioning this in the v3 submission.
I will revisit my pgindent setup to ensure it matches the project’s
expected behavior.
I have posted v4 incorporating all these changes.
Thank you again for your guidance.
Best regards,
Tatsuya Kawata
| Attachment | Content-Type | Size |
|---|---|---|
| v4-0001-Add-memory-usage-reporting-to-VACUUM-VERBOSE.patch | application/octet-stream | 3.5 KB |
| test_memory_tracking.sql | application/octet-stream | 4.7 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Alexander Pyhalov | 2025-11-20 14:22:32 | Re: Asynchronous MergeAppend |
| Previous Message | Mircea Cadariu | 2025-11-20 14:04:30 | Re: Metadata and record block access stats for indexes |