Re: Add pg_stat_vfdcache view for VFD cache statistics

Hi all,

On Wed, Aug 12, 2026 at 3:02 PM Ayoub Kazar <kazarayoub2004(at)gmail(dot)com> wrote:
>
> Hello everyone,
>
> This is Ayoub again, I've just changed my email address.
>
> On Sun, Jul 12, 2026 at 11:57 PM David Geier <geidav(dot)pg(at)gmail(dot)com> wrote:
>>
>> On 29.04.2026 15:45, KAZAR Ayoub wrote:
>> >>>> The global cache stats is going to be virtually free (at least the
>> >>>> hits/misses, I'm not sure about the number of entries and bytes), and
>> >>>> it's obviously useful for tuning the max_files_per_process GUC. I'd even
>> >>>> contemplate getting this into PG19, maybe.
>> >>
>> >> The number of used entries already exists, see nfile in fd.c.
>> >>
>> > Would one want the number of all entries (i.e SizeVfdCache see fd.c) or the
>> > number of used entries (i.e entries with fds in use, which is nfile) ? I
>> > thought of the first, that's what 0002 patch contains for the moment.
>>
>> I thought we would expose both. That way we can assess in the field if
>> being able to shrink the cache would be useful.
>
> Done.
>>
>>
>> >> Including the total cache size would also be virtually free if we don't
>> >> iterate over all VFDs each time, but update the size as we go. That
>> >> would have to happen when resizing the cache and when populating /
>> >> freeing a cache entry because extra memory is allocated / freed for
>> >> Vfd::fileName.
>> >>
>> > Is it a big deal if we miss some bytes of filename globally ?
>>
>> It's not just some bytes. sizeof(struct vfd) is 56 bytes and fileName
>> looks typically something like:
>>
>> - base/5/1249
>> - pg_wal/000000010000000000000001
>> - pg_wal/archive_status/000000010000000000000001.ready
>> - pg_xact/0000
>> - pg_multixact/offsets/0000
>>
>> File names can vary in length between 10 - 55 bytes, give or take. Most
>> files will be table and index segments and WAL files. We could maybe add
>> a fixed constant as "assumed average file name length" but I'm worried
>> that might end up being quite wrong.
>
> I chose to keep an accurate running count of the memory footprint per backend by tracking both the sizeof(Vfd) and the exact filename string lengths.
>
> We add the string length to the total footprint when a file is opened, and subtract it when the VFD is freed (i suppose this is not "Too much" of a work done, although it's done in a bit hot place); v5 of the patch is attached.
>>
>>

Thank you for the updated patches. I have completed the review and
testing of the v5 patch series for introducing the pg_stat_vfdcache
view and VFD cache footprint metrics. I tested the patch on PostgreSQL
20devel with the debug and assertion-enabled build and verified the
following:

1. Successfully built and ran the patched PostgreSQL tree.
2. Verified the newly introduced pg_stat_vfdcache view and its
reported statistics.
3. Tested VFD hit/miss accounting through repeated table accesses.
4. Tested VFD cache behavior using a partitioned table with 1500
partitions and 1.5 million rows.
5. Tested VFD pressure with a reduced max_files_per_process setting.
6. Verified the behavior of: hits, misses, open_entries,
allocated_entries, cache_bytes, max_open_fds, and hit_ratio.
7. Tested repeated scans and observed the corresponding changes in VFD
cache statistics.
8. Tested statistics reset using pg_stat_reset_shared('vfdcache').
9. Verified the behavior across multiple backend sessions.
10. Tested the statistics behavior across server restart.
11. Inspected the implementation of VFD cache footprint accounting in
fd.c, including VFD allocation, reuse, filename allocation/freeing,
and error handling.

And also reviewed the statistics flush mechanism and confirmed that
VFD hit/miss counters are maintained locally and flushed through the
existing fixed-statistics infrastructure rather than taking a
shared-memory lock on every VFD access.
The main point I noticed is related to regression test coverage. The
current regression test verifies that pg_stat_reset_shared('vfdcache')
updates the stats_reset timestamp, but it does not directly exercise
and verify the newly introduced VFD statistics such as hits, misses,
open_entries, allocated_entries, and cache_bytes. I suggest adding a
focused regression test covering the actual VFD activity and the newly
exposed statistics, while avoiding assertions on fragile exact counter
values where appropriate.
Apart from these, I did not observe any functional issues during my
testing and the patch looks good to me.

Regards,
Solai