| From: | "Tristan Partin" <tristan(at)partin(dot)io> |
|---|---|
| To: | "Peter Eisentraut" <peter(at)eisentraut(dot)org> |
| Cc: | "Andres Freund" <andres(at)anarazel(dot)de>, "pgsql-hackers" <pgsql-hackers(at)postgresql(dot)org>, "Zsolt Parragi" <zsolt(dot)parragi(at)percona(dot)com> |
| Subject: | Re: compiler attribute alloc_size |
| Date: | 2026-11-16 17:49:07 |
| Message-ID: | DMWTPA8UKGPT.22H43NQMBL2KK@partin.io |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
On Fri Nov 13, 2026 at 11:51 AM UTC, Peter Eisentraut wrote:
> In my ventures to apply more compiler attributes, I came to alloc_size.
> This is to annotate memory allocation functions so that the compiler can
> know the size of the allocated memory and so some cross-checking based
> on that. This is all pretty straightforward and I think we should do it.
>
> Andres had previously reported some issues he encountered in [0]. (That
> message didn't go into details on what attributes or compiler options
> were used, but it certainly looked very closely related.)
>
> The issue is that under-allocating a struct is bad. With this
> attribute, the compiler flags this, because it looks like a mistake.
> For uses where this is legitimate, you'd normally use a struct with a
> trailing flexible-array member, and we mostly make use of that. But
> there are a couple of cases where we assign to a struct pointer less
> memory than what the struct expects. In some cases, this is done with a
> union where we only allocate enough memory for one branch of the union,
> which seems like an intuitive thing to do but is actually also bad.
>
> In the attached patches I have some fixes and/or workarounds for the
> issues that I found.
>
> Note that as documented this attribute and the associated warning
> options are for diagnostics only, they don't change compilation. So in
> that sense, once all warnings are fixed, this shouldn't create any more
> hidden issues.
>
> - varattrib_4b is a union of a normal varlena with a 4-byte header and 0
> or more payload bytes, and a compressed format with a 4-byte header, a
> 4-byte "tcinfo", followed by payload data. The typical coding pattern
>
> val = palloc(VARHDRSZ + len);
> SET_VARSIZE(val, VARHDRSZ + len);
>
> could get flagged by -Warray-bounds because the allocation could be too
> short to cover both branches of the union. In this case, I think we
> don't actually need the union, so I'm replacing this with two separate
> structs.
>
> - TRGM in contrib/pg_trgm/ is an instance of what [0] talks about: It
> has a trailing flexible array member, but it also has trailing padding,
> so allocating palloc(TRGMHDRSIZE) causes a warning, because it's less
> than sizeof(TRGM). In my patch I work around this by allocating like
> palloc(Max(TRGMHDRSIZE, sizeof(TRGM))).
Maybe we should add an allocator function or macro for this? In the
counted_by discussion, I talked a bit about how core does a pretty good
job of providing allocator helpers for structs with flexible array
members. I realize this is contrib/, but it might still be valuable.
> - There is a bit of a maze of different structs and unions that underlie
> the numeric type. The allocation (Numeric) palloc(NUMERIC_HDRSZ_SHORT)
> gets flagged by the compiler. In my patch I change this to (Numeric)
> palloc(sizeof(struct NumericData)), which allocates a bit more but makes
> the sizes and types match. Maybe some more judicious reshuffling of the
> underlying data structures could avoid this, or maybe it's okay like this.
Like the previous comment, I wonder if some helper allocator
functions/macros could be useful. But as you said, maybe some
reshuffling could avoid that all together, not sure.
> - As of commit 6a3da13d376 from this week, FormData_pg_attribute has 4
> bytes of trailing padding, and so sizeof(FormData_pg_attribute) is now
> different from ATTRIBUTE_FIXED_PART_SIZE. This causes a warning in one
> place. We had a discussion about this some time ago [1], where the
> opinions diverged between like, it's important to keep this distinction
> in case we ever have padding bytes, and, it's already incorrect and
> would be even more incorrect if we had padding bytes. ;-) Well, we're
> here now, so perhaps this should be revisited.
In [1], Tom said:
> What I'm trying to figure out is if we need to put in place some
> provisions to prevent there from being pad bytes at the end of any
> catalog struct.
I figured I could propose at least one way to flag padding bytes in the
attached patch. I am currently not proposing a solution.
Two options proposed in the thread[1]:
- Placeholder fields exactly matching the size of the padding bytes.
#ifdef CATALOG_VARLEN /* variable-length fields start here */
pg_node_tree ev_qual;
pg_node_tree ev_action;
#else
uint8 dummy;
#endif
- Use different structs for on-media and in-memory representations.
To not rock the boat too much, maybe I would throw my hat in the ring
for option 1? It seems like the simplest and most immediate solution.
> This is all that I found. [0] lists a lot more sources of potential
> problems, but apparently they are not flagged as they are currently used.
>
>
> [0]:
> https://www.postgresql.org/message-id/flat/axxbw7ku4w24hwe6uvkm4vdufo4zr4k5344m6naahkpsoafag5%40llovwgptzbgq
>
> [1]:
> https://www.postgresql.org/message-id/flat/7c7d41c1-fd5d-f9a5-afd5-f00168d91fe2%402ndquadrant.com
Anyway, thanks for the patch. I think the attribute is valuable, and
it's good that we are having this discussion.
--
Tristan Partin
PostgreSQL Contributors Team
AWS (https://aws.amazon.com)
| Attachment | Content-Type | Size |
|---|---|---|
| nocfbot-v1-trailing-padding-test.patch | text/x-patch | 12.3 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Stepan Neretin | 2026-11-16 17:58:12 | FATAL: could not send end-of-streaming message to primary: no COPY in progress |
| Previous Message | Melanie Plageman | 2026-11-16 17:37:22 | Re: WAL_LOG CREATE DATABASE strategy broken for non-standard page layouts |