| From: | Peter Eisentraut <peter(at)eisentraut(dot)org> |
|---|---|
| To: | Tristan Partin <tristan(at)partin(dot)io> |
| Cc: | Andres Freund <andres(at)anarazel(dot)de>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>, Zsolt Parragi <zsolt(dot)parragi(at)percona(dot)com> |
| Subject: | Re: compiler attribute alloc_size |
| Date: | 2026-11-20 15:34:53 |
| Message-ID: | fe7b8ebb-2163-469c-a693-72aebaf68c55@eisentraut.org |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
On 16.11.26 18:49, Tristan Partin wrote:
>> - TRGM in contrib/pg_trgm/ is an instance of what [0] talks about: It
>> has a trailing flexible array member, but it also has trailing padding,
>> so allocating palloc(TRGMHDRSIZE) causes a warning, because it's less
>> than sizeof(TRGM). In my patch I work around this by allocating like
>> palloc(Max(TRGMHDRSIZE, sizeof(TRGM))).
>
> Maybe we should add an allocator function or macro for this? In the
> counted_by discussion, I talked a bit about how core does a pretty good
> job of providing allocator helpers for structs with flexible array
> members. I realize this is contrib/, but it might still be valuable.
>
>> - There is a bit of a maze of different structs and unions that underlie
>> the numeric type. The allocation (Numeric) palloc(NUMERIC_HDRSZ_SHORT)
>> gets flagged by the compiler. In my patch I change this to (Numeric)
>> palloc(sizeof(struct NumericData)), which allocates a bit more but makes
>> the sizes and types match. Maybe some more judicious reshuffling of the
>> underlying data structures could avoid this, or maybe it's okay like this.
>
> Like the previous comment, I wonder if some helper allocator
> functions/macros could be useful. But as you said, maybe some
> reshuffling could avoid that all together, not sure.
In the attached patch series, I have an idea for a more generalized
approach: A macro that computes the required size and then a wrapper
macro around it for allocating. This then can be applied to any
structure with a flexible array member, and it eliminates some
duplication with the offsetof calls and gets you the kind of type safety
that palloc_object and palloc_array provide.
I have also added a patch that makes some more uses of this macro.
These are the same kinds of problems as the one I showed for pg_trgm and
numeric, except that the size isn't constant and so the compiler won't
flag it.
>> - As of commit 6a3da13d376 from this week, FormData_pg_attribute has 4
>> bytes of trailing padding, and so sizeof(FormData_pg_attribute) is now
>> different from ATTRIBUTE_FIXED_PART_SIZE. This causes a warning in one
>> place. We had a discussion about this some time ago [1], where the
>> opinions diverged between like, it's important to keep this distinction
>> in case we ever have padding bytes, and, it's already incorrect and
>> would be even more incorrect if we had padding bytes. ;-) Well, we're
>> here now, so perhaps this should be revisited.
I have adjusted the description of this patch a bit. There are a lot of
details to be figured out in this area, but at least we can fix this
particular issue easily and in a consistent way. tupdesc.c already uses
sizeof(FormData_pg_attribute) consistently for allocating pg_attribute
structures. It only uses ATTRIBUTE_FIXED_PART_SIZE for things like
memcpy(). With the proposed patch, we make bootstrap.c consistent in
this regard, and we make the warning go away. The deeper issues can
then be researched independent of this.
> In [1], Tom said:
>
>> What I'm trying to figure out is if we need to put in place some
>> provisions to prevent there from being pad bytes at the end of any
>> catalog struct.
>
> I figured I could propose at least one way to flag padding bytes in the
> attached patch. I am currently not proposing a solution.
>
> Two options proposed in the thread[1]:
>
> - Placeholder fields exactly matching the size of the padding bytes.
>
> #ifdef CATALOG_VARLEN /* variable-length fields start here */
> pg_node_tree ev_qual;
> pg_node_tree ev_action;
> #else
> uint8 dummy;
> #endif
>
> - Use different structs for on-media and in-memory representations.
>
> To not rock the boat too much, maybe I would throw my hat in the ring
> for option 1? It seems like the simplest and most immediate solution.
I think we could get the same check simpler and more portably by having
genbki.pl generate some static assertions that check like
static_assert(sizeof(FormData_pg_attribute) ==
offsetof(FormData_pg_attribute, attcollation) +
sizeof(FormData_pg_attribute.attcollation));
But this flags a bunch of issues. Sorting that out might be a big project.
| Attachment | Content-Type | Size |
|---|---|---|
| v2-0001-Apply-alloc_size-attribute-to-memory-allocation-f.patch | text/plain | 11.8 KB |
| v2-0002-Fixes-for-alloc_size-attribute-varatt.patch | text/plain | 7.6 KB |
| v2-0003-Fixes-for-alloc_size-attribute-pg_trgm.patch | text/plain | 2.2 KB |
| v2-0004-Fixes-for-alloc_size-attribute-numeric.patch | text/plain | 1.2 KB |
| v2-0005-Fixes-for-alloc_size-attribute-bootstrap.patch | text/plain | 1.7 KB |
| v2-0006-WIP-flex_size.patch | text/plain | 6.2 KB |
| v2-0007-WIP-More-flex_size-candidates.patch | text/plain | 6.5 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | yuansong | 2026-11-20 15:37:56 | [PATCH v1] Fix stale composite type OIDs in cached PL/pgSQL expressions |
| Previous Message | Robert Haas | 2026-11-20 15:33:15 | Re: O(N^3) planning time in choose_plan_name() since 8c49a484e8e |