| From: | Nathan Bossart <nathandbossart(at)gmail(dot)com> |
|---|---|
| To: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
| Cc: | Maxim Zibitsker <max(dot)zibitsker(at)gmail(dot)com>, pgsql-hackers(at)postgresql(dot)org |
| Subject: | Re: Support allocating memory for large strings |
| Date: | 2025-11-10 20:35:09 |
| Message-ID: | aRJMffw0OEScpr3F@nathan |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
On Fri, Nov 07, 2025 at 09:32:45PM -0500, Tom Lane wrote:
> Maxim Zibitsker <max(dot)zibitsker(at)gmail(dot)com> writes:
>> PostgreSQL's MaxAllocSize limit prevents storing individual
>> variable-length character strings exceeding ~1GB, causing "invalid
>> memory alloc request size" errors during INSERT operations on tables
>> with large text columns.
>
> This is news to no one. We are not especially interested in trying to
> relax that limit, because doing so would bleed over into approximately
> everything in the backend, and create opportunities for
> integer-overflow bugs in many places that are perfectly okay today.
> The cost-benefit ratio for changing this decision is horrible.
FWIW something I am hearing about more often these days, and what I believe
Maxim's patch is actually after, is the 1GB limit on row size. Even if
each field doesn't exceed 1GB (which is what artifacts.md seems to
demonstrate), heap_form_tuple() and friends can fail to construct the whole
tuple. This doesn't seem to be covered in the existing documentation about
limits [0].
[0] https://www.postgresql.org/docs/devel/limits.html
--
nathan
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Jeff Davis | 2025-11-10 20:39:28 | Re: Remaining dependency on setlocale() |
| Previous Message | Thomas Munro | 2025-11-10 20:27:47 | Re: GNU/Hurd portability patches |