Re: Support allocating memory for large strings

From: Nathan Bossart <nathandbossart(at)gmail(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Maxim Zibitsker <max(dot)zibitsker(at)gmail(dot)com>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: Support allocating memory for large strings
Date: 2025-11-10 20:35:09
Message-ID: aRJMffw0OEScpr3F@nathan
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Fri, Nov 07, 2025 at 09:32:45PM -0500, Tom Lane wrote:
> Maxim Zibitsker <max(dot)zibitsker(at)gmail(dot)com> writes:
>> PostgreSQL's MaxAllocSize limit prevents storing individual
>> variable-length character strings exceeding ~1GB, causing "invalid
>> memory alloc request size" errors during INSERT operations on tables
>> with large text columns.
>
> This is news to no one. We are not especially interested in trying to
> relax that limit, because doing so would bleed over into approximately
> everything in the backend, and create opportunities for
> integer-overflow bugs in many places that are perfectly okay today.
> The cost-benefit ratio for changing this decision is horrible.

FWIW something I am hearing about more often these days, and what I believe
Maxim's patch is actually after, is the 1GB limit on row size. Even if
each field doesn't exceed 1GB (which is what artifacts.md seems to
demonstrate), heap_form_tuple() and friends can fail to construct the whole
tuple. This doesn't seem to be covered in the existing documentation about
limits [0].

[0] https://www.postgresql.org/docs/devel/limits.html

--
nathan

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Jeff Davis 2025-11-10 20:39:28 Re: Remaining dependency on setlocale()
Previous Message Thomas Munro 2025-11-10 20:27:47 Re: GNU/Hurd portability patches