Re: Support allocating memory for large strings

Maxim Zibitsker <max(dot)zibitsker(at)gmail(dot)com> writes:
> PostgreSQL's MaxAllocSize limit prevents storing individual variable-length character strings exceeding ~1GB, causing "invalid memory alloc request size" errors during INSERT operations on tables with large text columns.

This is news to no one. We are not especially interested in trying to
relax that limit, because doing so would bleed over into approximately
everything in the backend, and create opportunities for
integer-overflow bugs in many places that are perfectly okay today.
The cost-benefit ratio for changing this decision is horrible.

> The attached patches demonstrates a proof of concept using
> palloc_extended with MCXT_ALLOC_HUGE in the write path.

"Proof of concept"? This can't possibly fix your problem, because it
does nothing for the fact that tuple size fields are still limited
to 1GB, as are varlena headers for individual fields. A serious
attack on this limitation, at a guess, would require a patch on the
order of 100K lines, and that might be an underestimate.

regards, tom lane