BUG #19063: Heavily nesting trivial ROW projections produces out of memory error

The following bug has been logged on the website:

Bug reference: 19063
Logged by: Lukas Eder
Email address: lukas(dot)eder(at)gmail(dot)com
PostgreSQL version: 18rc1
Operating system: Linux in Docker in Windows
Description:

Try this query:

select
row (row (row (row (row (row (row (row (
row (row (row (row (row (row (row (row (
row (row (row (row (row (row (row (row (
row (row (row (row (row (row (row (row (
1
))))))))
))))))))
))))))))
)))))))) as "nested";

It produces the following error:

SQL Error [54000]: ERROR: string buffer exceeds maximum allowed length
(1073741823 bytes)
Detail: Cannot enlarge string buffer containing 1073741822 bytes by 1 more
bytes.

I don't understand how such excessive memory consumption is produced by such
a "simple" query. The nesting is a bit excessive, sure, but I think there's
a deeper underlying inefficiency that might be worth addressing.

I've tried both these versions:

- PostgreSQL 17.5 (Debian 17.5-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled
by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
- PostgreSQL 18rc1 (Debian 18~rc1-1.pgdg13+1) on x86_64-pc-linux-gnu,
compiled by gcc (Debian 14.2.0-19) 14.2.0, 64-bit