[HACKERS] [bug-fix] Cannot select big bytea values (~600MB)

From: Anna Akenteva <a(dot)akenteva(at)postgrespro(dot)ru>
To: pgsql-hackers(at)postgresql(dot)org
Subject: [HACKERS] [bug-fix] Cannot select big bytea values (~600MB)
Date: 2018-02-16 14:23:50
Message-ID: c8bdf802d41ec37003ec3b726db79428@postgrespro.ru
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hello!

If I create a big bytea value and try to select it from a table, I get
an error, something like: "ERROR: invalid memory alloc request size
...".

So basically we can insert data into a table but then we can't even work
with it. Sounds like a bug. Attaching a patch that fixes it (applies to
2a41507dab0f293ff241fe8ae326065998668af8).

And as it seems like quite a serious issue, would it be possible to
backport a fix for it to earlier versions?

HOW TO RECREATE:
1) generate some random data (in this case, 600 MB):
dd if=/dev/urandom of=rand.dat bs=1M count=600

2) postgres=# select lo_import('/PATH/TO/rand.dat');
lo_import
-----------
16397 [USE THIS ID FOR THE NEXT STEP]
(1 row)

3) postgres=# create table big_data as select (string_agg(data,'')) as
data from pg_largeobject where loid =16397;
SELECT 1

4) postgres=# select * from big_data;
ERROR: invalid memory alloc request size 1468006403

--
Anna Akenteva
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Attachment Content-Type Size
string_info_master.patch text/x-diff 7.6 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2018-02-16 14:58:29 Re: [HACKERS] [bug-fix] Cannot select big bytea values (~600MB)
Previous Message Masahiko Sawada 2018-02-16 12:48:14 Re: autovacuum: change priority of the vacuumed tables