BUG #16167: frontend psql fails 'select * from table' if nrows * ncolumns = 2**32 in a table.

From: PG Bug reporting form <noreply(at)postgresql(dot)org>
To: pgsql-bugs(at)lists(dot)postgresql(dot)org
Cc: cilizili(at)protonmail(dot)com
Subject: BUG #16167: frontend psql fails 'select * from table' if nrows * ncolumns = 2**32 in a table.
Date: 2019-12-16 12:36:09
Message-ID: 16167-ceb3fb3f8732e776@postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

The following bug has been logged on the website:

Bug reference: 16167
Logged by: cili
Email address: cilizili(at)protonmail(dot)com
PostgreSQL version: 12.1
Operating system: CentOS7.7 (1908)
Description:

I tested in 13devel.
In the fe_utils and libpq, nrows as tuples and ncolumns as attributes are
int type parameters, and the maximum number of tuples is INT_MAX which is
hard coded.
I create a table with 67108864 tuples and 64 attributes in following sample.
I've made sure it is possible, even though it requires a lot of memory and
storage.
It causes integer overflows in the following two places [1] and [2], since
67108864 (0x04000000) times 64 (0x40) is 2**32 (0x100000000) that is larger
than UINT_MAX of 32 bits, then it is truncated to 0.

[1] src/fe_utils/print.c:L3084 in printTableInit()
content->cell = pg_malloc0((ncolumns * nrows + 1) *
sizeof(*content->cells));

[2] src/fe_utils/print.c:L3518 in printTableAddCell()
if (content->cellsadded >= content->ncolumns * content->nrows)
{
fprintf(stderr, ...

In the first place, (ncolumns * nrows + 1) will be truncated to 0+1=1, then
it allocates a small memory, such as 8 bytes.
Before to use it, the client process is terminated with error condition in
the second place where cellsadded has been initialized with 0 and compared
to ncoluns*nrows=0 within integer type.
The error message, "Cannot add header to table content: column count of 0
exceeded." may not be correct.

>>>>>>>>>>>>>>> insturctions and result <<<<<<<<<<<<<<<<

postgres=# psql postgres
psql (13devel)
Type "help" for help
postgres=# \i test.sql
CREATE FUNCTION
CREATE TABLE
foo
----

(1 row)

postgres=# select * from bar;
Cannot add header to table content: column count of 0 exceeded.

>>>>>>>>>>>>>>> test.sql <<<<<<<<<<<<<<<<
-- WARNING: This sample will require Mem: 16GB, Swap: 72GB, 22GB of storage,
and a lot of time.
-- It takes 2**32 cells in a table where the tuples(columns) and
attributes(columns) are within the limits specified in the document.

-- create function
CREATE OR REPLACE FUNCTION foo(n INTEGER)
RETURNS VOID AS $$
BEGIN
FOR i IN 1..n LOOP
INSERT INTO bar (
id, i0, i1, i2, i3, i4, i5, i6,
i7, i8, i9, i10,i11,i12,i13,i14,
i15,i16,i17,i18,i19,i20,i21,i22,
i23,i24,i25,i26,i27,i28,i29,i30,
i31,i32,i33,i34,i35,i36,i37,i38,
i39,i40,i41,i42,i43,i44,i45,i46,
i47,i48,i49,i50,i51,i52,i53,i54,
i55,i56,i57,i58,i59,i60,i61,i62
) VALUES (
i, 0, 1, 2, 3, 4, 5, 6,
7, 8, 9, 10,11,12,13,14,
15,16,17,18,19,20,21,22,
23,24,25,26,27,28,29,30,
31,32,33,34,35,36,37,38,
39,40,41,42,43,44,45,46,
47,48,49,50,51,52,53,54,
55,56,57,58,59,60,61,62);
END LOOP;
END;
$$ LANGUAGE plpgsql;

-- create table
CREATE TABLE bar (
id INTEGER, i0 INTEGER, i1 INTEGER, i2 INTEGER,
i3 INTEGER, i4 INTEGER, i5 INTEGER, i6 INTEGER,
i7 INTEGER, i8 INTEGER, i9 INTEGER, i10 INTEGER,
i11 INTEGER,i12 INTEGER,i13 INTEGER,i14 INTEGER,
i15 INTEGER,i16 INTEGER,i17 INTEGER,i18 INTEGER,
i19 INTEGER,i20 INTEGER,i21 INTEGER,i22 INTEGER,
i23 INTEGER,i24 INTEGER,i25 INTEGER,i26 INTEGER,
i27 INTEGER,i28 INTEGER,i29 INTEGER,i30 INTEGER,
i31 INTEGER,i32 INTEGER,i33 INTEGER,i34 INTEGER,
i35 INTEGER,i36 INTEGER,i37 INTEGER,i38 INTEGER,
i39 INTEGER,i40 INTEGER,i41 INTEGER,i42 INTEGER,
i43 INTEGER,i44 INTEGER,i45 INTEGER,i46 INTEGER,
i47 INTEGER,i48 INTEGER,i49 INTEGER,i50 INTEGER,
i51 INTEGER,i52 INTEGER,i53 INTEGER,i54 INTEGER,
i55 INTEGER,i56 INTEGER,i57 INTEGER,i58 INTEGER,
i59 INTEGER,i60 INTEGER,i61 INTEGER,i62 INTEGER,
PRIMARY KEY (id));

-- set values into table. It takes a lot of time.
select foo(67108864);

-- Let's test with 'select * from bar;' in interactive command line.

Browse pgsql-bugs by date

  From Date Subject
Next Message Tomas Vondra 2019-12-16 14:09:22 Re: BUG #16104: Invalid DSA Memory Alloc Request in Parallel Hash
Previous Message Heikki Linnakangas 2019-12-16 12:23:58 Re: Planning time is high in Postgres 11.5 Compared with Postgres 10.11