palloc() too large on pg_buffercache with large shared_buffers

From: Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com>
To: "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: palloc() too large on pg_buffercache with large shared_buffers
Date: 2016-09-14 04:13:58
Message-ID: 9A28C8860F777E439AA12E8AEA7694F80122A1AA@BPXM15GP.gisp.nec.co.jp
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hello,

It looks to me pg_buffercache tries to allocate more than 1GB using
palloc(), when shared_buffers is more than 256GB.

# show shared_buffers ;
shared_buffers
----------------
280GB
(1 row)

# SELECT buffers, d.datname, coalesce(c.relname, '???')
FROM (SELECT count(*) buffers, reldatabase, relfilenode
FROM pg_buffercache group by reldatabase, relfilenode) b
LEFT JOIN pg_database d ON d.oid = b.reldatabase
LEFT JOIN pg_class c ON d.oid = (SELECT oid FROM pg_database
WHERE datname = current_database())
AND b.relfilenode = pg_relation_filenode(c.oid)
ORDER BY buffers desc;
ERROR: invalid memory alloc request size 1174405120

It is a situation to use MemoryContextAllocHuge(), instead of palloc().
Also, it may need a back patching?

Thanks,
--
NEC Business Creation Division / PG-Strom Project
KaiGai Kohei <kaigai(at)ak(dot)jp(dot)nec(dot)com>

Attachment Content-Type Size
pgsql-fix-pg_buffercache-palloc-huge.patch application/octet-stream 851 bytes

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Dilip Kumar 2016-09-14 04:55:16 Re: Speed up Clog Access by increasing CLOG buffers
Previous Message Thomas Munro 2016-09-14 03:58:07 Re: kqueue