Should we warn against using too many partitions?

From: David Rowley <david(dot)rowley(at)2ndquadrant(dot)com>
To: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Should we warn against using too many partitions?
Date: 2019-05-23 09:02:40
Message-ID: CAKJS1f-2rx+E9mG3xrCVHupefMjAp1+tpczQa9SEOZWyU7fjEA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Over on [1] I raised a concern about the lack of any warning in our
documents to inform users that they might not want to use thousands of
partitions. More recently there's [2], also suffering from OOM using
100 partitions. Perhaps there's more too this, but the planner using
a lot of memory planning updates and deletes to partitioned tables
does seem to be a surprise to many people.

I had hoped we could get something it the documents sooner rather than
later about this. Probably the v12 patch will need to be adjusted now
that the memory consumption will be reduced when many partitions are
pruned, but I still think v12 needs to have some sort of warning in
there.

https://commitfest.postgresql.org/23/2065/

I'm moving this to a new thread with a better title, rather than
tagging onto that old thread that's become rather long.

[1] https://www.postgresql.org/message-id/CAKJS1f8RW-mHQ8aEWD5Dv0+8A1wH5tHHdYMGW9y5sXqnE0X9wA@mail.gmail.com
[2] https://www.postgresql.org/message-id/87ftp6l2qr.fsf@credativ.de

--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Julian Schauder 2019-05-23 09:18:37 Re: Excessive memory usage in multi-statement queries w/ partitioning
Previous Message Andrew Gierth 2019-05-23 08:57:18 Re: Why could GEQO produce plans with lower costs than the standard_join_search?