pg_partman 3.0.0 - real-world usage of native partitioning and a case for native default

From: Keith Fiske <keith(at)omniti(dot)com>
To: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: pg_partman 3.0.0 - real-world usage of native partitioning and a case for native default
Date: 2017-03-31 15:30:56
Message-ID: CAG1_KcA2pxO5+Fviwb-4nrdF+FXuqV9YkKyE+oH3FGGMjyZktA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

I've gotten pg_partman working with native partitioning already so I can
hopefully have things ready to work when 10 is released. I've got a branch
on github with this version for anyone to test and I'll hopefully have this
released in the next few weeks after I finish some more testing myself. Any
feedback would be appreciated!

https://github.com/keithf4/pg_partman/tree/v3.0.0

Thankfully since native partitioning still uses inheritance internally for
the most part, pg_partman works pretty well without nearly as much change
as I thought I would need. The biggest deficiency I'm seeing has to do with
not having a "default" partition to put data that doesn't match any
children. The fact that it throws an error is a concern, but it's not where
I see the main problem. Where this really comes into play is when someone
wants to make an existing table into a partitioned table. There's really no
easy way to do this outside of making a completely brand new partition set
and copying/moving the data from the old to the new. Yes there are
technically ways to do this fairly seamlessly to the user of the
partitioned table, but the complexity of those methods seems absurd in the
face of just allowing a default partition.

A default would basically allow a method similar to what pg_partman does
prior to native. Before I would just make the old table the parent and then
the user could move data to the children as needed, eventually leaving the
parent empty. All data is still accessible during this period and new
writes go to the new children. The old table can't be made the parent with
native obviously, but being able to define a default partition would allow
defining the old table as the default and pretty much give the exact same,
easy migration path. Without defining it as a default, you really can't
attach the old table since it would require a constraint interval that
would likely interfere with other children.

I recall reading before that there are performance implications of having
the default. I think as long as those performance issues are clearly
documented and have no affect if there is no default, it shouldn't be a
concern that would hold this feature up. I believe the benefit of the
default partition to the migration process makes it more than worth it and
I'm hoping it's possible to get into 10 so users can more easily use this
new feature without having to wait for the next major version. Spoke with
several core members at PGConf this year and they asked me to send an email
to hackers making my case, so here it is!

--
Keith Fiske
Database Administrator
OmniTI Computer Consulting, Inc.
http://www.keithf4.com

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Banck 2017-03-31 15:35:45 Re: [patch] reorder tablespaces in basebackup tar stream for backup_label
Previous Message Tom Lane 2017-03-31 15:29:24 Re: REFERENCES privilege should not be symmetric (was Re: [GENERAL] Postgres Permissions Article)