Fast Default WIP patch for discussion

As promised and requested find attached a work in progress patch for fast defaults.
This is my first patch, I hope I used the right format…..

The premise of the feature is to avoid a table rewrite when adding a column with a default.

This is done by remembering the default value in pg_attribute instead of updating all the rows.
When a tuple is read from disk which is shorter than the tuple descriptor mandates the default is “plugged in”
either when filling in the datum/null array of a virtual tuple, or by “expanding” the tuple to a complete heap or minimal tuple
with all attributes.

Example:
CREATE TABLE t (pk INT NOT NULL PRIMARY KEY);
INSERT INTO t VALUES (1);
=> t: (1)

ALTER TABLE t ADD COLUMN c1 INT DEFAULT 5;
=> 1. Stores the default expression in pg_attrdef.adbin (no news)
2. a. Stores the default values (computed at time of ALTER TABLE) in pg_attribute.att_existdef
b. Sets pg_attribute.att_hasexistdef to true

SELECT * FROM t;
=> Build a virtual tuple using getAttr() API (no news)
but since tuple has fewer mats than table signature fill in extra attributes from pg_attribute.att_existdef
if att_hasexistdef is true
=> (1) becomes (1, 5)

INSERT INTO t VALUES (2);
=> Fill in DEFAULT for t.c1 from pg_attrdef.adbin as usual: (2, 5)
=> t: (1), (2, 5)

ALTER TABLE t ALTER COLUMN c1 SET DEFAULT -1;
=> 1. Drop row from pg_attrdef for c1 (no news)
2. Add row to pg_attrdef for c1 DEFAULT -1 (no news)
3. Leave pg_atribute.att_existdef alone!!

INSERT INTO t VALUES (3);
=> Fill in DEFAULT for t.c1 from pg_attrdef.adbin as usual: (3, -1)
=> t: (1), (2, 5), (3, -1)

SELECT * FROM t;
=> Build a virtual tuple using get*Attr() API (no news)
but since tuple has fewer mats than table signature fill in extra attributes from pg_attribute.att_existdef
if att_hasexistdef is true
=> (1) becomes (1, 5)
(2, 5)
(3, -1)

ALTER TABLE t ALTER COLUMN c1 DROP DEFAULT;
=> 1. Drop row from pg_attrdef for c1 (no news)
2. Leave pg_atribute.att_existdef alone!!

INSERT INTO t VALUES (4);
=> Fill in default DEFAULT (4, NULL)
=> t: (1), (2, 5), (3, -1), (4, NULL)

SELECT * FROM t;
=> Build a virtual tuple using get*Attr() API (no news)
but since tuple has fewer mats than table signature fill in extra attributes from pg_attribute.att_existdef
if att_hasexistdef is true
=> (1) becomes (1, 5)
(2, 5)
(3, -1)
(4, NULL)

You can find a new (incomplete) test file fast_default.sql in regress/sql

Some key design points requiring discussion:
1. Storage of the “exist” (working name) default
Right now the patch stores the default value in its binary form as it would be in the tuple into a BYTEA.
It would be feasible to store the pretty printed literal, however this requires calling the io functions when a
tuple descriptor is built.
2. The exist default is cached alongside the “current” default in the tuple descriptor’s constraint structure.
Seems most natural too me, but debatable.
3. Delayed vs. early expansion of the tuples.
To avoid having to decide when to copy tuple descriptors with or without constraints and defaults
I have opted to expand the tuple at the core entry points.
How do I know I have them all? An omission means wrong results!
4. attisnull()
This routine is used in many places, but to give correct result sit must now be accompanied
by the tuple descriptor. This becomes moderately messy and it’s not always clear where to get that.
Interestingly most usages are related to catalog lookups.
Assuming we have no intention to support fast default for catalog tables we could keep using the
existing attisnull() api for catalog lookups and use a new version (name tbd) for user tables.
5. My head hurts looking at the PK/FK code - it’s not always clear which tuple descriptor belongs
to which tuple
6. Performance of the expansion code.
The current code needs to walk all defaults and then start padding by filling in values.
But the outcome is always the same. We will produce the same payload and the name null map.
It would be feasible to cache an “all defaults tuple”, remember the offsets (and VARWIDTH, HASNULL)
for each attribute and then simply splice the short and default tuples together.
This ought to be faster, but the meta data to cache is not insignificant and the expansion code is messy enough
without this already.
7. Grooming
Obviously we can remove all exist defaults for a table from pg_attribute whenever the table is rewrittem.
That’s easy.
But could we/should we keep track of the short tuples and either eliminate them or drop exist defaults once they
become obsolete because there is no tuple short enough for them to matter.
8. Do we need to worry about toasted defaults?

Cheers
Serge Rielau
Salesforce.com <http://salesforce.com/>