Managing my own index partitions

From: Chris Cleveland <ccleve+github(at)dieselpoint(dot)com>
To: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Managing my own index partitions
Date: 2022-08-08 18:40:00
Message-ID: CABSN6VdDgNVjqpydmdRu1zFeTnLFY8mipsR0jVTjbFeg0eJoig@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

I'm building a Postgres index access method. For a variety of reasons it's
more efficient to store the index data in multiple physical files on disk
rather in the index's main fork.

I'm trying to create separate rels that can be created and destroyed by the
parent index access method. I've succeeded in creating them with
heap.c/heap_create() and also with heap_create_with_catalog(). Where I'm
struggling is in getting the rels to be dropped when the parent index is
dropped.

When I use heap_create(), followed by recordDependencyOn(parent_oid,
child_oid), the child doesn't get dropped when the parent index is dropped.

When I use heap_create_with_catalog(), followed by
recordDependencyOn(parent_oid, child_oid), I get a "cache lookup failed for
index <my_child_oid>" triggered from within CommandCounterIncrement(), and
no amount of spelunking in the code turns up which cache entry is the
problem.

Backing up just a minute, am I going about this the right way?

Should I be using heap_create() at all?

Is there some other way to do this?

***********************************************
Here is my code:

Oid namespaceId = get_namespace_oid("relevantdb", false);
Oid tablespaceId = parent_index_rel->rd_rel->reltablespace;
Oid new_seg_oid = get_new_filenode(tablespaceId);
char *new_seg_name = make_seg_name(parent_index_rel, new_seg_oid);
Oid new_seg_filenode = InvalidOid; // heap_create() will create it

// this is required, unfortunately. It goes in the relcache.
// it creates a totally empty tupdesc. The natts=1 arg is to avoid an
error.
TupleDesc segTupleDesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(segTupleDesc, (AttrNumber) 1,
"dummy",
INT4OID,
-1, 0);
bool shared_relation = parent_index_rel->rd_rel->relisshared;
bool mapped_relation = RelationIsMapped(parent_index_rel);
bool allow_system_table_mods = false;

// not used
TransactionId relfrozenxid;
MultiXactId relminmxid;

Relation seg_rel = heap_create(
new_seg_name,
namespaceId,
tablespaceId,
new_seg_oid,
new_seg_filenode,
0, // access method oid
segTupleDesc,
RELKIND_INDEX, // or RELKIND_RELATION or
RELKIND_PARTITIONED_INDEX?
RELPERSISTENCE,
shared_relation,
mapped_relation,
allow_system_table_mods,
&relfrozenxid,
&relminmxid
);

Assert(relfrozenxid == InvalidTransactionId);
Assert(relminmxid == InvalidMultiXactId);
Assert(new_seg_oid == RelationGetRelid(seg_rel));

// make changes visible
CommandCounterIncrement();

// record dependency so seg gets dropped when index dropped
Oid parent_oid = parent_index_rel->rd_id;
record_dependency(parent_oid, new_seg_oid);

table_close(seg_rel, NoLock); /* do not unlock till end of xact */

...

void record_dependency(Oid parent_oid, Oid child_oid) {
ObjectAddress baseobject;
ObjectAddress segobject;
baseobject.classId = IndexRelationId;
baseobject.objectId = parent_oid;
baseobject.objectSubId = 0;
segobject.classId = IndexRelationId;
segobject.objectId = child_oid;
segobject.objectSubId = 0;
recordDependencyOn(&segobject, &baseobject, DEPENDENCY_INTERNAL);
}

The code where I use heap_create_with_catalog() is substantially the same.

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2022-08-08 18:44:29 Re: automatically generating node support functions
Previous Message Jonathan S. Katz 2022-08-08 18:35:25 Re: 2022-08-11 release announcement draft