*** doc/src/sgml/ref/postgres-ref.sgml.orig Thu Sep 18 16:30:15 2003 --- doc/src/sgml/ref/postgres-ref.sgml Mon Sep 22 20:10:32 2003 *************** *** 177,185 **** ! Ignore system indexes while scanning/updating system tables. The ! REINDEX command for system tables/indexes ! requires this option to be used. --- 177,185 ---- ! Ignore system indexes when reading system tables (but still update ! the indexes when modifying the tables). This is useful when ! recovering from damaged system indexes. *** doc/src/sgml/ref/reindex.sgml.orig Thu Sep 11 21:47:19 2003 --- doc/src/sgml/ref/reindex.sgml Mon Sep 22 20:23:17 2003 *************** *** 56,98 **** - - - If you suspect corruption of an index on a user table, you can - simply rebuild that index, or all indexes on the table, using - REINDEX INDEX or REINDEX - TABLE. Another approach to dealing with a corrupted - user-table index is just to drop and recreate it. This may in fact - be preferable if you would like to maintain some semblance of - normal operation on the table meanwhile. REINDEX - acquires exclusive lock on the table, while CREATE - INDEX only locks out writes not reads of the table. - - - - Things are more difficult if you need to recover from corruption of - an index on a system table. In this case it's important for the - system to not have used any of the suspect indexes itself. - (Indeed, in this sort of scenario you may find that server - processes are crashing immediately at start-up, due to reliance on - the corrupted indexes.) To recover safely, the server must be shut - down and a stand-alone PostgreSQL server - must be started instead with the command-line options - and . (These options allow - system table modifications and prevent use of system indexes, - respectively.) Then, REINDEX DATABASE, - REINDEX TABLE, or REINDEX INDEX can be - issued, depending on how much you want to reconstruct. If in - doubt, use REINDEX DATABASE FORCE to force - reconstruction of all system indexes in the database. Then quit - the standalone server session and restart the real server. - - - - See the reference page for more - information about how to interact with the stand-alone server - interface. - --- 56,61 ---- *************** *** 104,111 **** Recreate all system indexes of a specified database. Indexes on ! user tables are not included. This form of REINDEX ! can only be used in stand-alone mode (see above). --- 67,74 ---- Recreate all system indexes of a specified database. Indexes on ! user tables are not processed. Also, indexes on shared system ! catalogs are skipped except in stand-alone mode (see below). *************** *** 142,151 **** FORCE ! Force rebuild of system indexes. Without this key word, ! REINDEX skips system indexes that are not marked ! invalid. FORCE is irrelevant for REINDEX ! INDEX or when reindexing user indexes. --- 105,111 ---- FORCE ! This is an obsolete option; it is ignored if specified. *************** *** 153,158 **** --- 113,190 ---- + Notes + + + If you suspect corruption of an index on a user table, you can + simply rebuild that index, or all indexes on the table, using + REINDEX INDEX or REINDEX + TABLE. Another approach to dealing with a corrupted + user-table index is just to drop and recreate it. This may in fact + be preferable if you would like to maintain some semblance of + normal operation on the table meanwhile. REINDEX + acquires exclusive lock on the table, while CREATE + INDEX only locks out writes not reads of the table. + + + + Things are more difficult if you need to recover from corruption of + an index on a system table. In this case it's important for the + system to not have used any of the suspect indexes itself. + (Indeed, in this sort of scenario you may find that server + processes are crashing immediately at start-up, due to reliance on + the corrupted indexes.) To recover safely, the server must be started + with the option, which prevents it from using + indexes for system catalog lookups. + + + + One way to do this is to shut down the postmaster and start a stand-alone + PostgreSQL server + with the option included on its command line. + Then, REINDEX DATABASE, + REINDEX TABLE, or REINDEX INDEX can be + issued, depending on how much you want to reconstruct. If in + doubt, use REINDEX DATABASE to select + reconstruction of all system indexes in the database. Then quit + the standalone server session and restart the regular server. + See the reference page for more + information about how to interact with the stand-alone server + interface. + + + + Alternatively, a regular server session can be started with + included in its command line options. + The method for doing this varies across clients, but in all + libpq-based clients, it is possible to set + the PGOPTIONS environment variable to -P + before starting the client. Note that while this method does not + require locking out other clients, it may still be wise to prevent + other users from connecting to the damaged database until repairs + have been completed. + + + + If corruption is suspected in the indexes of any of the shared + system catalogs (pg_database, + pg_group, or + pg_shadow), then a standalone server + must be used to repair it. REINDEX will not process + shared catalogs in multiuser mode. + + + + For all indexes except the shared system catalogs, REINDEX + is crash-safe and transaction-safe. REINDEX is not + crash-safe for shared indexes, which is why this case is disallowed + during normal operation. If a failure occurs while reindexing one + of these catalogs in standalone mode, it is important that the failure + be rectified before attempting to restart the regular server. + + + + Examples *************** *** 172,182 **** ! Rebuild all system indexes (this will only work in a stand-alone ! server session): ! REINDEX DATABASE my_database FORCE; --- 204,218 ---- ! Rebuild all system indexes in a particular database, without trusting them ! to be valid already: ! $ export PGOPTIONS="-P" ! $ psql broken_db ! ... ! broken_db=> REINDEX DATABASE broken_db; ! broken_db=> \q *** src/backend/access/index/genam.c.orig Sun Aug 3 23:00:25 2003 --- src/backend/access/index/genam.c Mon Sep 22 17:44:29 2003 *************** *** 184,204 **** int nkeys, ScanKey key) { SysScanDesc sysscan; sysscan = (SysScanDesc) palloc(sizeof(SysScanDescData)); sysscan->heap_rel = heapRelation; ! if (indexOK && ! heapRelation->rd_rel->relhasindex && ! !IsIgnoringSystemIndexes()) { - Relation irel; int i; - /* We assume it's a system index, so index_openr is OK */ - sysscan->irel = irel = index_openr(indexRelname); - /* * Change attribute numbers to be index column numbers. * --- 184,215 ---- int nkeys, ScanKey key) { SysScanDesc sysscan; + Relation irel; + + if (indexOK && !IsIgnoringSystemIndexes()) + { + /* We assume it's a system index, so index_openr is OK */ + irel = index_openr(indexRelname); + + if (ReindexIsProcessingIndex(RelationGetRelid(irel))) + { + /* oops, can't use index that's being rebuilt */ + index_close(irel); + irel = NULL; + } + } + else + irel = NULL; sysscan = (SysScanDesc) palloc(sizeof(SysScanDescData)); sysscan->heap_rel = heapRelation; + sysscan->irel = irel; ! if (irel) { int i; /* * Change attribute numbers to be index column numbers. * *************** *** 210,222 **** Assert(key[i].sk_attno == irel->rd_index->indkey[i]); key[i].sk_attno = i + 1; } sysscan->iscan = index_beginscan(heapRelation, irel, snapshot, nkeys, key); sysscan->scan = NULL; } else { - sysscan->irel = NULL; sysscan->scan = heap_beginscan(heapRelation, snapshot, nkeys, key); sysscan->iscan = NULL; } --- 221,233 ---- Assert(key[i].sk_attno == irel->rd_index->indkey[i]); key[i].sk_attno = i + 1; } + sysscan->iscan = index_beginscan(heapRelation, irel, snapshot, nkeys, key); sysscan->scan = NULL; } else { sysscan->scan = heap_beginscan(heapRelation, snapshot, nkeys, key); sysscan->iscan = NULL; } *** src/backend/access/transam/xact.c.orig Fri Aug 8 17:47:02 2003 --- src/backend/access/transam/xact.c Mon Sep 22 17:44:36 2003 *************** *** 834,841 **** */ s->state = TRANS_START; - SetReindexProcessing(false); - /* * generate a new transaction id */ --- 834,839 ---- *************** *** 1085,1090 **** --- 1083,1089 ---- AtEOXact_Namespace(false); AtEOXact_CatCache(false); AtEOXact_Files(); + SetReindexProcessing(InvalidOid, InvalidOid); pgstat_count_xact_rollback(); /* *** src/backend/catalog/index.c.orig Fri Sep 19 15:57:42 2003 --- src/backend/catalog/index.c Mon Sep 22 19:03:52 2003 *************** *** 78,101 **** static Oid IndexGetRelation(Oid indexId); - static bool reindexing = false; - - - bool - SetReindexProcessing(bool reindexmode) - { - bool old = reindexing; - - reindexing = reindexmode; - return old; - } - - bool - IsReindexProcessing(void) - { - return reindexing; - } - /* * ConstructTupleDescriptor * --- 78,83 ---- *************** *** 497,504 **** Oid indexoid; int i; - SetReindexProcessing(false); - /* * Only SELECT ... FOR UPDATE are allowed while doing this */ --- 479,484 ---- *************** *** 972,1017 **** } - /* --------------------------------------------- - * Indexes of the relation active ? - * - * Caller must hold an adequate lock on the relation to ensure the - * answer won't be changing. - * --------------------------------------------- - */ - bool - IndexesAreActive(Relation heaprel) - { - bool isactive; - Relation indexRelation; - HeapScanDesc scan; - ScanKeyData entry; - - if (heaprel->rd_rel->relkind != RELKIND_RELATION && - heaprel->rd_rel->relkind != RELKIND_TOASTVALUE) - ereport(ERROR, - (errcode(ERRCODE_WRONG_OBJECT_TYPE), - errmsg("relation \"%s\" isn't an indexable relation", - RelationGetRelationName(heaprel)))); - - /* If pg_class.relhasindex is set, indexes are active */ - isactive = heaprel->rd_rel->relhasindex; - if (isactive) - return isactive; - - /* Otherwise, look to see if there are any indexes */ - indexRelation = heap_openr(IndexRelationName, AccessShareLock); - ScanKeyEntryInitialize(&entry, 0, - Anum_pg_index_indrelid, F_OIDEQ, - ObjectIdGetDatum(RelationGetRelid(heaprel))); - scan = heap_beginscan(indexRelation, SnapshotNow, 1, &entry); - if (heap_getnext(scan, ForwardScanDirection) == NULL) - isactive = true; /* no indexes, so report "active" */ - heap_endscan(scan); - heap_close(indexRelation, AccessShareLock); - return isactive; - } - /* ---------------- * set relhasindex of relation's pg_class entry * --- 952,957 ---- *************** *** 1037,1048 **** HeapScanDesc pg_class_scan = NULL; /* ! * Find the tuple to update in pg_class. */ pg_class = heap_openr(RelationRelationName, RowExclusiveLock); ! if (!IsIgnoringSystemIndexes() && ! (!IsReindexProcessing() || pg_class->rd_rel->relhasindex)) { tuple = SearchSysCacheCopy(RELOID, ObjectIdGetDatum(relid), --- 977,989 ---- HeapScanDesc pg_class_scan = NULL; /* ! * Find the tuple to update in pg_class. In bootstrap mode we can't ! * use heap_update, so cheat and overwrite the tuple in-place. In ! * normal processing, make a copy to scribble on. */ pg_class = heap_openr(RelationRelationName, RowExclusiveLock); ! if (!IsBootstrapProcessingMode()) { tuple = SearchSysCacheCopy(RELOID, ObjectIdGetDatum(relid), *************** *** 1063,1077 **** if (!HeapTupleIsValid(tuple)) elog(ERROR, "could not find tuple for relation %u", relid); - /* - * Update fields in the pg_class tuple. - */ if (pg_class_scan) LockBuffer(pg_class_scan->rs_cbuf, BUFFER_LOCK_EXCLUSIVE); - classtuple = (Form_pg_class) GETSTRUCT(tuple); - if (classtuple->relhasindex != hasindex) { classtuple->relhasindex = hasindex; --- 1004,1016 ---- if (!HeapTupleIsValid(tuple)) elog(ERROR, "could not find tuple for relation %u", relid); + classtuple = (Form_pg_class) GETSTRUCT(tuple); + + /* Apply required updates */ if (pg_class_scan) LockBuffer(pg_class_scan->rs_cbuf, BUFFER_LOCK_EXCLUSIVE); if (classtuple->relhasindex != hasindex) { classtuple->relhasindex = hasindex; *************** *** 1140,1219 **** Relation pg_class; HeapTuple tuple; Form_pg_class rd_rel; - HeapScanDesc pg_class_scan = NULL; - bool in_place_upd; RelationData workrel; ! Assert(!IsSystemRelation(relation) || IsToastRelation(relation) || relation->rd_rel->relkind == RELKIND_INDEX); /* Allocate a new relfilenode */ newrelfilenode = newoid(); /* ! * Find the RELATION relation tuple for the given relation. */ pg_class = heap_openr(RelationRelationName, RowExclusiveLock); ! in_place_upd = IsIgnoringSystemIndexes(); ! ! if (!in_place_upd) ! { ! tuple = SearchSysCacheCopy(RELOID, ! ObjectIdGetDatum(RelationGetRelid(relation)), ! 0, 0, 0); ! } ! else ! { ! ScanKeyData key[1]; ! ! ScanKeyEntryInitialize(&key[0], 0, ! ObjectIdAttributeNumber, ! F_OIDEQ, ! ObjectIdGetDatum(RelationGetRelid(relation))); ! ! pg_class_scan = heap_beginscan(pg_class, SnapshotNow, 1, key); ! tuple = heap_getnext(pg_class_scan, ForwardScanDirection); ! } ! if (!HeapTupleIsValid(tuple)) elog(ERROR, "could not find tuple for relation %u", RelationGetRelid(relation)); rd_rel = (Form_pg_class) GETSTRUCT(tuple); - /* schedule unlinking old relfilenode */ - smgrunlink(DEFAULT_SMGR, relation); - /* create another storage file. Is it a little ugly ? */ memcpy((char *) &workrel, relation, sizeof(RelationData)); workrel.rd_fd = -1; workrel.rd_node.relNode = newrelfilenode; heap_storage_create(&workrel); smgrclose(DEFAULT_SMGR, &workrel); /* update the pg_class row */ ! if (in_place_upd) ! { ! LockBuffer(pg_class_scan->rs_cbuf, BUFFER_LOCK_EXCLUSIVE); ! rd_rel->relfilenode = newrelfilenode; ! LockBuffer(pg_class_scan->rs_cbuf, BUFFER_LOCK_UNLOCK); ! WriteNoReleaseBuffer(pg_class_scan->rs_cbuf); ! BufferSync(); ! /* Send out shared cache inval if necessary */ ! if (!IsBootstrapProcessingMode()) ! CacheInvalidateHeapTuple(pg_class, tuple); ! } ! else ! { ! rd_rel->relfilenode = newrelfilenode; ! simple_heap_update(pg_class, &tuple->t_self, tuple); ! CatalogUpdateIndexes(pg_class, tuple); ! } ! if (!pg_class_scan) ! heap_freetuple(tuple); ! else ! heap_endscan(pg_class_scan); heap_close(pg_class, RowExclusiveLock); --- 1079,1126 ---- Relation pg_class; HeapTuple tuple; Form_pg_class rd_rel; RelationData workrel; ! /* Can't change relfilenode for nailed tables (indexes ok though) */ ! Assert(!relation->rd_isnailed || relation->rd_rel->relkind == RELKIND_INDEX); + /* Can't change for shared tables or indexes */ + Assert(!relation->rd_rel->relisshared); /* Allocate a new relfilenode */ newrelfilenode = newoid(); /* ! * Find the pg_class tuple for the given relation. This is not used ! * during bootstrap, so okay to use heap_update always. */ pg_class = heap_openr(RelationRelationName, RowExclusiveLock); ! tuple = SearchSysCacheCopy(RELOID, ! ObjectIdGetDatum(RelationGetRelid(relation)), ! 0, 0, 0); if (!HeapTupleIsValid(tuple)) elog(ERROR, "could not find tuple for relation %u", RelationGetRelid(relation)); rd_rel = (Form_pg_class) GETSTRUCT(tuple); /* create another storage file. Is it a little ugly ? */ + /* NOTE: any conflict in relfilenode value will be caught here */ memcpy((char *) &workrel, relation, sizeof(RelationData)); workrel.rd_fd = -1; workrel.rd_node.relNode = newrelfilenode; heap_storage_create(&workrel); smgrclose(DEFAULT_SMGR, &workrel); + /* schedule unlinking old relfilenode */ + smgrunlink(DEFAULT_SMGR, relation); + /* update the pg_class row */ ! rd_rel->relfilenode = newrelfilenode; ! simple_heap_update(pg_class, &tuple->t_self, tuple); ! CatalogUpdateIndexes(pg_class, tuple); ! heap_freetuple(tuple); heap_close(pg_class, RowExclusiveLock); *************** *** 1263,1273 **** whichRel = relation_open(relid, ShareLock); /* ! * Find the RELATION relation tuple for the given relation. */ pg_class = heap_openr(RelationRelationName, RowExclusiveLock); ! in_place_upd = (IsIgnoringSystemIndexes() || IsReindexProcessing()); if (!in_place_upd) { --- 1170,1190 ---- whichRel = relation_open(relid, ShareLock); /* ! * Find the tuple to update in pg_class. Normally we make a copy of ! * the tuple using the syscache, modify it, and apply heap_update. ! * But in bootstrap mode we can't use heap_update, so we cheat and ! * overwrite the tuple in-place. ! * ! * We also must cheat if reindexing pg_class itself, because the ! * target index may presently not be part of the set of indexes that ! * CatalogUpdateIndexes would update (see reindex_relation). In this ! * case the stats updates will not be WAL-logged and so could be lost ! * in a crash. This seems OK considering VACUUM does the same thing. */ pg_class = heap_openr(RelationRelationName, RowExclusiveLock); ! in_place_upd = IsBootstrapProcessingMode() || ! ReindexIsProcessingHeap(RelationGetRelid(pg_class)); if (!in_place_upd) { *************** *** 1290,1295 **** --- 1207,1213 ---- if (!HeapTupleIsValid(tuple)) elog(ERROR, "could not find tuple for relation %u", relid); + rd_rel = (Form_pg_class) GETSTRUCT(tuple); /* * Figure values to insert. *************** *** 1330,1347 **** * also reduces the window wherein concurrent CREATE INDEX commands * may conflict.) */ - rd_rel = (Form_pg_class) GETSTRUCT(tuple); - if (rd_rel->relpages != (int32) relpages || rd_rel->reltuples != (float4) reltuples) { if (in_place_upd) { ! /* ! * At bootstrap time, we don't need to worry about concurrency ! * or visibility of changes, so we cheat. Also cheat if ! * REINDEX. ! */ LockBuffer(pg_class_scan->rs_cbuf, BUFFER_LOCK_EXCLUSIVE); rd_rel->relpages = (int32) relpages; rd_rel->reltuples = (float4) reltuples; --- 1248,1259 ---- * also reduces the window wherein concurrent CREATE INDEX commands * may conflict.) */ if (rd_rel->relpages != (int32) relpages || rd_rel->reltuples != (float4) reltuples) { if (in_place_upd) { ! /* Bootstrap or reindex case: overwrite fields in place. */ LockBuffer(pg_class_scan->rs_cbuf, BUFFER_LOCK_EXCLUSIVE); rd_rel->relpages = (int32) relpages; rd_rel->reltuples = (float4) reltuples; *************** *** 1561,1570 **** * should not see any tuples inserted by open * transactions --- unless it's our own transaction. * (Consider INSERT followed by CREATE INDEX within a ! * transaction.) */ if (!TransactionIdIsCurrentTransactionId( ! HeapTupleHeaderGetXmin(heapTuple->t_data))) elog(ERROR, "concurrent insert in progress"); indexIt = true; tupleIsAlive = true; --- 1473,1485 ---- * should not see any tuples inserted by open * transactions --- unless it's our own transaction. * (Consider INSERT followed by CREATE INDEX within a ! * transaction.) An exception occurs when reindexing ! * a system catalog, because we often release lock on ! * system catalogs before committing. */ if (!TransactionIdIsCurrentTransactionId( ! HeapTupleHeaderGetXmin(heapTuple->t_data)) ! && !IsSystemRelation(heapRelation)) elog(ERROR, "concurrent insert in progress"); indexIt = true; tupleIsAlive = true; *************** *** 1576,1585 **** * should not see any tuples deleted by open * transactions --- unless it's our own transaction. * (Consider DELETE followed by CREATE INDEX within a ! * transaction.) */ if (!TransactionIdIsCurrentTransactionId( ! HeapTupleHeaderGetXmax(heapTuple->t_data))) elog(ERROR, "concurrent delete in progress"); indexIt = true; tupleIsAlive = false; --- 1491,1503 ---- * should not see any tuples deleted by open * transactions --- unless it's our own transaction. * (Consider DELETE followed by CREATE INDEX within a ! * transaction.) An exception occurs when reindexing ! * a system catalog, because we often release lock on ! * system catalogs before committing. */ if (!TransactionIdIsCurrentTransactionId( ! HeapTupleHeaderGetXmax(heapTuple->t_data)) ! && !IsSystemRelation(heapRelation)) elog(ERROR, "concurrent delete in progress"); indexIt = true; tupleIsAlive = false; *************** *** 1692,1754 **** /* * reindex_index - This routine is used to recreate an index */ ! bool ! reindex_index(Oid indexId, bool force, bool inplace) { Relation iRel, heapRelation; IndexInfo *indexInfo; Oid heapId; ! bool old; /* * Open our index relation and get an exclusive lock on it. * ! * Note: doing this before opening the parent heap relation means there's ! * a possibility for deadlock failure against another xact that is ! * doing normal accesses to the heap and index. However, it's not ! * real clear why you'd be needing to do REINDEX on a table that's in ! * active use, so I'd rather have the protection of making sure the ! * index is locked down. */ iRel = index_open(indexId); LockRelation(iRel, AccessExclusiveLock); - old = SetReindexProcessing(true); - /* Get OID of index's parent table */ heapId = iRel->rd_index->indrelid; ! /* Open the parent heap relation */ heapRelation = heap_open(heapId, AccessExclusiveLock); /* * If it's a shared index, we must do inplace processing (because we ! * have no way to update relfilenode in other databases). Also, if ! * it's a nailed-in-cache index, we must do inplace processing because ! * the relcache can't cope with changing its relfilenode. * ! * In either of these cases, we are definitely processing a system index, ! * so we'd better be ignoring system indexes. (These checks are just ! * for paranoia's sake --- upstream code should have disallowed reindex ! * in such cases already.) ! */ ! if (iRel->rd_rel->relisshared) ! { ! if (!IsIgnoringSystemIndexes()) ! elog(ERROR, ! "must be ignoring system indexes to reindex shared index %u", ! indexId); ! inplace = true; ! } ! if (iRel->rd_isnailed) ! { ! if (!IsIgnoringSystemIndexes()) ! elog(ERROR, ! "must be ignoring system indexes to reindex nailed index %u", ! indexId); ! inplace = true; ! } /* Fetch info needed for index_build */ indexInfo = BuildIndexInfo(iRel); --- 1610,1663 ---- /* * reindex_index - This routine is used to recreate an index */ ! void ! reindex_index(Oid indexId) { Relation iRel, heapRelation; IndexInfo *indexInfo; Oid heapId; ! bool inplace; /* * Open our index relation and get an exclusive lock on it. * ! * Note: for REINDEX INDEX, doing this before opening the parent heap ! * relation means there's a possibility for deadlock failure against ! * another xact that is doing normal accesses to the heap and index. ! * However, it's not real clear why you'd be wanting to do REINDEX INDEX ! * on a table that's in active use, so I'd rather have the protection of ! * making sure the index is locked down. In the REINDEX TABLE and ! * REINDEX DATABASE cases, there is no problem because caller already ! * holds exclusive lock on the parent table. */ iRel = index_open(indexId); LockRelation(iRel, AccessExclusiveLock); /* Get OID of index's parent table */ heapId = iRel->rd_index->indrelid; ! /* Open and lock the parent heap relation */ heapRelation = heap_open(heapId, AccessExclusiveLock); + SetReindexProcessing(heapId, indexId); + /* * If it's a shared index, we must do inplace processing (because we ! * have no way to update relfilenode in other databases). Otherwise ! * we can do it the normal transaction-safe way. * ! * Since inplace processing isn't crash-safe, we only allow it in a ! * standalone backend. (In the REINDEX TABLE and REINDEX DATABASE cases, ! * the caller should have detected this.) ! */ ! inplace = iRel->rd_rel->relisshared; ! ! if (inplace && IsUnderPostmaster) ! ereport(ERROR, ! (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE), ! errmsg("shared index \"%s\" can only be reindexed in standalone mode", ! RelationGetRelationName(iRel)))); /* Fetch info needed for index_build */ indexInfo = BuildIndexInfo(iRel); *************** *** 1781,1902 **** * index_build will close both the heap and index relations (but not * give up the locks we hold on them). So we're done. */ ! ! SetReindexProcessing(old); ! ! return true; ! } ! ! #ifdef NOT_USED ! /* ! * activate_indexes_of_a_table ! * activate/deactivate indexes of the specified table. ! * ! * Caller must already hold exclusive lock on the table. ! */ ! bool ! activate_indexes_of_a_table(Relation heaprel, bool activate) ! { ! if (IndexesAreActive(heaprel)) ! { ! if (!activate) ! setRelhasindex(RelationGetRelid(heaprel), false, false, ! InvalidOid); ! else ! return false; ! } ! else ! { ! if (activate) ! reindex_relation(RelationGetRelid(heaprel), false); ! else ! return false; ! } ! return true; } - #endif /* NOT_USED */ /* * reindex_relation - This routine is used to recreate all indexes * of a relation. */ bool ! reindex_relation(Oid relid, bool force) { - Relation indexRelation; - ScanKeyData entry; - HeapScanDesc scan; - HeapTuple indexTuple; - bool old, - reindexed; - bool overwrite; Relation rel; ! ! overwrite = false; /* * Ensure to hold an exclusive lock throughout the transaction. The ! * lock could be less intensive (in the non-overwrite path) but for ! * now it's AccessExclusiveLock for simplicity. */ rel = heap_open(relid, AccessExclusiveLock); /* ! * Should be ignoring system indexes if we are reindexing a system table. ! * (This is elog not ereport because caller should have caught it.) ! */ ! if (!IsIgnoringSystemIndexes() && ! IsSystemRelation(rel) && !IsToastRelation(rel)) ! elog(ERROR, ! "must be ignoring system indexes to reindex system table %u", ! relid); /* ! * Shared system indexes must be overwritten because it's impossible ! * to update pg_class tuples of all databases. */ ! if (rel->rd_rel->relisshared) { ! if (!IsIgnoringSystemIndexes()) /* shouldn't happen */ ! elog(ERROR, ! "must be ignoring system indexes to reindex shared table %u", ! relid); ! overwrite = true; ! } ! old = SetReindexProcessing(true); ! /* ! * Continue to hold the lock. ! */ ! heap_close(rel, NoLock); ! /* ! * Find table's indexes by looking in pg_index (not trusting indexes...) ! */ ! indexRelation = heap_openr(IndexRelationName, AccessShareLock); ! ScanKeyEntryInitialize(&entry, 0, ! Anum_pg_index_indrelid, ! F_OIDEQ, ! ObjectIdGetDatum(relid)); ! scan = heap_beginscan(indexRelation, SnapshotNow, 1, &entry); ! reindexed = false; ! while ((indexTuple = heap_getnext(scan, ForwardScanDirection)) != NULL) ! { ! Form_pg_index index = (Form_pg_index) GETSTRUCT(indexTuple); ! if (reindex_index(index->indexrelid, false, overwrite)) ! reindexed = true; ! else ! { ! reindexed = false; ! break; ! } } - heap_endscan(scan); - heap_close(indexRelation, AccessShareLock); ! SetReindexProcessing(old); ! return reindexed; } --- 1690,1770 ---- * index_build will close both the heap and index relations (but not * give up the locks we hold on them). So we're done. */ ! SetReindexProcessing(InvalidOid, InvalidOid); } /* * reindex_relation - This routine is used to recreate all indexes * of a relation. + * + * Returns true if any indexes were rebuilt. */ bool ! reindex_relation(Oid relid) { Relation rel; ! bool is_pg_class; ! List *indexIds, ! *doneIndexes, ! *indexId; /* * Ensure to hold an exclusive lock throughout the transaction. The ! * lock could perhaps be less intensive (in the non-overwrite case) ! * but for now it's AccessExclusiveLock for simplicity. */ rel = heap_open(relid, AccessExclusiveLock); /* ! * Get the list of index OIDs for this relation. (We trust to the ! * relcache to get this with a sequential scan if ignoring system ! * indexes.) ! */ ! indexIds = RelationGetIndexList(rel); /* ! * reindex_index will attempt to update the pg_class rows for the ! * relation and index. If we are processing pg_class itself, we ! * want to make sure that the updates do not try to insert index ! * entries into indexes we have not processed yet. (When we are ! * trying to recover from corrupted indexes, that could easily ! * cause a crash.) We can accomplish this because CatalogUpdateIndexes ! * will use the relcache's index list to know which indexes to update. ! * We just force the index list to be only the stuff we've processed. ! * ! * It is okay to not insert entries into the indexes we have not ! * processed yet because all of this is transaction-safe. If we fail ! * partway through, the updated rows are dead and it doesn't matter ! * whether they have index entries. Also, a new pg_class index will ! * be created with an entry for its own pg_class row because we do ! * setNewRelfilenode() before we do index_build(). */ ! is_pg_class = (RelationGetRelid(rel) == RelOid_pg_class); ! doneIndexes = NIL; ! ! /* Reindex all the indexes. */ ! foreach(indexId, indexIds) { ! Oid indexOid = lfirsto(indexId); ! if (is_pg_class) ! RelationSetIndexList(rel, doneIndexes); ! reindex_index(indexOid); ! CommandCounterIncrement(); ! if (is_pg_class) ! doneIndexes = lappendo(doneIndexes, indexOid); } ! if (is_pg_class) ! RelationSetIndexList(rel, indexIds); ! ! /* ! * Close rel, but continue to hold the lock. ! */ ! heap_close(rel, NoLock); ! return (indexIds != NIL); } *** src/backend/commands/indexcmds.c.orig Fri Sep 19 16:07:33 2003 --- src/backend/commands/indexcmds.c Mon Sep 22 18:22:19 2003 *************** *** 112,125 **** relationId = RelationGetRelid(rel); namespaceId = RelationGetNamespace(rel); - if (!IsBootstrapProcessingMode() && - IsSystemRelation(rel) && - !IndexesAreActive(rel)) - ereport(ERROR, - (errcode(ERRCODE_INDEXES_DEACTIVATED), - errmsg("existing indexes are inactive"), - errhint("REINDEX the table first."))); - heap_close(rel, NoLock); /* --- 112,117 ---- *************** *** 599,608 **** { Oid indOid; HeapTuple tuple; - bool overwrite; - - /* Choose in-place-or-not mode */ - overwrite = IsIgnoringSystemIndexes(); indOid = RangeVarGetRelid(indexRelation, false); tuple = SearchSysCache(RELOID, --- 591,596 ---- *************** *** 617,653 **** errmsg("relation \"%s\" is not an index", indexRelation->relname))); ! if (IsSystemClass((Form_pg_class) GETSTRUCT(tuple)) && ! !IsToastClass((Form_pg_class) GETSTRUCT(tuple))) ! { ! if (!allowSystemTableMods) ! ereport(ERROR, ! (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE), ! errmsg("permission denied: \"%s\" is a system index", ! indexRelation->relname), ! errhint("Do REINDEX in standalone postgres with -O -P options."))); ! if (!IsIgnoringSystemIndexes()) ! ereport(ERROR, ! (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE), ! errmsg("permission denied: \"%s\" is a system index", ! indexRelation->relname), ! errhint("Do REINDEX in standalone postgres with -P -O options."))); ! } ReleaseSysCache(tuple); ! /* ! * In-place REINDEX within a transaction block is dangerous, because ! * if the transaction is later rolled back we have no way to undo ! * truncation of the index's physical file. Disallow it. ! */ ! if (overwrite) ! PreventTransactionChain((void *) indexRelation, "REINDEX"); ! ! if (!reindex_index(indOid, force, overwrite)) ! ereport(WARNING, ! (errmsg("index \"%s\" wasn't reindexed", ! indexRelation->relname))); } /* --- 605,618 ---- errmsg("relation \"%s\" is not an index", indexRelation->relname))); ! /* Check permissions */ ! if (!pg_class_ownercheck(indOid, GetUserId())) ! aclcheck_error(ACLCHECK_NOT_OWNER, ACL_KIND_CLASS, ! indexRelation->relname); ReleaseSysCache(tuple); ! reindex_index(indOid); } /* *************** *** 655,661 **** * Recreate indexes of a table. */ void ! ReindexTable(RangeVar *relation, bool force) { Oid heapOid; HeapTuple tuple; --- 620,626 ---- * Recreate indexes of a table. */ void ! ReindexTable(RangeVar *relation, bool force /* currently unused */ ) { Oid heapOid; HeapTuple tuple; *************** *** 674,712 **** errmsg("relation \"%s\" is not a table", relation->relname))); ! if (IsSystemClass((Form_pg_class) GETSTRUCT(tuple)) && ! !IsToastClass((Form_pg_class) GETSTRUCT(tuple))) ! { ! if (!allowSystemTableMods) ! ereport(ERROR, ! (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE), ! errmsg("permission denied: \"%s\" is a system table", ! relation->relname), ! errhint("Do REINDEX in standalone postgres with -O -P options."))); ! if (!IsIgnoringSystemIndexes()) ! ereport(ERROR, ! (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE), ! errmsg("permission denied: \"%s\" is a system table", ! relation->relname), ! errhint("Do REINDEX in standalone postgres with -P -O options."))); ! } ! ReleaseSysCache(tuple); ! /* ! * In-place REINDEX within a transaction block is dangerous, because ! * if the transaction is later rolled back we have no way to undo ! * truncation of the index's physical file. Disallow it. ! * ! * XXX we assume that in-place reindex will only be done if ! * IsIgnoringSystemIndexes() is true. ! */ ! if (IsIgnoringSystemIndexes()) ! PreventTransactionChain((void *) relation, "REINDEX"); ! if (!reindex_relation(heapOid, force)) ereport(WARNING, ! (errmsg("table \"%s\" wasn't reindexed", relation->relname))); } --- 639,661 ---- errmsg("relation \"%s\" is not a table", relation->relname))); ! /* Check permissions */ ! if (!pg_class_ownercheck(heapOid, GetUserId())) ! aclcheck_error(ACLCHECK_NOT_OWNER, ACL_KIND_CLASS, ! relation->relname); ! /* Can't reindex shared tables except in standalone mode */ ! if (((Form_pg_class) GETSTRUCT(tuple))->relisshared && IsUnderPostmaster) ! ereport(ERROR, ! (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE), ! errmsg("shared table \"%s\" can only be reindexed in standalone mode", ! relation->relname))); ! ReleaseSysCache(tuple); ! if (!reindex_relation(heapOid)) ereport(WARNING, ! (errmsg("table \"%s\" has no indexes", relation->relname))); } *************** *** 715,732 **** * Recreate indexes of a database. */ void ! ReindexDatabase(const char *dbname, bool force, bool all) { Relation relationRelation; HeapScanDesc scan; HeapTuple tuple; MemoryContext private_context; MemoryContext old; ! int relcnt, ! relalc, ! i, ! oncealc = 200; ! Oid *relids = (Oid *) NULL; AssertArg(dbname); --- 664,678 ---- * Recreate indexes of a database. */ void ! ReindexDatabase(const char *dbname, bool force /* currently unused */, ! bool all) { Relation relationRelation; HeapScanDesc scan; HeapTuple tuple; MemoryContext private_context; MemoryContext old; ! List *relids = NIL; AssertArg(dbname); *************** *** 739,759 **** aclcheck_error(ACLCHECK_NOT_OWNER, ACL_KIND_DATABASE, dbname); - if (!allowSystemTableMods) - ereport(ERROR, - (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE), - errmsg("REINDEX DATABASE must be done in standalone postgres with -O -P options"))); - if (!IsIgnoringSystemIndexes()) - ereport(ERROR, - (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE), - errmsg("REINDEX DATABASE must be done in standalone postgres with -P -O options"))); - /* * We cannot run inside a user transaction block; if we were inside a * transaction, then our commit- and start-transaction-command calls * would not have the intended effect! */ ! PreventTransactionChain((void *) dbname, "REINDEX"); /* * Create a memory context that will survive forced transaction --- 685,696 ---- aclcheck_error(ACLCHECK_NOT_OWNER, ACL_KIND_DATABASE, dbname); /* * We cannot run inside a user transaction block; if we were inside a * transaction, then our commit- and start-transaction-command calls * would not have the intended effect! */ ! PreventTransactionChain((void *) dbname, "REINDEX DATABASE"); /* * Create a memory context that will survive forced transaction *************** *** 768,821 **** ALLOCSET_DEFAULT_MAXSIZE); /* * Scan pg_class to build a list of the relations we need to reindex. */ relationRelation = heap_openr(RelationRelationName, AccessShareLock); scan = heap_beginscan(relationRelation, SnapshotNow, 0, NULL); - relcnt = relalc = 0; while ((tuple = heap_getnext(scan, ForwardScanDirection)) != NULL) { ! char relkind; ! if (!all) { ! if (!(IsSystemClass((Form_pg_class) GETSTRUCT(tuple)) && ! !IsToastClass((Form_pg_class) GETSTRUCT(tuple)))) continue; } ! relkind = ((Form_pg_class) GETSTRUCT(tuple))->relkind; ! if (relkind == RELKIND_RELATION || relkind == RELKIND_TOASTVALUE) { ! old = MemoryContextSwitchTo(private_context); ! if (relcnt == 0) ! { ! relalc = oncealc; ! relids = palloc(sizeof(Oid) * relalc); ! } ! else if (relcnt >= relalc) ! { ! relalc *= 2; ! relids = repalloc(relids, sizeof(Oid) * relalc); ! } ! MemoryContextSwitchTo(old); ! relids[relcnt] = HeapTupleGetOid(tuple); ! relcnt++; } } heap_endscan(scan); heap_close(relationRelation, AccessShareLock); /* Now reindex each rel in a separate transaction */ CommitTransactionCommand(); ! for (i = 0; i < relcnt; i++) { StartTransactionCommand(); SetQuerySnapshot(); /* might be needed for functions in * indexes */ ! if (reindex_relation(relids[i], force)) ereport(NOTICE, ! (errmsg("relation %u was reindexed", relids[i]))); CommitTransactionCommand(); } StartTransactionCommand(); --- 705,769 ---- ALLOCSET_DEFAULT_MAXSIZE); /* + * We always want to reindex pg_class first. This ensures that if + * there is any corruption in pg_class' indexes, they will be fixed + * before we process any other tables. This is critical because + * reindexing itself will try to update pg_class. + */ + old = MemoryContextSwitchTo(private_context); + relids = lappendo(relids, RelOid_pg_class); + MemoryContextSwitchTo(old); + + /* * Scan pg_class to build a list of the relations we need to reindex. */ relationRelation = heap_openr(RelationRelationName, AccessShareLock); scan = heap_beginscan(relationRelation, SnapshotNow, 0, NULL); while ((tuple = heap_getnext(scan, ForwardScanDirection)) != NULL) { ! Form_pg_class classtuple = (Form_pg_class) GETSTRUCT(tuple); ! if (classtuple->relkind != RELKIND_RELATION && ! classtuple->relkind != RELKIND_TOASTVALUE) ! continue; ! ! if (!all) /* only system tables? */ { ! if (!(IsSystemClass(classtuple) && !IsToastClass(classtuple))) continue; } ! ! if (IsUnderPostmaster) /* silently ignore shared tables */ { ! if (classtuple->relisshared) ! continue; } + + if (HeapTupleGetOid(tuple) == RelOid_pg_class) + continue; /* got it already */ + + old = MemoryContextSwitchTo(private_context); + relids = lappendo(relids, HeapTupleGetOid(tuple)); + MemoryContextSwitchTo(old); } heap_endscan(scan); heap_close(relationRelation, AccessShareLock); /* Now reindex each rel in a separate transaction */ CommitTransactionCommand(); ! while (relids) { + Oid relid = lfirsto(relids); + StartTransactionCommand(); SetQuerySnapshot(); /* might be needed for functions in * indexes */ ! if (reindex_relation(relid)) ereport(NOTICE, ! (errmsg("table \"%s\" was reindexed", ! get_rel_name(relid)))); CommitTransactionCommand(); + relids = lnext(relids); } StartTransactionCommand(); *** src/backend/commands/vacuum.c.orig Sun Aug 3 23:00:32 2003 --- src/backend/commands/vacuum.c Sun Sep 21 16:55:42 2003 *************** *** 904,914 **** int nindexes, i; VRelStats *vacrelstats; - bool reindex = false; - - if (IsIgnoringSystemIndexes() && - IsSystemRelation(onerel)) - reindex = true; vacuum_set_xid_limits(vacstmt, onerel->rd_rel->relisshared, &OldestXmin, &FreezeLimit); --- 904,909 ---- *************** *** 927,953 **** /* Now open all indexes of the relation */ vac_open_indexes(onerel, &nindexes, &Irel); - if (!Irel) - reindex = false; - else if (!RelationGetForm(onerel)->relhasindex) - reindex = true; if (nindexes > 0) vacrelstats->hasindex = true; - #ifdef NOT_USED - - /* - * reindex in VACUUM is dangerous under WAL. ifdef out until it - * becomes safe. - */ - if (reindex) - { - vac_close_indexes(nindexes, Irel); - Irel = (Relation *) NULL; - activate_indexes_of_a_table(onerel, false); - } - #endif /* NOT_USED */ - /* Clean/scan index relation(s) */ if (Irel != (Relation *) NULL) { --- 922,930 ---- *************** *** 993,1003 **** elog(ERROR, "FlushRelationBuffers returned %d", i); } } - - #ifdef NOT_USED - if (reindex) - activate_indexes_of_a_table(onerel, true); - #endif /* NOT_USED */ /* update shared free space map with final free space info */ vac_update_fsm(onerel, &fraged_pages, vacrelstats->rel_pages); --- 970,975 ---- *** src/backend/executor/execUtils.c.orig Fri Aug 8 17:47:05 2003 --- src/backend/executor/execUtils.c Sun Sep 21 16:55:36 2003 *************** *** 647,657 **** resultRelInfo->ri_NumIndices = 0; ! /* checks for disabled indexes */ if (!RelationGetForm(resultRelation)->relhasindex) - return; - if (IsIgnoringSystemIndexes() && - IsSystemRelation(resultRelation)) return; /* --- 647,654 ---- resultRelInfo->ri_NumIndices = 0; ! /* fast path if no indexes */ if (!RelationGetForm(resultRelation)->relhasindex) return; /* *** src/backend/executor/nodeIndexscan.c.orig Fri Aug 22 16:26:43 2003 --- src/backend/executor/nodeIndexscan.c Mon Sep 22 18:39:30 2003 *************** *** 964,975 **** currentRelation = heap_open(reloid, AccessShareLock); - if (!RelationGetForm(currentRelation)->relhasindex) - ereport(ERROR, - (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), - errmsg("indexes of relation %u were deactivated", - reloid))); - indexstate->ss.ss_currentRelation = currentRelation; indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */ --- 964,969 ---- *** src/backend/storage/ipc/sinval.c.orig Tue Aug 19 11:39:55 2003 --- src/backend/storage/ipc/sinval.c Sun Sep 21 18:54:08 2003 *************** *** 73,78 **** --- 73,84 ---- /* * ReceiveSharedInvalidMessages * Process shared-cache-invalidation messages waiting for this backend + * + * NOTE: it is entirely possible for this routine to be invoked recursively + * as a consequence of processing inside the invalFunction or resetFunction. + * Hence, we must be holding no SI resources when we call them. The only + * bad side-effect is that SIDelExpiredDataEntries might be called extra + * times on the way out of a nested call. */ void ReceiveSharedInvalidMessages( *** src/backend/tcop/postgres.c.orig Sat Sep 13 20:03:32 2003 --- src/backend/tcop/postgres.c Mon Sep 22 19:30:50 2003 *************** *** 2252,2260 **** /* * ignore system indexes */ ! if (secure) /* XXX safe to allow from client??? */ ! IgnoreSystemIndexes(true); break; case 'o': --- 2252,2263 ---- /* * ignore system indexes + * + * As of PG 7.4 this is safe to allow from the client, + * since it only disables reading the system indexes, + * not writing them. Worst case consequence is slowness. */ ! IgnoreSystemIndexes(true); break; case 'o': *** src/backend/tcop/utility.c.orig Wed Sep 10 11:16:08 2003 --- src/backend/tcop/utility.c Mon Sep 22 18:16:18 2003 *************** *** 992,1002 **** switch (stmt->kind) { case OBJECT_INDEX: - CheckRelationOwnership(stmt->relation, false); ReindexIndex(stmt->relation, stmt->force); break; case OBJECT_TABLE: - CheckRelationOwnership(stmt->relation, false); ReindexTable(stmt->relation, stmt->force); break; case OBJECT_DATABASE: --- 992,1000 ---- *** src/backend/utils/cache/relcache.c.orig Sun Aug 3 23:01:06 2003 --- src/backend/utils/cache/relcache.c Mon Sep 22 14:32:08 2003 *************** *** 279,287 **** static void RelationClearRelation(Relation relation, bool rebuild); - #ifdef ENABLE_REINDEX_NAILED_RELATIONS static void RelationReloadClassinfo(Relation relation); - #endif /* ENABLE_REINDEX_NAILED_RELATIONS */ static void RelationFlushRelation(Relation relation); static Relation RelationSysNameCacheGetRelation(const char *relationName); static bool load_relcache_init_file(void); --- 279,285 ---- *************** *** 290,296 **** static void formrdesc(const char *relationName, int natts, FormData_pg_attribute *att); ! static HeapTuple ScanPgRelation(RelationBuildDescInfo buildinfo); static Relation AllocateRelationDesc(Relation relation, Form_pg_class relp); static void RelationBuildTupleDesc(RelationBuildDescInfo buildinfo, Relation relation); --- 288,294 ---- static void formrdesc(const char *relationName, int natts, FormData_pg_attribute *att); ! static HeapTuple ScanPgRelation(RelationBuildDescInfo buildinfo, bool indexOK); static Relation AllocateRelationDesc(Relation relation, Form_pg_class relp); static void RelationBuildTupleDesc(RelationBuildDescInfo buildinfo, Relation relation); *************** *** 322,328 **** * and must eventually be freed with heap_freetuple. */ static HeapTuple ! ScanPgRelation(RelationBuildDescInfo buildinfo) { HeapTuple pg_class_tuple; Relation pg_class_desc; --- 320,326 ---- * and must eventually be freed with heap_freetuple. */ static HeapTuple ! ScanPgRelation(RelationBuildDescInfo buildinfo, bool indexOK) { HeapTuple pg_class_tuple; Relation pg_class_desc; *************** *** 367,377 **** /* * Open pg_class and fetch a tuple. Force heap scan if we haven't yet * built the critical relcache entries (this includes initdb and ! * startup without a pg_internal.init file). */ pg_class_desc = heap_openr(RelationRelationName, AccessShareLock); pg_class_scan = systable_beginscan(pg_class_desc, indexRelname, ! criticalRelcachesBuilt, SnapshotNow, nkeys, key); --- 365,376 ---- /* * Open pg_class and fetch a tuple. Force heap scan if we haven't yet * built the critical relcache entries (this includes initdb and ! * startup without a pg_internal.init file). The caller can also ! * force a heap scan by setting indexOK == false. */ pg_class_desc = heap_openr(RelationRelationName, AccessShareLock); pg_class_scan = systable_beginscan(pg_class_desc, indexRelname, ! indexOK && criticalRelcachesBuilt, SnapshotNow, nkeys, key); *************** *** 834,840 **** /* * find the tuple in pg_class corresponding to the given relation id */ ! pg_class_tuple = ScanPgRelation(buildinfo); /* * if no such tuple exists, return NULL --- 833,839 ---- /* * find the tuple in pg_class corresponding to the given relation id */ ! pg_class_tuple = ScanPgRelation(buildinfo, true); /* * if no such tuple exists, return NULL *************** *** 875,881 **** * it could be new too, but it's okay to forget that fact if forced to * flush the entry.) */ ! relation->rd_isnailed = false; relation->rd_isnew = false; relation->rd_istemp = isTempNamespace(relation->rd_rel->relnamespace); --- 874,880 ---- * it could be new too, but it's okay to forget that fact if forced to * flush the entry.) */ ! relation->rd_isnailed = 0; relation->rd_isnew = false; relation->rd_istemp = isTempNamespace(relation->rd_rel->relnamespace); *************** *** 1386,1392 **** * all entries built with this routine are nailed-in-cache; none are * for new or temp relations. */ ! relation->rd_isnailed = true; relation->rd_isnew = false; relation->rd_istemp = false; --- 1385,1391 ---- * all entries built with this routine are nailed-in-cache; none are * for new or temp relations. */ ! relation->rd_isnailed = 1; relation->rd_isnew = false; relation->rd_istemp = false; *************** *** 1500,1506 **** * Lookup an existing reldesc by OID. * * Only try to get the reldesc by looking in the cache, ! * do not go to the disk. * * NB: relation ref count is incremented if successful. * Caller should eventually decrement count. (Usually, --- 1499,1505 ---- * Lookup an existing reldesc by OID. * * Only try to get the reldesc by looking in the cache, ! * do not go to the disk if it's not present. * * NB: relation ref count is incremented if successful. * Caller should eventually decrement count. (Usually, *************** *** 1514,1520 **** --- 1513,1524 ---- RelationIdCacheLookup(relationId, rd); if (RelationIsValid(rd)) + { RelationIncrementReferenceCount(rd); + /* revalidate nailed index if necessary */ + if (rd->rd_isnailed == 2) + RelationReloadClassinfo(rd); + } return rd; } *************** *** 1538,1548 **** --- 1542,1568 ---- RelationSysNameCacheLookup(NameStr(name), rd); if (RelationIsValid(rd)) + { RelationIncrementReferenceCount(rd); + /* revalidate nailed index if necessary */ + if (rd->rd_isnailed == 2) + RelationReloadClassinfo(rd); + } return rd; } + /* + * RelationNodeCacheGetRelation + * + * As above, but lookup by relfilenode. + * + * NOTE: this must NOT try to revalidate invalidated nailed indexes, since + * that could cause us to return an entry with a different relfilenode than + * the caller asked for. Currently this is used only by the buffer manager. + * Really the bufmgr's idea of relations should be separated out from the + * relcache ... + */ Relation RelationNodeCacheGetRelation(RelFileNode rnode) { *************** *** 1647,1685 **** #endif } - #ifdef ENABLE_REINDEX_NAILED_RELATIONS /* ! * RelationReloadClassinfo * ! * This function is especially for nailed relations. ! * relhasindex/relfilenode could be changed even for ! * nailed relations. */ static void RelationReloadClassinfo(Relation relation) { RelationBuildDescInfo buildinfo; HeapTuple pg_class_tuple; Form_pg_class relp; ! if (!relation->rd_rel) ! return; buildinfo.infotype = INFO_RELID; buildinfo.i.info_id = relation->rd_id; ! pg_class_tuple = ScanPgRelation(buildinfo); if (!HeapTupleIsValid(pg_class_tuple)) elog(ERROR, "could not find tuple for system relation %u", relation->rd_id); - RelationCacheDelete(relation); relp = (Form_pg_class) GETSTRUCT(pg_class_tuple); ! memcpy((char *) relation->rd_rel, (char *) relp, CLASS_TUPLE_SIZE); ! relation->rd_node.relNode = relp->relfilenode; ! RelationCacheInsert(relation); heap_freetuple(pg_class_tuple); ! ! return; } - #endif /* ENABLE_REINDEX_NAILED_RELATIONS */ /* * RelationClearRelation --- 1667,1726 ---- #endif } /* ! * RelationReloadClassinfo - reload the pg_class row (only) * ! * This function is used only for nailed indexes. Since a REINDEX can ! * change the relfilenode value for a nailed index, we have to reread ! * the pg_class row anytime we get an SI invalidation on a nailed index ! * (without throwing away the whole relcache entry, since we'd be unable ! * to rebuild it). ! * ! * We can't necessarily reread the pg_class row right away; we might be ! * in a failed transaction when we receive the SI notification. If so, ! * RelationClearRelation just marks the entry as invalid by setting ! * rd_isnailed to 2. This routine is called to fix the entry when it ! * is next needed. */ static void RelationReloadClassinfo(Relation relation) { RelationBuildDescInfo buildinfo; + bool indexOK; HeapTuple pg_class_tuple; Form_pg_class relp; ! /* Should be called only for invalidated nailed indexes */ ! Assert(relation->rd_isnailed == 2 && ! relation->rd_rel->relkind == RELKIND_INDEX); ! /* Read the pg_class row */ buildinfo.infotype = INFO_RELID; buildinfo.i.info_id = relation->rd_id; ! /* ! * Don't try to use an indexscan of pg_class_oid_index to reload the ! * info for pg_class_oid_index ... ! */ ! indexOK = strcmp(RelationGetRelationName(relation), ClassOidIndex) != 0; ! pg_class_tuple = ScanPgRelation(buildinfo, indexOK); if (!HeapTupleIsValid(pg_class_tuple)) elog(ERROR, "could not find tuple for system relation %u", relation->rd_id); relp = (Form_pg_class) GETSTRUCT(pg_class_tuple); ! if (relation->rd_node.relNode != relp->relfilenode) ! { ! /* We have to re-insert the entry into the relcache indexes */ ! RelationCacheDelete(relation); ! memcpy((char *) relation->rd_rel, (char *) relp, CLASS_TUPLE_SIZE); ! relation->rd_node.relNode = relp->relfilenode; ! RelationCacheInsert(relation); ! } heap_freetuple(pg_class_tuple); ! /* Must adjust number of blocks after we know the new relfilenode */ ! relation->rd_targblock = InvalidBlockNumber; ! RelationUpdateNumberOfBlocks(relation); ! /* Okay, now it's valid again */ ! relation->rd_isnailed = 1; } /* * RelationClearRelation *************** *** 1712,1726 **** * Never, never ever blow away a nailed-in system relation, because * we'd be unable to recover. However, we must update rd_nblocks and * reset rd_targblock, in case we got called because of a relation ! * cache flush that was triggered by VACUUM. */ if (relation->rd_isnailed) { ! relation->rd_targblock = InvalidBlockNumber; ! RelationUpdateNumberOfBlocks(relation); ! #ifdef ENABLE_REINDEX_NAILED_RELATIONS ! RelationReloadClassinfo(relation); ! #endif /* ENABLE_REINDEX_NAILED_RELATIONS */ return; } --- 1753,1779 ---- * Never, never ever blow away a nailed-in system relation, because * we'd be unable to recover. However, we must update rd_nblocks and * reset rd_targblock, in case we got called because of a relation ! * cache flush that was triggered by VACUUM. If it's a nailed index, ! * then we need to re-read the pg_class row to see if its relfilenode ! * changed. We can't necessarily do that here, because we might be in ! * a failed transaction. We assume it's okay to do it if there are open ! * references to the relcache entry (cf notes for AtEOXact_RelationCache). ! * Otherwise just mark the entry as possibly invalid, and it'll be fixed ! * when next opened. */ if (relation->rd_isnailed) { ! if (relation->rd_rel->relkind == RELKIND_INDEX) ! { ! relation->rd_isnailed = 2; /* needs to be revalidated */ ! if (relation->rd_refcnt > 1) ! RelationReloadClassinfo(relation); ! } ! else ! { ! relation->rd_targblock = InvalidBlockNumber; ! RelationUpdateNumberOfBlocks(relation); ! } return; } *************** *** 1928,1933 **** --- 1981,1992 ---- * because (a) during the first pass we won't process any more SI messages, * so hash_seq_search will complete safely; (b) during the second pass we * only hold onto pointers to nondeletable entries. + * + * The two-phase approach also makes it easy to ensure that we process + * nailed-in-cache indexes before other nondeletable items, and that we + * process pg_class_oid_index first of all. In scenarios where a nailed + * index has been given a new relfilenode, we have to detect that update + * before the nailed index is used in reloading any other relcache entry. */ void RelationCacheInvalidate(void) *************** *** 1935,1940 **** --- 1994,2000 ---- HASH_SEQ_STATUS status; RelIdCacheEnt *idhentry; Relation relation; + List *rebuildFirstList = NIL; List *rebuildList = NIL; List *l; *************** *** 1954,1968 **** if (RelationHasReferenceCountZero(relation)) { /* Delete this entry immediately */ RelationClearRelation(relation, false); } else { ! /* Add entry to list of stuff to rebuild in second pass */ ! rebuildList = lcons(relation, rebuildList); } } /* Phase 2: rebuild the items found to need rebuild in phase 1 */ foreach(l, rebuildList) { --- 2014,2046 ---- if (RelationHasReferenceCountZero(relation)) { /* Delete this entry immediately */ + Assert(!relation->rd_isnailed); RelationClearRelation(relation, false); } else { ! /* ! * Add this entry to list of stuff to rebuild in second pass. ! * pg_class_oid_index goes on the front of rebuildFirstList, ! * other nailed indexes on the back, and everything else into ! * rebuildList (in no particular order). ! */ ! if (relation->rd_isnailed && ! relation->rd_rel->relkind == RELKIND_INDEX) ! { ! if (strcmp(RelationGetRelationName(relation), ! ClassOidIndex) == 0) ! rebuildFirstList = lcons(relation, rebuildFirstList); ! else ! rebuildFirstList = lappend(rebuildFirstList, relation); ! } ! else ! rebuildList = lcons(relation, rebuildList); } } + rebuildList = nconc(rebuildFirstList, rebuildList); + /* Phase 2: rebuild the items found to need rebuild in phase 1 */ foreach(l, rebuildList) { *************** *** 1976,1981 **** --- 2054,2064 ---- * AtEOXact_RelationCache * * Clean up the relcache at transaction commit or abort. + * + * Note: this must be called *before* processing invalidation messages. + * In the case of abort, we don't want to try to rebuild any invalidated + * cache entries (since we can't safely do database accesses). Therefore + * we must reset refcnts before handling pending invalidations. */ void AtEOXact_RelationCache(bool commit) *************** *** 2045,2050 **** --- 2128,2143 ---- /* abort case, just reset it quietly */ RelationSetReferenceCount(relation, expected_refcnt); } + + /* + * Flush any temporary index list. + */ + if (relation->rd_indexvalid == 2) + { + freeList(relation->rd_indexlist); + relation->rd_indexlist = NIL; + relation->rd_indexvalid = 0; + } } } *************** *** 2101,2107 **** * want it kicked out. e.g. pg_attribute!!! */ if (nailit) ! rel->rd_isnailed = true; /* * create a new tuple descriptor from the one passed in. We do this --- 2194,2200 ---- * want it kicked out. e.g. pg_attribute!!! */ if (nailit) ! rel->rd_isnailed = 1; /* * create a new tuple descriptor from the one passed in. We do this *************** *** 2288,2294 **** buildinfo.infotype = INFO_RELNAME; \ buildinfo.i.info_name = (indname); \ ird = RelationBuildDesc(buildinfo, NULL); \ ! ird->rd_isnailed = true; \ RelationSetReferenceCount(ird, 1); \ } while (0) --- 2381,2387 ---- buildinfo.infotype = INFO_RELNAME; \ buildinfo.i.info_name = (indname); \ ird = RelationBuildDesc(buildinfo, NULL); \ ! ird->rd_isnailed = 1; \ RelationSetReferenceCount(ird, 1); \ } while (0) *************** *** 2575,2581 **** * The index list is created only if someone requests it. We scan pg_index * to find relevant indexes, and add the list to the relcache entry so that * we won't have to compute it again. Note that shared cache inval of a ! * relcache entry will delete the old list and set rd_indexfound to false, * so that we must recompute the index list on next request. This handles * creation or deletion of an index. * --- 2668,2674 ---- * The index list is created only if someone requests it. We scan pg_index * to find relevant indexes, and add the list to the relcache entry so that * we won't have to compute it again. Note that shared cache inval of a ! * relcache entry will delete the old list and set rd_indexvalid to 0, * so that we must recompute the index list on next request. This handles * creation or deletion of an index. * *************** *** 2602,2608 **** MemoryContext oldcxt; /* Quick exit if we already computed the list. */ ! if (relation->rd_indexfound) return listCopy(relation->rd_indexlist); /* --- 2695,2701 ---- MemoryContext oldcxt; /* Quick exit if we already computed the list. */ ! if (relation->rd_indexvalid != 0) return listCopy(relation->rd_indexlist); /* *************** *** 2638,2644 **** /* Now save a copy of the completed list in the relcache entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); relation->rd_indexlist = listCopy(result); ! relation->rd_indexfound = true; MemoryContextSwitchTo(oldcxt); return result; --- 2731,2737 ---- /* Now save a copy of the completed list in the relcache entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); relation->rd_indexlist = listCopy(result); ! relation->rd_indexvalid = 1; MemoryContextSwitchTo(oldcxt); return result; *************** *** 2677,2682 **** --- 2770,2804 ---- } /* + * RelationSetIndexList -- externally force the index list contents + * + * This is used to temporarily override what we think the set of valid + * indexes is. The forcing will be valid only until transaction commit + * or abort. + * + * This should only be applied to nailed relations, because in a non-nailed + * relation the hacked index list could be lost at any time due to SI + * messages. In practice it is only used on pg_class (see REINDEX). + * + * It is up to the caller to make sure the given list is correctly ordered. + */ + void + RelationSetIndexList(Relation relation, List *indexIds) + { + MemoryContext oldcxt; + + Assert(relation->rd_isnailed == 1); + /* Copy the list into the cache context (could fail for lack of mem) */ + oldcxt = MemoryContextSwitchTo(CacheMemoryContext); + indexIds = listCopy(indexIds); + MemoryContextSwitchTo(oldcxt); + /* Okay to replace old list */ + freeList(relation->rd_indexlist); + relation->rd_indexlist = indexIds; + relation->rd_indexvalid = 2; /* mark list as forced */ + } + + /* * RelationGetIndexExpressions -- get the index expressions for an index * * We cache the result of transforming pg_index.indexprs into a node tree. *************** *** 3087,3093 **** RelationSetReferenceCount(rel, 1); else RelationSetReferenceCount(rel, 0); ! rel->rd_indexfound = false; rel->rd_indexlist = NIL; MemSet(&rel->pgstat_info, 0, sizeof(rel->pgstat_info)); --- 3209,3215 ---- RelationSetReferenceCount(rel, 1); else RelationSetReferenceCount(rel, 0); ! rel->rd_indexvalid = 0; rel->rd_indexlist = NIL; MemSet(&rel->pgstat_info, 0, sizeof(rel->pgstat_info)); *** src/backend/utils/cache/syscache.c.orig Sun Aug 3 23:01:06 2003 --- src/backend/utils/cache/syscache.c Sun Sep 21 16:55:31 2003 *************** *** 436,452 **** }} }; ! static CatCache *SysCache[ ! lengthof(cacheinfo)]; static int SysCacheSize = lengthof(cacheinfo); static bool CacheInitialized = false; - - - bool - IsCacheInitialized(void) - { - return CacheInitialized; - } /* --- 436,444 ---- }} }; ! static CatCache *SysCache[lengthof(cacheinfo)]; static int SysCacheSize = lengthof(cacheinfo); static bool CacheInitialized = false; /* *** src/backend/utils/init/miscinit.c.orig Mon Aug 4 00:03:10 2003 --- src/backend/utils/init/miscinit.c Mon Sep 22 17:44:19 2003 *************** *** 51,56 **** --- 51,61 ---- /* ---------------------------------------------------------------- * ignoring system indexes support stuff + * + * NOTE: "ignoring system indexes" means we do not use the system indexes + * for lookups (either in hardwired catalog accesses or in planner-generated + * plans). We do, however, still update the indexes when a catalog + * modification is made. * ---------------------------------------------------------------- */ *************** *** 61,80 **** * True if ignoring system indexes. */ bool ! IsIgnoringSystemIndexes() { return isIgnoringSystemIndexes; } /* * IgnoreSystemIndexes ! * Set true or false whether PostgreSQL ignores system indexes. ! * */ void IgnoreSystemIndexes(bool mode) { isIgnoringSystemIndexes = mode; } /* ---------------------------------------------------------------- --- 66,131 ---- * True if ignoring system indexes. */ bool ! IsIgnoringSystemIndexes(void) { return isIgnoringSystemIndexes; } /* * IgnoreSystemIndexes ! * Set true or false whether PostgreSQL ignores system indexes. */ void IgnoreSystemIndexes(bool mode) { isIgnoringSystemIndexes = mode; + } + + /* ---------------------------------------------------------------- + * system index reindexing support + * + * When we are busy reindexing a system index, this code provides support + * for preventing catalog lookups from using that index. + * ---------------------------------------------------------------- + */ + + static Oid currentlyReindexedHeap = InvalidOid; + static Oid currentlyReindexedIndex = InvalidOid; + + /* + * ReindexIsProcessingHeap + * True if heap specified by OID is currently being reindexed. + */ + bool + ReindexIsProcessingHeap(Oid heapOid) + { + return heapOid == currentlyReindexedHeap; + } + + /* + * ReindexIsProcessingIndex + * True if index specified by OID is currently being reindexed. + */ + bool + ReindexIsProcessingIndex(Oid indexOid) + { + return indexOid == currentlyReindexedIndex; + } + + /* + * SetReindexProcessing + * Set flag that specified heap/index are being reindexed. + * Pass InvalidOid to indicate that reindexing is not active. + */ + void + SetReindexProcessing(Oid heapOid, Oid indexOid) + { + /* Args should be both, or neither, InvalidOid */ + Assert((heapOid == InvalidOid) == (indexOid == InvalidOid)); + /* Reindexing is not re-entrant. */ + Assert(indexOid == InvalidOid || currentlyReindexedIndex == InvalidOid); + currentlyReindexedHeap = heapOid; + currentlyReindexedIndex = indexOid; } /* ---------------------------------------------------------------- *** src/include/catalog/index.h.orig Sun Aug 3 23:01:28 2003 --- src/include/catalog/index.h Sun Sep 21 19:25:22 2003 *************** *** 51,65 **** char *nullv); extern void UpdateStats(Oid relid, double reltuples); ! extern bool IndexesAreActive(Relation heaprel); extern void setRelhasindex(Oid relid, bool hasindex, bool isprimary, Oid reltoastidxid); extern void setNewRelfilenode(Relation relation); - extern bool SetReindexProcessing(bool processing); - extern bool IsReindexProcessing(void); - extern void index_build(Relation heapRelation, Relation indexRelation, IndexInfo *indexInfo); --- 51,62 ---- char *nullv); extern void UpdateStats(Oid relid, double reltuples); ! extern void setRelhasindex(Oid relid, bool hasindex, bool isprimary, Oid reltoastidxid); extern void setNewRelfilenode(Relation relation); extern void index_build(Relation heapRelation, Relation indexRelation, IndexInfo *indexInfo); *************** *** 69,77 **** IndexBuildCallback callback, void *callback_state); ! extern bool activate_indexes_of_a_table(Relation heaprel, bool activate); ! ! extern bool reindex_index(Oid indexId, bool force, bool inplace); ! extern bool reindex_relation(Oid relid, bool force); #endif /* INDEX_H */ --- 66,72 ---- IndexBuildCallback callback, void *callback_state); ! extern void reindex_index(Oid indexId); ! extern bool reindex_relation(Oid relid); #endif /* INDEX_H */ *** src/include/miscadmin.h.orig Tue Aug 26 11:38:25 2003 --- src/include/miscadmin.h Mon Sep 22 17:44:14 2003 *************** *** 296,313 **** extern void BaseInit(void); /* in utils/init/miscinit.c */ extern void CreateDataDirLockFile(const char *datadir, bool amPostmaster); extern void CreateSocketLockFile(const char *socketfile, bool amPostmaster); extern void TouchSocketLockFile(void); extern void RecordSharedMemoryInLockFile(unsigned long id1, unsigned long id2); - extern void ValidatePgVersion(const char *path); extern void process_preload_libraries(char *preload_libraries_string); - - /* these externs do not belong here... */ - extern void IgnoreSystemIndexes(bool mode); - extern bool IsIgnoringSystemIndexes(void); - extern bool IsCacheInitialized(void); #endif /* MISCADMIN_H */ --- 296,312 ---- extern void BaseInit(void); /* in utils/init/miscinit.c */ + extern void IgnoreSystemIndexes(bool mode); + extern bool IsIgnoringSystemIndexes(void); + extern void SetReindexProcessing(Oid heapOid, Oid indexOid); + extern bool ReindexIsProcessingHeap(Oid heapOid); + extern bool ReindexIsProcessingIndex(Oid indexOid); extern void CreateDataDirLockFile(const char *datadir, bool amPostmaster); extern void CreateSocketLockFile(const char *socketfile, bool amPostmaster); extern void TouchSocketLockFile(void); extern void RecordSharedMemoryInLockFile(unsigned long id1, unsigned long id2); extern void ValidatePgVersion(const char *path); extern void process_preload_libraries(char *preload_libraries_string); #endif /* MISCADMIN_H */ *** src/include/utils/errcodes.h.orig Tue Aug 26 17:15:27 2003 --- src/include/utils/errcodes.h Mon Sep 22 18:40:53 2003 *************** *** 281,287 **** /* Class 55 - Object Not In Prerequisite State (class borrowed from DB2) */ #define ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE MAKE_SQLSTATE('5','5', '0','0','0') #define ERRCODE_OBJECT_IN_USE MAKE_SQLSTATE('5','5', '0','0','6') - #define ERRCODE_INDEXES_DEACTIVATED MAKE_SQLSTATE('5','5', 'P','0','1') #define ERRCODE_CANT_CHANGE_RUNTIME_PARAM MAKE_SQLSTATE('5','5', 'P','0','2') /* Class 57 - Operator Intervention (class borrowed from DB2) */ --- 281,286 ---- *** src/include/utils/rel.h.orig Sun Aug 3 23:01:45 2003 --- src/include/utils/rel.h Mon Sep 22 13:34:07 2003 *************** *** 119,126 **** * it is possible for new-ness to be "forgotten" (eg, after CLUSTER). */ bool rd_istemp; /* rel uses the local buffer mgr */ ! bool rd_isnailed; /* rel is nailed in cache */ ! bool rd_indexfound; /* true if rd_indexlist is valid */ Form_pg_class rd_rel; /* RELATION tuple */ TupleDesc rd_att; /* tuple descriptor */ Oid rd_id; /* relation's object id */ --- 119,128 ---- * it is possible for new-ness to be "forgotten" (eg, after CLUSTER). */ bool rd_istemp; /* rel uses the local buffer mgr */ ! char rd_isnailed; /* rel is nailed in cache: 0 = no, 1 = yes, ! * 2 = yes but possibly invalid */ ! char rd_indexvalid; /* state of rd_indexlist: 0 = not valid, ! * 1 = valid, 2 = temporarily forced */ Form_pg_class rd_rel; /* RELATION tuple */ TupleDesc rd_att; /* tuple descriptor */ Oid rd_id; /* relation's object id */ *** src/include/utils/relcache.h.orig Sun Aug 3 23:01:45 2003 --- src/include/utils/relcache.h Sun Sep 21 19:25:15 2003 *************** *** 35,40 **** --- 35,42 ---- extern List *RelationGetIndexExpressions(Relation relation); extern List *RelationGetIndexPredicate(Relation relation); + extern void RelationSetIndexList(Relation relation, List *indexIds); + extern void RelationInitIndexAccessInfo(Relation relation); /*