From a287c635266b70666cbd1031991bf7b5f950dc0f Mon Sep 17 00:00:00 2001 From: Andrey Borodin Date: Tue, 21 Jul 2026 00:49:38 +0500 Subject: [PATCH 1/3] Delete GIN posting tree pages without locking out the whole tree Previously, deleting empty posting tree pages required a second pass over the tree under a cleanup lock on its root, blocking all searches and insertions in the tree for the duration of the pass; the pass was also uninterruptible, as interrupts are not processed while holding a buffer lock. Instead, delete empty leaf pages on the fly during the regular left-to-right vacuum sweep of the leaf level. The sweep couples leaf locks in the same left-to-right order used everywhere else in GIN, and the parent is found by walking right along the level above the leaves. A page is deleted only if nobody else holds a pin on it. The leftmost and rightmost leaves, leaves referenced by the last downlink of their parent, and internal pages are never deleted. Inserters reaching a deleted page through a stale downlink recover by moving right; the existing XID stamping of deleted pages prevents premature reuse. Discussion: https://www.postgresql.org/message-id/70A65F5B-D9BB-4CD8-9A50-B33B7C1AC7DD@yandex-team.ru --- src/backend/access/gin/README | 147 +++++----- src/backend/access/gin/ginbtree.c | 20 +- src/backend/access/gin/ginvacuum.c | 421 +++++++++++++++-------------- 3 files changed, 298 insertions(+), 290 deletions(-) diff --git a/src/backend/access/gin/README b/src/backend/access/gin/README index 4c16dc4d4e0..2ae49fad652 100644 --- a/src/backend/access/gin/README +++ b/src/backend/access/gin/README @@ -390,88 +390,95 @@ produced by split. Parent 'c' is about to have downlink inserted. Vacuum never deletes tuples or pages from the entry tree. It traverses entry tree leafs in logical order by rightlinks and removes deletable TIDs from -posting lists. Posting trees are processed by links from entry tree leafs. They -are vacuumed in two stages. At first stage, deletable TIDs are removed from -leafs. If first stage detects at least one empty page, then at the second stage -ginScanPostingTreeToDelete() deletes empty pages. +posting lists. Posting trees are processed by links from entry tree leafs. + +A posting tree is vacuumed in a single left-to-right sweep over its leaf +level: vacuum descends to the leftmost leaf and follows rightlinks, removing +deletable TIDs from each leaf, and deleting leaf pages that have become +completely empty. While advancing, the current page is kept locked until the +lock on its right sibling is acquired (lock coupling), following the same +left-to-right locking order as stepping right, so the sweep cannot deadlock +with concurrent insertions or searches. A page is deleted only if no other +backend holds a pin on it (so that, together with the exclusive lock, we +effectively hold a cleanup lock): a pin would mean that some scan has copied +the page's contents and may yet return to it. If the page is pinned, we +just leave it in place; a future vacuum can delete it. Concurrent +insertions into the tree are not locked out; correctness is maintained as +described below. + +The picture below shows the sweep positioned at leaf 'f', with its left +sibling 'e' still locked. -ginScanPostingTreeToDelete() traverses the whole tree in depth-first manner. -It starts from the full cleanup lock on the tree root. This lock prevents all -the concurrent insertions into this tree while we're deleting pages. However, -there are still might be some in-progress readers, who traversed root before -we locked it. + a + / | \ + b c d + / | \ | \ | \ + eE fE g h i j k -The picture below shows tree state after page deletion algorithm traversed to -leftmost leaf of the tree. +To delete leaf 'f', vacuum locates the parent page by descending to the +leftmost internal page one level above the leaves before the sweep, and then +walking right, as downlinks can only move rightwards (when internal pages +split). With 'e', 'f' and the parent 'b' exclusively locked, the downlink to +'f' is removed from 'b', the rightlink of 'e' is redirected from 'f' to 'g', +and 'f' is marked with the 'deleted' flag and stamped with the newest xid +which might visit it. - aE + a / | \ bE c d - / | \ | \ | \ - eE f g h i j k + / \ | \ | \ + eE fDE g h i j k -Deletion algorithm keeps exclusive locks on left siblings of pages comprising -currently investigated path. Thus, if current page is to be removed, all -required pages to remove both downlink and rightlink are already locked. That -avoids potential right to left page locking order, which could deadlock with -concurrent stepping right. - -A search concurrent to page deletion might already have read a pointer to the -page to be deleted, and might be just about to follow it. A page can be reached +A search or insertion concurrent to the page deletion might already have read +a pointer to 'f', and might be just about to follow it. A page can be reached via the right-link of its left sibling, or via its downlink in the parent. To prevent a backend from reaching a deleted page via a right-link, stepping right algorithm doesn't release lock on the current page until lock of the -right page is acquired. - -The downlink is more tricky. A search descending the tree must release the lock -on the parent page before locking the child, or it could deadlock with a -concurrent split of the child page; a page split locks the parent, while already -holding a lock on the child page. So, deleted page cannot be reclaimed -immediately. Instead, we have to wait for every transaction, which might wait -to reference this page, to finish. Corresponding processes must observe that -the page is marked deleted and recover accordingly. - -The picture below shows tree state after page deletion algorithm further -traversed the tree. Currently investigated path is 'a-c-h'. Left siblings 'b' -and 'g' of 'c' and 'h' correspondingly are also exclusively locked. - - aE - / | \ - bE cE d - / | \ | \ | \ - e f gE hE i j k - -The next picture shows tree state after page 'h' was deleted. It's marked with -'deleted' flag and newest xid, which might visit it. Downlink from 'c' to 'h' -is also deleted. - - aE - / | \ - bE cE d - / | \ \ | \ - e f gE hD iE j k - -However, it's still possible that concurrent reader has seen downlink from 'c' -to 'h' before we deleted it. In that case this reader will step right from 'h' -to till find non-deleted page. Xid-marking of page 'h' guarantees that this -page wouldn't be reused till all such readers gone. Next leaf page under -investigation is 'i'. 'g' remains locked as it becomes left sibling of 'i'. - -The next picture shows tree state after 'i' and 'c' was deleted. Internal page -'c' was deleted because it appeared to have no downlinks. The path under -investigation is 'a-d-j'. Pages 'b' and 'g' are locked as self siblings of 'd' -and 'j'. - - aE - / \ - bE cD dE - / | \ | \ - e f gE hD iD jE k +right page is acquired; and vacuum, in turn, only deletes a page while holding +the lock on its left sibling, so it cannot overtake such a backend. + +The downlink is more tricky. A search descending the tree must release the +lock on the parent page before locking the child, or it could deadlock with a +concurrent split of the child page; a page split locks the parent, while +already holding a lock on the child page. So, a backend that read the +downlink to 'f' before vacuum removed it may arrive at 'f' after 'f' was +deleted. Both searches and insertions recover by stepping right from the +deleted page, which keeps its rightlink; the keyspace of the deleted page has +been implicitly merged into its right sibling (in the parent, keys between +the previous downlink's key and the deleted downlink's key now fall to the +next downlink). For the same reason, a deleted page cannot be reclaimed +immediately: we have to wait for every transaction, which might visit this +page, to finish. The xid stamped on the deleted page guarantees that the +page wouldn't be reused till all such backends are gone. + +Some pages must survive even when empty, for the tree to stay navigable: + +* The leftmost leaf, and more generally the leftmost page of each level, is + never deleted. Full scans of a posting tree descend by leftmost downlinks + without comparing keys and cannot recover by moving right, so the leftmost + page they land on must not go away. + +* A page referenced by the last downlink of its parent is never deleted. + Posting tree internal pages have no high keys; the key of the last + downlink is treated as "right infinity" within the parent's keyspace. + Removing the last downlink would cut the parent's effective keyspace + short, misrouting subsequent insertions of keys between the remaining + downlinks' keys and the parent's right bound into pages under the + parent's right sibling, and corrupting the key order of the tree. A + useful consequence of this rule is that an internal page always keeps at + least one downlink. + +* The rightmost leaf is never deleted. + +* Internal pages are never deleted. Deleting them safely would require + true high keys, as in the regular B-tree indexam. Empty internal pages + cannot arise anyway, as the last downlink is never removed. During the replay of page deletion at standby, the page's left sibling, the -target page, and its parent, are locked in that order. This order guarantees -no deadlock with concurrent reads. +target page, and its parent, are locked in that order. This is the same +order in which vacuum acquires them, and it guarantees no deadlock with +concurrent reads. Predicate Locking ----------------- diff --git a/src/backend/access/gin/ginbtree.c b/src/backend/access/gin/ginbtree.c index 3d3a9da56b1..019ea47730c 100644 --- a/src/backend/access/gin/ginbtree.c +++ b/src/backend/access/gin/ginbtree.c @@ -168,10 +168,8 @@ ginFindLeafPage(GinBtree btree, bool searchMode, * * The next page is locked first, before releasing the current page. This is * crucial to prevent concurrent VACUUM from deleting a page that we are about - * to step to. (The lock-coupling isn't strictly necessary when we are - * traversing the tree to find an insert location, because page deletion grabs - * a cleanup lock on the root to prevent any concurrent inserts. See Page - * deletion section in the README. But there's no harm in doing it always.) + * to step to: VACUUM deletes a page only while holding a lock on its left + * sibling. See Page deletion section in the README. */ Buffer ginStepRight(Buffer buffer, Relation index, int lockmode) @@ -229,8 +227,9 @@ ginFindParents(GinBtree btree, GinBtreeStack *stack) * Unwind the stack all the way up to the root, leaving only the root * item. * - * Be careful not to release the pin on the root page! The pin on root - * page is required to lock out concurrent vacuums on the tree. + * This search visits only internal pages, which vacuum never deletes, + * so we don't need any special protection from concurrent vacuums. + * Keeping the pin on the root page just saves a buffer lookup. */ root = stack->parent; while (root->parent) @@ -769,11 +768,10 @@ ginFinishSplit(GinBtree btree, GinBtreeStack *stack, bool freestack, * split that we just made ourselves. The difference is that stack->buffer may * be merely share-locked on entry, and will be upgraded to exclusive mode. * - * Note: Upgrading the lock momentarily releases it. Doing that in a scan - * would not be OK, because a concurrent VACUUM might delete the page while - * we're not holding the lock. It's OK in an insert, though, because VACUUM - * has a different mechanism that prevents it from running concurrently with - * inserts. (Namely, it holds a cleanup lock on the root.) + * Note: Upgrading the lock momentarily releases it. That is OK, because + * we hold a pin on the page throughout, and a concurrent VACUUM can only + * delete a page after acquiring a cleanup lock on it, which requires that + * nobody else holds a pin on it. */ static void ginFinishOldSplit(GinBtree btree, GinBtreeStack *stack, GinStatsData *buildStats, int access) diff --git a/src/backend/access/gin/ginvacuum.c b/src/backend/access/gin/ginvacuum.c index 840543eb664..ca4c2c4d953 100644 --- a/src/backend/access/gin/ginvacuum.c +++ b/src/backend/access/gin/ginvacuum.c @@ -111,59 +111,32 @@ xlogVacuumPage(Relation index, Buffer buffer) } -/* - * Stack entry used during posting tree empty-page deletion scan. - * - * One DataPageDeleteStack entry is allocated per tree level. As - * ginScanPostingTreeToDelete() recurses down the tree, each entry tracks - * the buffer of the page currently being visited at that level ('buffer'), - * and the buffer of its left sibling ('leftBuffer'). The left page is kept - * pinned and exclusively locked because ginDeletePostingPage() needs it to - * update the sibling chain; acquiring it later could deadlock with - * ginStepRight(), which locks pages left-to-right. - */ -typedef struct DataPageDeleteStack -{ - struct DataPageDeleteStack *child; - struct DataPageDeleteStack *parent; - - Buffer buffer; /* buffer for the page being visited at this - * tree level */ - Buffer leftBuffer; /* pinned and locked rightmost non-deleted - * sibling to the left of the current page */ - OffsetNumber myoff; /* offset of this page's downlink in the - * parent */ - bool isRoot; -} DataPageDeleteStack; - - /* * Delete a posting tree page. * * Removes the page identified by dBuffer from the posting tree by updating * the left sibling's rightlink (in lBuffer) to skip over the deleted page, - * and removing the downlink from the parent page (in pBuffer). All three - * buffers must already have been pinned and exclusively locked by the caller. + * and removing the downlink from the parent page (in pBuffer). + * + * The caller must hold exclusive locks on all three buffers, and no other + * backend may hold a pin on dBuffer (IsBufferCleanupOK). Concurrent + * inserters and searchers that read the downlink or the left sibling's + * rightlink before we remove them recover by observing the deleted flag + * and moving right; the page is stamped with an XID and cannot be recycled + * until they are all gone (see README). * * The buffers are NOT released nor unlocked here; the caller is responsible * for this. */ static void ginDeletePostingPage(GinVacuumState *gvs, Buffer dBuffer, Buffer lBuffer, - Buffer pBuffer, OffsetNumber myoff, bool isParentRoot) + Buffer pBuffer, OffsetNumber myoff) { Page page, parentPage; BlockNumber rightlink; BlockNumber deleteBlkno = BufferGetBlockNumber(dBuffer); - /* - * This function MUST be called only if someone of parent pages hold - * exclusive cleanup lock. This guarantees that no insertions currently - * happen in this subtree. Caller also acquires Exclusive locks on - * deletable, parent and left pages. - */ - page = BufferGetPage(dBuffer); rightlink = GinPageGetOpaque(page)->rightlink; @@ -247,241 +220,271 @@ ginDeletePostingPage(GinVacuumState *gvs, Buffer dBuffer, Buffer lBuffer, /* - * Scans a posting tree and deletes empty pages. - * - * The caller must hold a cleanup lock on the root page to prevent concurrent - * inserts. The entire path from the root down to the current page is kept - * exclusively locked throughout the scan. The left sibling at each level is - * also kept locked, because ginDeletePostingPage() needs it to update the - * rightlink of the left sibling; re-acquiring the left sibling lock later - * could deadlock with ginStepRight(), which acquires page locks - * left-to-right. + * Descend from the posting tree root to its leftmost leaf page. * - * All per-level state is carried in 'myStackItem': the buffer to process - * (must already be pinned and exclusively locked), the left sibling buffer, - * and this page's offset in the parent's downlink array. The root entry is - * set up by ginVacuumPostingTree(); child entries are populated here before - * recursing. + * On return, the leftmost leaf is pinned and exclusively locked, and + * *parentBlkno is set to the block number of its parent (the leftmost + * internal page one level above the leaves), or InvalidBlockNumber if the + * root itself is a leaf. * - * Returns true if the page was deleted, false otherwise. + * The leftmost path is stable: page splits move keyspace to the right, and + * the leftmost page of each level is never deleted (see README), so the + * downlinks we follow cannot go away under us. */ -static bool -ginScanPostingTreeToDelete(GinVacuumState *gvs, DataPageDeleteStack *myStackItem) +static Buffer +ginStepToLeftmostLeaf(GinVacuumState *gvs, BlockNumber rootBlkno, + BlockNumber *parentBlkno) { - Buffer buffer = myStackItem->buffer; - Page page; - bool pageWasDeleted = false; - bool isempty; + BlockNumber blkno = rootBlkno; - page = BufferGetPage(buffer); + *parentBlkno = InvalidBlockNumber; - Assert(GinPageIsData(page)); - - if (!GinPageIsLeaf(page)) + while (true) { - OffsetNumber i; - - for (i = FirstOffsetNumber; i <= GinPageGetOpaque(page)->maxoff;) - { - PostingItem *pitem = GinDataPageGetPostingItem(page, i); - Buffer childBuffer; + Buffer buffer; + Page page; + PostingItem *pitem; - childBuffer = ReadBufferExtended(gvs->index, - MAIN_FORKNUM, - PostingItemGetBlockNumber(pitem), - RBM_NORMAL, gvs->strategy); - LockBuffer(childBuffer, GIN_EXCLUSIVE); + buffer = ReadBufferExtended(gvs->index, MAIN_FORKNUM, blkno, + RBM_NORMAL, gvs->strategy); + LockBuffer(buffer, GIN_SHARE); + page = BufferGetPage(buffer); - /* Allocate a child stack entry on first use; reuse thereafter */ - if (!myStackItem->child) - { - myStackItem->child = palloc0_object(DataPageDeleteStack); - myStackItem->child->parent = myStackItem; - myStackItem->child->leftBuffer = InvalidBuffer; - } + Assert(GinPageIsData(page)); - myStackItem->child->buffer = childBuffer; - myStackItem->child->isRoot = false; - myStackItem->child->myoff = i; + if (GinPageIsLeaf(page)) + { + LockBuffer(buffer, GIN_UNLOCK); + LockBuffer(buffer, GIN_EXCLUSIVE); /* - * Recurse into child. If the child page was deleted, its - * downlink was removed from our page, so re-examine the same - * offset; otherwise advance to the next downlink. + * While the page was unlocked, a concurrent insert could have + * turned the root into an internal page. Retry from the same + * block if so. (Non-root pages never change their leaf-ness.) */ - if (!ginScanPostingTreeToDelete(gvs, myStackItem->child)) - i++; - } - myStackItem->buffer = InvalidBuffer; + if (GinPageIsLeaf(page)) + return buffer; - /* - * After processing all children at this level, release the child - * level's leftBuffer if we're at the rightmost page. There is no - * right sibling that could need it for deletion. - */ - if (GinPageRightMost(page) && BufferIsValid(myStackItem->child->leftBuffer)) - { - UnlockReleaseBuffer(myStackItem->child->leftBuffer); - myStackItem->child->leftBuffer = InvalidBuffer; + UnlockReleaseBuffer(buffer); + continue; } - } - if (GinPageIsLeaf(page)) - isempty = GinDataLeafPageIsEmpty(page); - else - isempty = GinPageGetOpaque(page)->maxoff < FirstOffsetNumber; + Assert(PageGetMaxOffsetNumber(page) >= FirstOffsetNumber); - if (isempty) - { - /* - * Proceed to the ginDeletePostingPage() if that's not the leftmost or - * the rightmost page. - */ - if (BufferIsValid(myStackItem->leftBuffer) && !GinPageRightMost(page)) - { - Assert(!myStackItem->isRoot); - ginDeletePostingPage(gvs, buffer, myStackItem->leftBuffer, - myStackItem->parent->buffer, - myStackItem->myoff, - myStackItem->parent->isRoot); - pageWasDeleted = true; - } - } + *parentBlkno = blkno; + pitem = GinDataPageGetPostingItem(page, FirstOffsetNumber); + blkno = PostingItemGetBlockNumber(pitem); + Assert(blkno != InvalidBlockNumber); - if (!pageWasDeleted) - { - /* - * Keep this page as the new leftBuffer for this level: the next - * sibling to the right might need it for deletion. Release any - * previously held left page first. - */ - if (BufferIsValid(myStackItem->leftBuffer)) - UnlockReleaseBuffer(myStackItem->leftBuffer); - myStackItem->leftBuffer = buffer; - } - else - { - /* - * Page was deleted; release the buffer. leftBuffer remains the same. - */ UnlockReleaseBuffer(buffer); } - - return pageWasDeleted; } - /* - * Scan through posting tree leafs, delete empty tuples. Returns true if there - * is at least one empty page. + * Find and exclusively lock the parent of leaf page leafBlkno, in + * preparation for deleting the leaf. + * + * *parentBlkno is used as a search hint, pointing to some page of the + * internal level immediately above the leaves, at or to the left of the + * parent we are looking for. Because downlinks only ever move right (when + * internal pages split) and our caller processes the leaves in + * left-to-right order, the hint page from a previous call remains valid. + * The search walks right from the hint until the downlink is found. + * + * On success, returns the pinned and exclusively locked parent buffer, + * sets *off to the downlink's offset, and advances *parentBlkno to the + * parent's block number. Returns InvalidBuffer (with *parentBlkno + * unchanged, so that searches for subsequent leaves can still succeed) if: + * + * - the downlink was not found (e.g. the leaf's split was never completed, + * so the downlink was never inserted); or + * + * - the downlink is the last one on the parent page. Posting tree internal + * pages have no high keys; dataLocateItem() treats the last downlink as + * having no upper bound ("right infinity"). Removing it would cut the + * parent's keyspace short, sending insertions of keys between the + * remaining downlinks and the parent's right bound to pages under the + * parent's right sibling, corrupting the key order of the tree. So the + * last downlink, and thereby the page it points to, must stay. */ -static bool -ginVacuumPostingTreeLeaves(GinVacuumState *gvs, BlockNumber blkno) +static Buffer +ginLockLeafParent(GinVacuumState *gvs, BlockNumber *parentBlkno, + BlockNumber leafBlkno, OffsetNumber *off) { - Buffer buffer; - Page page; - bool hasVoidPage = false; - MemoryContext oldCxt; + BlockNumber blkno = *parentBlkno; - /* Find leftmost leaf page of posting tree and lock it in exclusive mode */ - while (true) + while (BlockNumberIsValid(blkno)) { - PostingItem *pitem; + Buffer buffer; + Page page; + OffsetNumber i, + maxoff; buffer = ReadBufferExtended(gvs->index, MAIN_FORKNUM, blkno, RBM_NORMAL, gvs->strategy); - LockBuffer(buffer, GIN_SHARE); + LockBuffer(buffer, GIN_EXCLUSIVE); page = BufferGetPage(buffer); Assert(GinPageIsData(page)); + Assert(!GinPageIsLeaf(page)); - if (GinPageIsLeaf(page)) + maxoff = GinPageGetOpaque(page)->maxoff; + for (i = FirstOffsetNumber; i <= maxoff; i++) { - LockBuffer(buffer, GIN_UNLOCK); - LockBuffer(buffer, GIN_EXCLUSIVE); - break; - } + PostingItem *pitem = GinDataPageGetPostingItem(page, i); - Assert(PageGetMaxOffsetNumber(page) >= FirstOffsetNumber); + if (PostingItemGetBlockNumber(pitem) == leafBlkno) + { + /* never delete the last downlink of an internal page */ + if (i == maxoff) + { + UnlockReleaseBuffer(buffer); + return InvalidBuffer; + } - pitem = GinDataPageGetPostingItem(page, FirstOffsetNumber); - blkno = PostingItemGetBlockNumber(pitem); - Assert(blkno != InvalidBlockNumber); + *parentBlkno = blkno; + *off = i; + return buffer; + } + } + blkno = GinPageGetOpaque(page)->rightlink; UnlockReleaseBuffer(buffer); } - /* Iterate all posting tree leaves using rightlinks and vacuum them */ - while (true) + return InvalidBuffer; +} + +/* + * Vacuum a posting tree: remove deletable TIDs from its leaf pages, and + * delete leaf pages that become completely empty. + * + * The leaves are processed in a single left-to-right sweep by following + * rightlinks, holding exclusive locks on the current page and its right + * sibling at once (lock coupling). Keeping the left sibling locked while + * acquiring the right one both pins down the sibling link needed for the + * deletion and follows the left-to-right page locking order used everywhere + * else in GIN, so it cannot deadlock with concurrent insertions or searches. + * + * A page is deleted only if, in addition to being empty, no other backend + * holds a pin on it (IsBufferCleanupOK), so nobody is about to insert into + * or read from it. If somebody does, we just leave the page in place; a + * future vacuum can delete it. + * + * Empty leaves are deleted right away, except for pages that must survive + * for the tree to stay navigable: the leftmost leaf (fullScan descents + * cannot recover from stepping onto a deleted page), the rightmost leaf, + * and leaves referenced by the last downlink of their parent (see + * ginLockLeafParent). Internal pages are never deleted; without high keys + * their empty siblings cannot be spliced out safely. Concurrent inserters + * and searchers that run into a page we deleted recover by moving right, + * as deleted pages keep their rightlink (see README). + */ +static void +ginVacuumPostingTree(GinVacuumState *gvs, BlockNumber rootBlkno) +{ + BlockNumber parentBlkno; + Buffer prevBuffer; + Page prevPage; + MemoryContext oldCxt; + + prevBuffer = ginStepToLeftmostLeaf(gvs, rootBlkno, &parentBlkno); + prevPage = BufferGetPage(prevBuffer); + + /* The leftmost leaf is never deleted, just vacuum its tuples */ + oldCxt = MemoryContextSwitchTo(gvs->tmpCxt); + ginVacuumPostingTreeLeaf(gvs->index, prevBuffer, gvs); + MemoryContextSwitchTo(oldCxt); + MemoryContextReset(gvs->tmpCxt); + + while (!GinPageRightMost(prevPage)) { - oldCxt = MemoryContextSwitchTo(gvs->tmpCxt); - ginVacuumPostingTreeLeaf(gvs->index, buffer, gvs); - MemoryContextSwitchTo(oldCxt); - MemoryContextReset(gvs->tmpCxt); + BlockNumber prevBlkno = BufferGetBlockNumber(prevBuffer); + BlockNumber blkno; + Buffer buffer; + Page page; - if (GinDataLeafPageIsEmpty(page)) - hasVoidPage = true; + /* + * Come up for air: release all locks and pins before the delay + * point, both to let it process interrupts (which is impossible + * while a buffer lock is held) and to avoid blocking readers of + * this page while cost-based vacuum delay makes us sleep. + * + * Only vacuum deletes posting tree pages, and there is at most one + * vacuum per index, so the page is still there when we re-lock it. + * If it was concurrently split, the pages that were split off to + * the right contain only tuples that we have already vacuumed, so + * it is fine to continue from the page's new rightlink. + */ + UnlockReleaseBuffer(prevBuffer); - blkno = GinPageGetOpaque(page)->rightlink; + vacuum_delay_point(false); - UnlockReleaseBuffer(buffer); + prevBuffer = ReadBufferExtended(gvs->index, MAIN_FORKNUM, prevBlkno, + RBM_NORMAL, gvs->strategy); + LockBuffer(prevBuffer, GIN_EXCLUSIVE); + prevPage = BufferGetPage(prevBuffer); - if (blkno == InvalidBlockNumber) + if (GinPageRightMost(prevPage)) break; + blkno = GinPageGetOpaque(prevPage)->rightlink; + /* + * Lock the right sibling before releasing the current page, so that + * we can delete it if it turns out to be empty. + */ buffer = ReadBufferExtended(gvs->index, MAIN_FORKNUM, blkno, RBM_NORMAL, gvs->strategy); LockBuffer(buffer, GIN_EXCLUSIVE); page = BufferGetPage(buffer); - } - return hasVoidPage; -} - -static void -ginVacuumPostingTree(GinVacuumState *gvs, BlockNumber rootBlkno) -{ - if (ginVacuumPostingTreeLeaves(gvs, rootBlkno)) - { - /* - * There is at least one empty page. So we have to rescan the tree - * deleting empty pages. - */ - Buffer buffer; - DataPageDeleteStack root, - *ptr, - *tmp; - bool deleted PG_USED_FOR_ASSERTS_ONLY; + Assert(GinPageIsData(page)); + Assert(GinPageIsLeaf(page)); + Assert(!GinPageIsDeleted(page)); - buffer = ReadBufferExtended(gvs->index, MAIN_FORKNUM, rootBlkno, - RBM_NORMAL, gvs->strategy); + oldCxt = MemoryContextSwitchTo(gvs->tmpCxt); + ginVacuumPostingTreeLeaf(gvs->index, buffer, gvs); + MemoryContextSwitchTo(oldCxt); + MemoryContextReset(gvs->tmpCxt); /* - * Lock posting tree root for cleanup to ensure there are no - * concurrent inserts. + * A page with an incomplete split must stay: its right sibling has + * no downlink yet, so deleting this page (and thereby its downlink) + * would leave the right sibling unreachable through the parent + * until the split is finished. */ - LockBufferForCleanup(buffer); - - memset(&root, 0, sizeof(DataPageDeleteStack)); - root.buffer = buffer; - root.leftBuffer = InvalidBuffer; - root.myoff = InvalidOffsetNumber; - root.isRoot = true; - - deleted = ginScanPostingTreeToDelete(gvs, &root); - Assert(!deleted); + if (GinDataLeafPageIsEmpty(page) && + !GinPageRightMost(page) && + !GinPageIsIncompleteSplit(page) && + BlockNumberIsValid(parentBlkno) && + IsBufferCleanupOK(buffer)) + { + OffsetNumber off; + Buffer parentBuffer; - ptr = root.child; + parentBuffer = ginLockLeafParent(gvs, &parentBlkno, blkno, &off); + if (BufferIsValid(parentBuffer)) + { + ginDeletePostingPage(gvs, buffer, prevBuffer, parentBuffer, + off); + UnlockReleaseBuffer(parentBuffer); + UnlockReleaseBuffer(buffer); - while (ptr) - { - tmp = ptr->child; - pfree(ptr); - ptr = tmp; + /* + * The current page's rightlink now points past the deleted + * page; continue the sweep from it. + */ + continue; + } } - UnlockReleaseBuffer(buffer); + /* Advance: the right sibling becomes the new current page */ + UnlockReleaseBuffer(prevBuffer); + prevBuffer = buffer; + prevPage = page; } + + UnlockReleaseBuffer(prevBuffer); } /* -- 2.50.1 (Apple Git-155)