Re: GiST VACUUM

From: Andrey Borodin <x4mmm(at)yandex-team(dot)ru>
To: Heikki Linnakangas <hlinnaka(at)iki(dot)fi>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Thomas Munro <thomas(dot)munro(at)enterprisedb(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>, Костя Кузнецов <chapaev28(at)ya(dot)ru>
Subject: Re: GiST VACUUM
Date: 2018-07-18 18:27:48
Message-ID: ADA3A0B7-A209-48E5-8942-BA913C3FD31D@yandex-team.ru
Views: Raw Message | Whole Thread | Download mbox
Thread:
Lists: pgsql-hackers

Hi!

> 18 июля 2018 г., в 16:02, Heikki Linnakangas <hlinnaka(at)iki(dot)fi> написал(а):
>
> In the corresponding B-tree code, we use don't do actual recursion, but a hand-optimized "tail recursion", to avoid stack overflow if there are a lot of splits. I think we need to do something like tha there, too. I don't think it's safe to assume that we have enough stack space for the recursion. You have a check_stack_depth() in the function, so you'll get ERROR, but it sucks if VACUUM errors out because of that.
Ok, will do that.

>
>
> It's not cool to use up to 'maintenance_work_mem' of memory for holding the in-memory graph. VACUUM already uses up to that much memory to hold the list of dead TIDs, so we would use up to 2x maintenance_work_mem in total.
>
> The only reason we still need the logical scan is to support page deletion, when there is not enough memory available. Can we get rid of that altogether? I think I'd prefer some other scheme to deal with that situation. For example, we could make note, in memory, of the empty pages during the physical scan, and perform a second physical scan of the index to find their downlinks. Two full-index scans is not nice, but it's actually not that bad if it's done in physical order.
I think this is a good idea. I will implement this.

> And you could have some heuristics, e.g. only do it if more than 10% of the pages were deletable.
>
> Sorry to ask you to rewrite this again
Piece of cake :)

> , but I think it would be better to split this into two patches as follows:
>
> 1st patch: Scan the index in physical rather than logical order. No attempt at deleting empty pages yet.
>
> 2nd patch: Add support for deleting empty pages.
>
> I would be more comfortable reviewing and committing that first patch, which just switches to doing physical-order scan, first.

This seems very unproportional division of complexity. First patch (PFA) is very simple. All work is done in one cycle, without memorizing anything. Actually, you do not even need to rescan rightlinks: there may be no splits to the left when no pages are deleted.
If you think it is proper way to go - OK, I'll prepare better version of attached diff (by omitting tail recursion and adding more comments).

Best regards, Andrey Borodin.

Attachment Content-Type Size
gist_phisical_vacuum_v1.diff application/octet-stream 7.6 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Alvaro Herrera 2018-07-18 18:30:53 Re: Possible bug in logical replication.
Previous Message Tom Lane 2018-07-18 18:05:17 Re: ENOSPC FailedAssertion("!(RefCountErrors == 0)"