Re: optimizing vacuum truncation scans

On Thu, Jul 9, 2015 at 4:37 PM, Jeff Janes <jeff(dot)janes(at)gmail(dot)com> wrote:
> On Wed, Jul 8, 2015 at 9:46 PM, Haribabu Kommi <kommi(dot)haribabu(at)gmail(dot)com>
> wrote:
>>
>> On Mon, Jun 29, 2015 at 3:54 PM, Jeff Janes <jeff(dot)janes(at)gmail(dot)com> wrote:
>> >
>> > There is still the case of pages which had their visibility bit set by a
>> > prior vacuum and then were not inspected by the current one. Once the
>> > truncation scan runs into these pages, it falls back to the previous
>> > behavior of reading block by block backwards. So there could still be
>> > reason to optimize that fallback using forward-reading prefetch.
>>
>> The case, I didn't understand is that, how the current vacuum misses
>> the page which
>> was set by the prior vacuum?
>
>
> The prior vacuum set them to all visible, but then doesn't delete them.
> Either because it got interrupted or because there were still some pages
> after them (at the time) that were not all empty.
>
> The current vacuum skips them entirely on the forward scan because it thinks
> it is waste of time to vacuum all visible pages. Since it skips them, it
> doesn't know if they are empty as well as being all-visible. There is no
> permanent indicator of the pages being all-empty, there is only the
> inference based on the current vacuum's counters and protected by the lock
> held on the table.
>
>>
>>
>> The page should be counted either in skipped_pages or in
>> nonempty_pages. Is it possible
>> that a page doesn't comes under these two categories and it is not empty
>> also?
>>
>> If the above doesn't exist, then we can directly truncate the relation
>> from the highest block
>> number of either nonempty_pages or skipped_pages to till the end of
>> the relation.
>
>
> Right, and that is what this does (provided the vm bit is still set, so it
> does still have to loop over the vm to verify that it is still set, while it
> holds the access exclusive lock).

Thanks I got it. To re-verify the vm bit of a page after getting the
access exclusive lock,
we have to do the backward scan.

I also feel that your prefetch buffer patch needed to improve the
performance, as because
there is a need of backward scan to re-verify vm bit,

I will do some performance tests and send you the results.

> The problem is that the pages between the two counters are not known to be
> empty, and also not known to be nonempty. Someone has to be willing to go
> check on those pages at some point, or else they will never be eligible for
> truncation.

Yes, there is no way to identify the page is empty or not without
reading the page.

Regards,
Hari Babu
Fujitsu Australia