Re: Proposing pg_hibernate

On Sun, Jun 15, 2014 at 2:51 AM, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> On Thu, Jun 12, 2014 at 9:31 AM, Gurjeet Singh <gurjeet(at)singh(dot)im> wrote:
>>
>> I don't have intimate knowledge of recovery but I think the above
>> assessment of recovery's operations holds true. If you still think
>> this is a concern, can you please provide a bit firm example using
>> which I can visualize the problem you're talking about.
>
> Okay, let me try with an example:
>
> Assume No. of shared buffers = 5
> Before Crash:
>
> 1. Pages in shared buffers numbered 3, 4, 5 have write operations
> performed on them, followed by lot of reads which makes their
> usage_count as 5.
> 2. Write operation happened on pages in shared buffers numbered
> 1, 2. Usage_count of these buffers is 1.
> 3. Now one read operation needs some different pages and evict
> pages in shared buffers numbered 1 and 2 and read the required
> pages, so buffers 1 and 2 will have usage count as 1.
> 4. At this moment shutdown initiated.
> 5. Bgwriter saved just buffers 1 and 2 and crashed.
>
> After Crash:
>
> 1. Recovery will read in pages on which operations happened in
> step-1 and 2 above.
> 2. Buffer loader (pg_hibernator) will load buffers on which operations
> happened in step-3, so here it might needs to evict buffers which are
> corresponding to buffers of step-1 before crash. So what this
> essentially means is that pg_hibernator can lead to eviction of more
> useful pages.

Granted, you have demonstrated that the blocks restored by
pg_hibernator can cause eviction of loaded-by-recovery blocks. But,
one can argue that pg_hibernator brought the shared-buffer contents to
to a state that is much closer to the pre-shutdown state than the
recovery would have restored them to. I think this supports the case
for pg_hibernator, that is, it is doing what it is supposed to do:
restore shared-buffers to pre-shutdown state.

I agree that there's uncertainty as to which buffers will be cleared,
and hence which blocks will be evicted. So pg_hibernator may cause
eviction of blocks that had higher usage count before the shutdown,
because they may have a lower/same usage count as other blocks'
buffers after recovery. There's not much that can be done for that,
because usage count information is not saved anywhere on disk, and I
don't think it's worth saving just for pg_hibernator's sake.

Best regards,
--
Gurjeet Singh http://gurjeet.singh.im/

EDB : www.EnterpriseDB.com : The Enterprise PostgreSQL Company