Re: Proposing pg_hibernate

On Wed, Jul 2, 2014 at 5:48 AM, Gurjeet Singh <gurjeet(at)singh(dot)im> wrote:
>
> Granted, you have demonstrated that the blocks restored by
> pg_hibernator can cause eviction of loaded-by-recovery blocks. But,
> one can argue that pg_hibernator brought the shared-buffer contents to
> to a state that is much closer to the pre-shutdown state than the
> recovery would have restored them to. I think this supports the case
> for pg_hibernator, that is, it is doing what it is supposed to do:
> restore shared-buffers to pre-shutdown state.
>
> I agree that there's uncertainty as to which buffers will be cleared,
> and hence which blocks will be evicted. So pg_hibernator may cause
> eviction of blocks that had higher usage count before the shutdown,
> because they may have a lower/same usage count as other blocks'
> buffers after recovery.

I think we should address this uncertainty, else it would be difficult to
claim whether running pg_hibernator will increase the performance of
applications or decrease it. As mentioned upthread there can be other
cases as well which can be affected, I think the idea proposed by Robert
(try to do this at cluster startup time, rather than once recovery has
reached consistency.) is a neat way to handle all such uncertainties.
Doing so can increase the recovery time as well, because without
pg_hibernator, there is a high chance that it always get free shared
buffers unless recovery ran for long time and with pg_hibernator
there can situations where recovery might need to evict buffers filled
be pg_hibernator.
Overall I agree that following Robert's idea will increase the time to
make database server up and reach a state where apps can connect
and start operations, but I think atleast with such an approach we can
claim that after warming buffers with pg_hibernator apps will always
have performance greater than equal to the case when there is no
pg_hibernator.

> There's not much that can be done for that,
> because usage count information is not saved anywhere on disk, and I
> don't think it's worth saving just for pg_hibernator's sake.

Thats right, but I think if pg_hibernator can save usage count along with
other info, then the claim that it can restore shared-buffers
to pre-shutdown
state has more value.

With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com