Re: posix_fadvise() and pg_receivexlog

Hi,

2014-08-07 13:47 GMT+09:00 Fujii Masao <masao(dot)fujii(at)gmail(dot)com>:

> On Thu, Aug 7, 2014 at 3:59 AM, Heikki Linnakangas
> <hlinnakangas(at)vmware(dot)com> wrote:
> > On 08/06/2014 08:39 PM, Fujii Masao wrote:
> >> The WAL files that pg_receivexlog writes will not be re-read soon
> >> basically,
> >> so we can advise the OS to release any cached pages when WAL file is
> >> closed. I feel inclined to change pg_receivexlog that way. Thought?
> >
> >
> > -1. The OS should be smart enough to not thrash the cache by files that
> are
> > written sequentially and never read.
>
OS's buffer strategy is optimized for general situation. Do you forget OS
hackers discussion last a half of year?

> Yep, the OS should be so smart, but I'm not sure if it actually is. Maybe
> not,
> so I was thinking that posix_fadvise is called when the server closes WAL
> file.

That's right.

> > If we go down this path, we'd need to
> > sprinkle posix_fadvises into many, many places.
>
Why do you aim to be perfect at the beginning?
It is as same as history of postgres, your concern doesn't make sense.

> > Anyway, who are we to say that they won't be re-read soon? You might e.g
> > have a secondary backup site where you copy the files received by
> > pg_receivexlog, as soon as they're completed.
>
> So whether posix_fadvise is called or not needs to be exposed as an
> user-configurable option. We would need to measure how useful exposing
> that is, though.

By the way, does pg_receivexlog process have fsync() in every WAL commit?
If yes, I think that we need no or less fsync() option for the better
performance. It is general in NOSQL storages.
If no, we need fsync() option for more getting reliability and data
integrarity.

Best regards,
--
Mitsumasa KONDO