Re: [HACKERS] TODO item

* Tatsuo Ishii <t-ishii(at)sra(dot)co(dot)jp> [000209 07:32] wrote:
> > > It seems that sync(2) blocks until data is written. So it would be ok
> > > at least with Linux. I'm not sure about other platforms, though.
> >
> > It is incorrect to assume that sync() wait until all buffers are
> > flushed on any other platform than Linux, I didn't think
> > that Linux even did so but the kernel sources say yes.
>
> Right. I have looked at Linux kernel sources and confirmed it.
>
> > Solaris doesn't do this and niether does FreeBSD/NetBSD.
>
> I'm not sure about Solaris since I don't have an access to its source
> codes. Will look at FreeBSD kernel sources.
>
> > I guess if you wanted to implement this for linux only then it would
> > work, you ought to then also warn people that a non-dedicated db server
> > could experiance different performance using this code.
>
> I just want to have more choices other than with/without -F. With -F
> looses ACID, without it implies per-page-fsync. Both choices are
> painful. But switching to expensive commercial DBMSs is much more
> painful at least for me.
>
> Even if it would be usefull on Linux only and in a certain situation,
> it would better than nothing IMHO (until WAL comes up).

Ok, here's a nifty idea, a slave process called pgsyncer.

At the end of a transaction a backend asks the syncer to fsync all files.

Now here's the cool part, this avoids the non-portability of the Linux
sync() problem and at the same time restricts the syncing to postgresql
and reduces 'cross-fsync' issues.

Imagine:

postgresql has 3 files open (a, b, c), so will the syncer.
backend 1 completes a request, communicates to the syncer that a flush
is needed.
syncer starts by fsync'ing 'a'
backend 2 completes a request, communicates to the syncer
syncer continues with 'b' then 'c'
syncer responds to backend 1 that it's safe to proceed.
syncer fsyncs 'a' again
syncer responds to backend 2 that it's all completed.

effectively the fsync of 'b' and 'c' have been batched.

It's just an elevator algorithm, perhaps this can be done without
a seperate slave process?

-Alfred