RE: O_DIRECT for relations and SLRUs (Prototype)

From: Michael Paquier [mailto:michael(at)paquier(dot)xyz]
> One of the reasons why I have begun this thread is that since we have heard
> about the fsync issues on Linux, I think that there is room for giving our
> user base more control of their fate without relying on the Linux community
> decisions to potentially eat data and corrupt a cluster with a page dirty
> bit cleared without its data actually flushed. Even the latest kernels
> are not fixing all the patterns with open fds across processes, switching
> the problem from one corner of the table to another, and there are folks
> patching the Linux kernel to make Postgres more reliable from this
> perspective, and living happily with this option. As long as the option
> can be controlled and defaults to false, it seems to be that we could do
> something. Even if the performance is bad, this gives the user control
> of how he/she wants things to be done.

Thank you for starting an interesting topic. We probably want the direct I/O. On a INSERT and UPDATE heavy system with PostgreSQL 9.2, we suffered from occasional high response times due to the Linux page cache activity. Postgres processes competed for the page cache to read/write the data files, write online and archive WAL files, and write the server log files (auto_explain and autovacuum workers emitted a lot of logs.) The user with Oracle experience asked why PostgreSQL doesn't handle database I/O by itself...

And I wonder how useful the direct I/O for low latency devices like the persistent memory. The overhead of the page cache may become relatively higher.

Regards
Takayuki Tsunakawa