Improve configurability for IO related behavoir

Dear developers:

I have some suggestions on PostgreSQL's IO behavior. I am not sure my opinions are right. Many thanks for your time!

It is documented that :
wal_sync_method​ (enum): The open_*​ options also use O_DIRECT if available.
Shall PostgreSQL consider making O_DIRECT configurable? In MySQL, innodb_flush_method​ can change if use O_DIRECT or not.
I made some benchmarking on O_DIRECT of MySQL, and I find that sometimes, using O_DIRECT may hurt performance.
https://ibb.co/BVWnwnh
[https://i.ibb.co/1fyzbzC/003.png]<https://ibb.co/BVWnwnh>
003<https://ibb.co/BVWnwnh>
Image 003 hosted in ImgBB
ibb.co






Comparing the blue (buffered IO) and green (direct IO) lines, I find buffered IO is faster. So I think having such parameter in PostgreSQL
is reasonable.

On the other hand, wal_sync_method​ only controls how WAL is written to devices, while for data file I notice that PostgreSQL uses
sync_file_range(2)​. So shall we also make it configurable? I also find that in some systems, open​ with O_SYNC is much more faster
than write​ + fsync​ :

https://ibb.co/f1VsCC1

[https://i.ibb.co/gjbxDDj/Picture1.png]<https://ibb.co/f1VsCC1>
Picture1<https://ibb.co/f1VsCC1>
Image Picture1 hosted in ImgBB
ibb.co
Furthermore, the results above are also related to IO API supported by OS. MySQL support synchronized IO and Linux libaio​. It seems
that PostgreSQL only supports synchronized IO, so shall we support more IO engines? like io_uring​ which is very popular in recent years.

Thanks & best,
Haochen