PostgreSQL Soft RAID 1

From: Pailloncy Jean-Gerard <jg(at)rilk(dot)com>
To: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: PostgreSQL Soft RAID 1
Date: 2005-02-23 19:31:25
Message-ID: 8dd357ba2f3899540310ad3176cef83c@rilk.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

I had this idea, after reading some of the code about storage manager.
There was in the past, other software manager in very old version of
PostgreSQL.
I do not want to bring to life the old system, but I get one idea about
to speed things with special soft raid 1 in PostgreSQL.

For simplicity, I did not consider the caching of the RAID controller
or of the disk.

I consider the case where the database is larger than the RAM.
When there is many queries runing, the main bootleneck is the disk.
A modern drive reading fully randomly: the drive may sustain few (X)
megabytes/second of data transfer. A seq reading: few (Y) hundred
megabytes/second.
With a hardware RAID1 I should able to get twice (2X) the speed for
random read.

The query may do "random read", "seq read" or "write" operations.

If the RAID1 was done by PostgreSQL, the random queries could be
redirect to the first drive (speed X) and the seq queries partially
serialized to the second drive (speed Y >> X).
In the same way when bgwriter works, it mays write sequentially to one
drive and then to the other.

A hardware RAID1 when there is a mix a random and seq scan, just do
random (speed = 2X).
Intelligent read in soft RAID may achieve the speed Y+X >> 2X MB/s.

I said partially serialized seq queries because if there are many seq
scan on the same drive, the drive will do random. So we need to
partially serialized operations so that big chunks of data are read
together.

There is many other idea readv/writev, DIRECT_IO, Scatter/Gather I/O,
too.

Cordialement,
Jean-Gérard Pailloncy

Browse pgsql-hackers by date

  From Date Subject
Next Message Bjoern Metzdorf 2005-02-23 21:10:52 invalid multibyte character for locale
Previous Message Michael Fuhr 2005-02-23 17:04:06 Re: problem with function loading