Re: Allow a per-tablespace effective_io_concurrency setting

On Thu, Sep 3, 2015 at 2:13 PM, Merlin Moncure <mmoncure(at)gmail(dot)com> wrote:
> I find this talk of platters and spindles to be somewhat
> baroque; for a 200$ part I have to work pretty hard to max out the
> drive when reading and I'm still not completely sure if it's the drive
> itself, postgres, cpu, or sata interface bottlenecking me. This will
> require a rethink of e_i_o configuration; in the old days there were
> physical limitations of the drives that were in the way regardless of
> the software stack but we are in a new era, I think. I'm convinced
> prefetching works and we're going to want to aggressively prefetch
> anything and everything possible. SSD controllers (at least the intel
> ones) are very smart.

Wouldn't SSDs need much *less* aggressive prefetching? There's still
latency and there are multiple I/O channels so they will still need
some. But spinning media gives latencies measured in milliseconds. You
can process a lot of tuples in milliseconds. If you have a hundred
spindles you want them all busy doing seeks because in the 5ms it
takes them to do that you can proess all the results on a single cpu
and the rest of time is spend waiting.

When your media has latency on the order of microseconds then you only
need to have a small handful of I/O requests in flight to keep your
processor busy.

--
greg