Re: ice-broker scan thread

Qingqing Zhou wrote:

>I am considering add an "ice-broker scan thread" to accelerate PostgreSQL
>sequential scan IO speed. The basic idea of this thread is just like the
>"read-ahead" method, but the difference is this one does not read the data
>into shared buffer pool directly, instead, it reads the data into file
>system cache, which makes the integration easy and this is unique to
>PostgreSQL.
>
>
Interesting, and I wondered about this too. But for my taste the
demonstrated benefit really
isn't large enough to make it worthwhile.
BTW, I heard a long time ago that NTFS has quite fancy read-ahead, where
it attempts
to detect the application's access pattern including if it is reading
sequentially and even
if there is a 'stride' to the accesses when they're not contiguous. I
would imagine that
other filesystems attempt similar tricks. So one might expect a simple
linear prefectch
to not help much in the presence of such a filesystem.

Were you worried about the icebreaker thread getting too far ahead of
the scan ?
If it did it might page out the data you're about to read, I think. Of
course this could
be fixed by having the read ahead thread perodically check the current
location being
read by the query thread and pausing if it's got too far ahead.

Anyway, the recent performance thread has been intersting to me because
in all my career
I've never seen a database that scanned scads of data from disk to
process a query.
Typically the problems I work on arrange to read the entire database
into memory.
I think I need to get out more... ;)