Skip site navigation (1) Skip section navigation (2)

Re: ice-broker scan thread

From: "Qingqing Zhou" <zhouqq(at)cs(dot)toronto(dot)edu>
To: pgsql-hackers(at)postgresql(dot)org
Subject: Re: ice-broker scan thread
Date: 2005-11-29 05:55:36
Message-ID: dmgqf2$2e82$1@news.hub.org (view raw or flat)
Thread:
Lists: pgsql-hackers
"David Boreham" <david_list(at)boreham(dot)org> wrote
>>
> I don't think your NT overlapped I/O code is quite right. At least
> I think it will issue reads at a high rate without waiting for any of them
> to complete. Beyond some point that has to give the kernel gut-rot.
>

[also with reply to Gavin] look up dictionary for "gut-rot", got it ... Uh, 
this behavior is intended - I try to push enough requests shortly to kernel 
so that it understands that I am doing sequential scan, so it would pull the 
data from disk to file system cache more efficiently. Some file systems may 
have "free-behind" mechanism, but our main thread (who really process the 
query) should be fast enough before the data vanished.

>
> You could re-write your program to have a single thread but use aio.
> In that case it should show the same read ahead benefit that you see
> with the thread.
>

I guess this is also Gavin's point - I understand that will be two different 
methodologies to handle "read-ahead". If no other thread/process involved, 
then the main thread will be responsible to grab a free buffer page from 
bufferpool and ask the kernel to put the data there by sync IO (current 
PostgreSQL does) or async IOs. And that's what I want to avoid. I'd like to 
use a dedicated thread/process to "break the ice" only, i.e., pull data from 
disk to file system cache, so that the main thread will only issue *logical* 
read.

Regards,
Qingqing 



In response to

Responses

pgsql-hackers by date

Next:From: Zeugswetter Andreas DCP SDDate: 2005-11-29 08:49:57
Subject: Re: gprof SELECT COUNT(*) results
Previous:From: Gavin SherryDate: 2005-11-29 05:25:17
Subject: Re: ice-broker scan thread

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group