Skip site navigation (1) Skip section navigation (2)

Re: ice-broker scan thread

From: David Boreham <david_list(at)boreham(dot)org>
To: pgsql-hackers(at)postgresql(dot)org
Subject: Re: ice-broker scan thread
Date: 2005-11-30 14:38:42
Message-ID: (view raw, whole thread or download thread mbox)
Lists: pgsql-hackers
Qingqing Zhou wrote:

>[also with reply to Gavin] look up dictionary for "gut-rot", got it ... Uh, 
>this behavior is intended - I try to push enough requests shortly to kernel 
>so that it understands that I am doing sequential scan, so it would pull the 
>data from disk to file system cache more efficiently. Some file systems may 
>have "free-behind" mechanism, but our main thread (who really process the 
>query) should be fast enough before the data vanished.
I guess I was concerned that very large numbers of concurrent operations 
on the same file handle
in flight at the same time might lead to poor performance or even 
instability. e.g. the kernel may
make long linked lists, it might create lock contention with itself, 
that kind of bad stuff. My thinking
being that the kernel wasn't designed with applications that fire off 
10,000 concurrent reads against
the same file.

>I guess this is also Gavin's point - I understand that will be two different 
>methodologies to handle "read-ahead". If no other thread/process involved, 
>then the main thread will be responsible to grab a free buffer page from 
>bufferpool and ask the kernel to put the data there by sync IO (current 
>PostgreSQL does) or async IOs. And that's what I want to avoid. I'd like to 
>use a dedicated thread/process to "break the ice" only, i.e., pull data from 
>disk to file system cache, so that the main thread will only issue *logical* 
Right, understood. My point was that a thread with sync I/O and the 
query thread with
async I/O are in fact logically identical. They're just two different 
implementation techniques
for the same fundemental functionality. In some cases the non-thread 
implementation might
be prefered (for example on a platform with no threads).

In response to

pgsql-hackers by date

Next:From: Joshua D. DrakeDate: 2005-11-30 15:42:46
Subject: Re: PL/php in pg_pltemplate
Previous:From: David BorehamDate: 2005-11-30 14:33:18
Subject: Re: ice-broker scan thread

Privacy Policy | About PostgreSQL
Copyright © 1996-2017 The PostgreSQL Global Development Group