Re: Introducing a new linux readahead framework

From: Wu Fengguang <wfg(at)mail(dot)ustc(dot)edu(dot)cn>
To: "Jim C(dot) Nasby" <jnasby(at)pervasive(dot)com>
Cc: pgsql-performance(at)postgresql(dot)org
Subject: Re: Introducing a new linux readahead framework
Date: 2006-04-21 07:15:11
Message-ID: 20060421071511.GA4885@mail.ustc.edu.cn
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

On Thu, Apr 20, 2006 at 11:31:47PM -0500, Jim C. Nasby wrote:
> > In adaptive readahead, the context based method may be of particular
> > interest to postgresql users. It works by peeking into the file cache
> > and check if there are any history pages present or accessed. In this
> > way it can detect almost all forms of sequential / semi-sequential read
> > patterns, e.g.
> > - parallel / interleaved sequential scans on one file
> > - sequential reads across file open/close
> > - mixed sequential / random accesses
> > - sparse / skimming sequential read
> >
> > It also have methods to detect some less common cases:
> > - reading backward
> > - seeking all over reading N pages
>
> Are there any ways to inform the kernel that you either are or aren't
> doing a sequential read? It seems that in some cases it would be better

This call will disable readahead totally for fd:
posix_fadvise(fd, any, any, POSIX_FADV_RANDOM);

This one will reenable it:
posix_fadvise(fd, any, any, POSIX_FADV_NORMAL);

This one will enable readahead _and_ set max readahead window to
2*max_readahead_kb:
posix_fadvise(fd, any, any, POSIX_FADV_SEQUENTIAL);

> to bypass a bunch of tricky logic trying to determine that it's doing a
> sequential read. A sequential scan in PostgreSQL would be such a case.

You do not need to worry about the detecting `overhead' on sequential
scans :) The adaptive readahead framework has a fast code path(the
stateful method) to handle normal sequential reads, the detection of
which is really trivial.

> The opposite example would be an index scan of a highly uncorrelated
> index, which would produce mostly random reads from the table. In that
> case, reading ahead probably makes very little sense, though your logic
> might have a better idea of the access pattern than PostgreSQL does.

As for the index scans, the failsafe code path(i.e. the context based
one) will normally be used, and it does have a little overhead in
looking up the page cache(about 0.4% more CPU time). However, the
penalty of random disk access is so large that if ever it helps
reducing a small fraction of disk accesses, you wins.

Thanks,
Wu

In response to

Responses

Browse pgsql-performance by date

  From Date Subject
Next Message Markus Schaber 2006-04-21 07:53:34 Re: Introducing a new linux readahead framework
Previous Message Milen Kulev 2006-04-21 05:22:24 Re: Quick Performance Poll