Re: contrib/cache_scan (Re: What's needed for cache-only table scan?)

From: Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com>
To: Haribabu Kommi <kommi(dot)haribabu(at)gmail(dot)com>
Cc: Kohei KaiGai <kaigai(at)kaigai(dot)gr(dot)jp>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, PgHacker <pgsql-hackers(at)postgresql(dot)org>, Robert Haas <robertmhaas(at)gmail(dot)com>
Subject: Re: contrib/cache_scan (Re: What's needed for cache-only table scan?)
Date: 2014-03-17 00:45:42
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers


The attached patches are revised ones according to the latest custom-plan
interface patch (v11).
The cache-scan module was re-implemented on the newer interface, and also
I noticed the extension does not handle the tuples being redirected correctly,
So, I revised the logic in ccache_vacuum_page() totally. It now becomes to
synchronize the cached tuples per page, not per tuple, basic and tries to
merge t-tree chunks per page basis also.

Also, I split the patches again because *demonstration* part is much larger
than the patches to the core backend. It will help reviewing.
* pgsql-v9.4-vacuum_page_hook.v11.patch
-> It adds a hook for each page being vacuumed; that needs to synchronize
the status of in-memory cache managed by extension.
* pgsql-v9.4-mvcc_allows_cache.v11.patch
-> It allows to run HeapTupleSatisfiesVisibility() towards the tuples
on the in-memory cache, not on the heap.
* pgsql-v9.4-example-cache_scan.v11.patch
-> It demonstrates the usage of above two patches. It allows to scan
a relation without storage access if possible.

NEC OSS Promotion Center / PG-Strom Project
KaiGai Kohei <kaigai(at)ak(dot)jp(dot)nec(dot)com>

> -----Original Message-----
> From: pgsql-hackers-owner(at)postgresql(dot)org
> [mailto:pgsql-hackers-owner(at)postgresql(dot)org] On Behalf Of Haribabu Kommi
> Sent: Wednesday, March 12, 2014 3:43 PM
> To: Kaigai Kouhei(海外 浩平)
> Cc: Kohei KaiGai; Tom Lane; PgHacker; Robert Haas
> Subject: Re: contrib/cache_scan (Re: [HACKERS] What's needed for cache-only
> table scan?)
> On Wed, Mar 12, 2014 at 5:26 PM, Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com> wrote:
> > Thanks for your efforts!
> >> Head patched
> >> Diff
> >> Select - 500K 772ms 2659ms -200%
> >> Insert - 400K 3429ms 1948ms 43% (I am
> >> not sure how it improved in this case)
> >> delete - 200K 2066ms 3978ms -92%
> >> update - 200K 3915ms 5899ms -50%
> >>
> >> This patch shown how the custom scan can be used very well but coming
> >> to patch as It is having some performance problem which needs to be
> >> investigated.
> >>
> >> I attached the test script file used for the performance test.
> >>
> > First of all, it seems to me your test case has too small data set
> > that allows to hold all the data in memory - briefly 500K of 200bytes
> > record will consume about 100MB. Your configuration allocates 512MB of
> > shared_buffer, and about 3GB of OS-level page cache is available.
> > (Note that Linux uses free memory as disk cache adaptively.)
> Thanks for the information and a small correction. The Total number of
> records are 5 million.
> The select operation is selecting 500K records. The total table size is
> around 1GB.
> Once I get your new patch re-based on the custom scan patch, I will test
> the performance again by increasing my database size more than the RAM size.
> And also I will make sure that memory available for disk cache is less.
> Regards,
> Hari Babu
> Fujitsu Australia
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org) To make
> changes to your subscription:

Attachment Content-Type Size
pgsql-v9.4-example-cache_scan.v11.patch application/octet-stream 94.7 KB
pgsql-v9.4-mvcc_allows_cache.v11.patch application/octet-stream 918 bytes
pgsql-v9.4-vacuum_page_hook.v11.patch application/octet-stream 1.8 KB

In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message Robert Haas 2014-03-17 01:20:14 Re: [RFC] What should we do for reliable WAL archiving?
Previous Message Kouhei Kaigai 2014-03-17 00:29:29 Re: Custom Scan APIs (Re: Custom Plan node)