|From:||Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com>|
|To:||Haribabu Kommi <kommi(dot)haribabu(at)gmail(dot)com>|
|Cc:||Kohei KaiGai <kaigai(at)kaigai(dot)gr(dot)jp>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, PgHacker <pgsql-hackers(at)postgresql(dot)org>, Robert Haas <robertmhaas(at)gmail(dot)com>|
|Subject:||Re: contrib/cache_scan (Re: What's needed for cache-only table scan?)|
|Views:||Raw Message | Whole Thread | Download mbox | Resend email|
The attached patches are revised ones according to the latest custom-plan
interface patch (v11).
The cache-scan module was re-implemented on the newer interface, and also
I noticed the extension does not handle the tuples being redirected correctly,
So, I revised the logic in ccache_vacuum_page() totally. It now becomes to
synchronize the cached tuples per page, not per tuple, basic and tries to
merge t-tree chunks per page basis also.
Also, I split the patches again because *demonstration* part is much larger
than the patches to the core backend. It will help reviewing.
-> It adds a hook for each page being vacuumed; that needs to synchronize
the status of in-memory cache managed by extension.
-> It allows to run HeapTupleSatisfiesVisibility() towards the tuples
on the in-memory cache, not on the heap.
-> It demonstrates the usage of above two patches. It allows to scan
a relation without storage access if possible.
NEC OSS Promotion Center / PG-Strom Project
KaiGai Kohei <kaigai(at)ak(dot)jp(dot)nec(dot)com>
> -----Original Message-----
> From: pgsql-hackers-owner(at)postgresql(dot)org
> [mailto:pgsql-hackers-owner(at)postgresql(dot)org] On Behalf Of Haribabu Kommi
> Sent: Wednesday, March 12, 2014 3:43 PM
> To: Kaigai Kouhei(海外 浩平)
> Cc: Kohei KaiGai; Tom Lane; PgHacker; Robert Haas
> Subject: Re: contrib/cache_scan (Re: [HACKERS] What's needed for cache-only
> table scan?)
> On Wed, Mar 12, 2014 at 5:26 PM, Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com> wrote:
> > Thanks for your efforts!
> >> Head patched
> >> Diff
> >> Select - 500K 772ms 2659ms -200%
> >> Insert - 400K 3429ms 1948ms 43% (I am
> >> not sure how it improved in this case)
> >> delete - 200K 2066ms 3978ms -92%
> >> update - 200K 3915ms 5899ms -50%
> >> This patch shown how the custom scan can be used very well but coming
> >> to patch as It is having some performance problem which needs to be
> >> investigated.
> >> I attached the test script file used for the performance test.
> > First of all, it seems to me your test case has too small data set
> > that allows to hold all the data in memory - briefly 500K of 200bytes
> > record will consume about 100MB. Your configuration allocates 512MB of
> > shared_buffer, and about 3GB of OS-level page cache is available.
> > (Note that Linux uses free memory as disk cache adaptively.)
> Thanks for the information and a small correction. The Total number of
> records are 5 million.
> The select operation is selecting 500K records. The total table size is
> around 1GB.
> Once I get your new patch re-based on the custom scan patch, I will test
> the performance again by increasing my database size more than the RAM size.
> And also I will make sure that memory available for disk cache is less.
> Hari Babu
> Fujitsu Australia
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org) To make
> changes to your subscription:
|Next Message||Robert Haas||2014-03-17 01:20:14||Re: [RFC] What should we do for reliable WAL archiving?|
|Previous Message||Kouhei Kaigai||2014-03-17 00:29:29||Re: Custom Scan APIs (Re: Custom Plan node)|