Re: Patch: Write Amplification Reduction Method (WARM)

On Thu, Sep 1, 2016 at 9:44 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:

> On Thu, Sep 1, 2016 at 02:37:40PM +0530, Pavan Deolasee wrote:
> > I like the simplified approach, as long as it doesn't block further
> > improvements.
> >
> >
> >
> > Yes, the proposed approach is simple yet does not stop us from improving
> things
> > further. Moreover it has shown good performance characteristics and I
> believe
> > it's a good first step.
>
> Agreed. This is BIG. Do you think it can be done for PG 10?
>

I definitely think so. The patches as submitted are fully functional and
sufficient. Of course, there are XXX and TODOs that I hope to sort out
during the review process. There are also further tests needed to ensure
that the feature does not cause significant regression in the worst cases.
Again something I'm willing to do once I get some feedback on the broader
design and test cases. What I am looking at this stage is to know if I've
missed something important in terms of design or if there is some show
stopper that I overlooked.

Latest patches rebased with current master are attached. I also added a few
more comments to the code. I forgot to give a brief about the patches, so
including that as well.

0001_track_root_lp_v4.patch: This patch uses a free t_infomask2 bit to
track latest tuple in an update chain. The t_ctid.ip_posid is used to track
the root line pointer of the update chain. We do this only in the latest
tuple in the chain because most often that tuple will be updated and we
need to quickly find the root only during update.

0002_warm_updates_v4.patch: This patch implements the core of WARM logic.
During WARM update, we only insert new entries in the indexes whose key has
changed. But instead of indexing the real TID of the new tuple, we index
the root line pointer and then use additional recheck logic to ensure only
correct tuples are returned from such potentially broken HOT chains. Each
index AM must implement a amrecheck method to support WARM. The patch
currently implements this for hash and btree indexes.

Thanks,
Pavan

--
Pavan Deolasee http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services