Re: Patch: Write Amplification Reduction Method (WARM)

From: Pavan Deolasee <pavan(dot)deolasee(at)gmail(dot)com>
To: Jaime Casanova <jaime(dot)casanova(at)2ndquadrant(dot)com>
Cc: Haribabu Kommi <kommi(dot)haribabu(at)gmail(dot)com>, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Patch: Write Amplification Reduction Method (WARM)
Date: 2017-01-03 17:45:47
Message-ID: CABOikdMzPC2=0G0wZv8P=zQzHW7z6Xm+jsKc_-bAuqrvXjbiLw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox
Thread:
Lists: pgsql-hackers

On Tue, Jan 3, 2017 at 9:43 AM, Pavan Deolasee <pavan(dot)deolasee(at)gmail(dot)com>
wrote:

> The patch still disables WARM on system tables, something I would like to
> fix. But I've been delaying that because it will require changes at several
> places since indexes on system tables are managed separately.
>

Here is another version which fixes a bug that I discovered while adding
support for system tables. The patch set now also includes a patch to
enable WARM on system tables. I'm attaching that as a separate patch
because while the changes to support WARM on system tables are many, almost
all of them are purely mechanical. We need to pass additional information
to CatalogUpdateIndexes()/CatalogIndexInsert(). We need to tell these
routines whether the update leading to them was a WARM update and which
columns were modified so that it can correctly avoid adding new index
tuples for indexes for which index keys haven't changed.

I wish I could find another way of passing this information instead of
making changes at so many places, but the only other way I could think of
was tracking that information as part of the HeapTuple itself, which
doesn't seem nice and may also require changes at many call sites where
tuples are constructed. One minor improvement could be that instead of two,
we could just pass "modified_attrs" and a NULL value may imply non-WARM
update. Other suggestions are welcome though.

I'm quite happy that all tests pass even after adding support for system
tables. One reason for testing support for system tables was to ensure some
more code paths get exercised. As before, I've included Alvaro's
refactoring patch too.

Thanks,
Pavan

--
Pavan Deolasee http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

Attachment Content-Type Size
0001_track_root_lp_v8.patch application/octet-stream 27.4 KB
0002_warm_updates_v8.patch application/octet-stream 102.7 KB
0003_warm_fixes_v6.patch application/octet-stream 3.7 KB
interesting-attrs-2.patch application/octet-stream 11.7 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Magnus Hagander 2017-01-03 17:46:32 Re: pgsql: Update copyright for 2017
Previous Message Jim Nasby 2017-01-03 17:41:24 Re: merging some features from plpgsql2 project