Re: Patch: Write Amplification Reduction Method (WARM)

From: Pavan Deolasee <pavan(dot)deolasee(at)gmail(dot)com>
To: Peter Geoghegan <pg(at)bowt(dot)ie>
Cc: Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Bruce Momjian <bruce(at)momjian(dot)us>, Jaime Casanova <jaime(dot)casanova(at)2ndquadrant(dot)com>, Haribabu Kommi <kommi(dot)haribabu(at)gmail(dot)com>, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Patch: Write Amplification Reduction Method (WARM)
Date: 2017-03-19 07:15:50
Message-ID: CABOikdNV253GMj3CwQEjrHiYWVJ-m9uLO8SqSSxBDJQKw9RMeA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox
Thread:
Lists: pgsql-hackers

On Tue, Mar 14, 2017 at 8:14 PM, Peter Geoghegan <pg(at)bowt(dot)ie> wrote:

> On Tue, Mar 14, 2017 at 12:19 PM, Alvaro Herrera
> <alvherre(at)2ndquadrant(dot)com> wrote:
> > Impressive results.
>
> Agreed.
>

Thanks. I repeated the same tests with slightly lower scale factor so that
most (but not all) data fits in memory. The results are kinda similar
(attached here). The spikes are still there and they correspond to the
checkpoint_timeout set for these tests.

> It seems like an important invariant for WARM is that any duplicate
> index values ought to have different TIDs (actually, it's a bit
> stricter than that, since btrecheck() cares about simple binary
> equality).

Yes. I think in the current code, indexes can never duplicate TIDs (at
least for btrees and hash). With WARM, indexes can have duplicate TIDs, but
iff index values differ. In addition there can only be one more duplicate
and one of them must be a Blue pointer (or a non-WARM pointer if we accept
the new nomenclature proposed a few mins back).

>
> You wouldn't have to teach amcheck about the heap, because a TID that
> points to the heap can only be duplicated within a B-Tree index
> because of WARM. So, if we find that two adjacent tuples are equal,
> check if the TIDs are equal. If they are also equal, check for strict
> binary equality. If strict binary equality is indicated, throw an
> error due to invariant failing.
>
>
Wouldn't this be much more expensive for non-unique indexes?

Thanks,
Pavan

--
Pavan Deolasee http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

Attachment Content-Type Size
Moderate_AV_4Indexes_100FF_SF800_Duration28800s.pdf application/pdf 248.7 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Ashutosh Sharma 2017-03-19 08:56:07 Re: ANALYZE command progress checker
Previous Message Pavan Deolasee 2017-03-19 07:05:10 Re: Patch: Write Amplification Reduction Method (WARM)