Skip site navigation (1) Skip section navigation (2)

Re: Performance Improvement by reducing WAL for Update Operation

From: Amit kapila <amit(dot)kapila(at)huawei(dot)com>
To: Simon Riggs <simon(at)2ndQuadrant(dot)com>
Cc: "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Performance Improvement by reducing WAL for Update Operation
Date: 2013-01-12 14:07:26
Message-ID: 6C0B27F7206C9E4CA54AE035729E9C383BEB016D@szxeml509-mbs (view raw, whole thread or download thread mbox)
Lists: pgsql-hackers
On Saturday, January 12, 2013 3:45 PM Simon Riggs wrote:
On 12 January 2013 03:50, Amit kapila <amit(dot)kapila(at)huawei(dot)com> wrote:
> On Saturday, January 12, 2013 12:23 AM Simon Riggs wrote:
> On 11 January 2013 18:11, Amit kapila <amit(dot)kapila(at)huawei(dot)com> wrote:
>>>>> Can we identify which columns have changed? i.e. 1st, 3rd and 12th columns?
>>>>   As per current algorithm, we can't as it is based on offsets.
>>>>   What I mean to say is that the basic idea to reconstruct tuple during recovery
>>>>   is copy data from old tuple offset-wise (offsets stored in encoded tuple) and use new data (modified column data)
>>>>   from encoded tuple directly. So we don't need exact column numbers.
>>> Another patch is going through next CF related to reassembling changes
>>> from WAL records.
>>> To do that efficiently, we would want to store a bitmap showing which
>>> columns had changed in each update. Would that be an easy addition, or
>>> is that blocked by some aspect of the current design?
>>   I don't think it should be a problem, as it can go in current way of WAL tuple construction as
>>   we do in this patch when old and new buf are different. This differentiation is done in
>>   log_heap_update.
>>   IMO, for now we can avoid this optimization (way we have done incase updated tuple is not on same page)
>>   for the bitmap storing patch and later we can evaluate if we can do this optimization for
>>   the feature of that patch.

> Yes, we can simply disable this feature. But that is just bad planning
> and we should give some thought to having new features play nicely
> together.

> I would like to work out how to modify this so it can work with wal
> decoding enabled. I know we can do this, I want to look at how,
> because we know we're going to do it.

  I am sure this can be done, as for WAL decoding we mainly new values and column numbers
  So if we include bitmap in WAL tuple and teach WAL decoding method how to decode this new format WAL tuple
  it can be done.
  However it will need changes in algorithm for both the patches and it can be risk for one or for both patches.
  I am open to have discussion about how both can work together, but IMHO at this moment (as this will be last CF)
  it will be little risky.
  If there is some way such that with minor modifications, we can address this scenario, I will be happy to see both
  working together.

With Regards,
Amit Kapila.

In response to

pgsql-hackers by date

Next:From: Amit kapilaDate: 2013-01-12 14:10:45
Subject: Re: Performance Improvement by reducing WAL for Update Operation
Previous:From: Jan UrbaƄskiDate: 2013-01-12 11:09:01
Subject: fix SQL example syntax in file comment

Privacy Policy | About PostgreSQL
Copyright © 1996-2018 The PostgreSQL Global Development Group