Re: [RFC] LSN Map

On Mon, Feb 23, 2015 at 12:52 PM, Heikki Linnakangas
<hlinnakangas(at)vmware(dot)com> wrote:
> Dunno, but Jim's got a point. This is a maintenance burden to all indexams,
> if they all have to remember to update the LSN map separately. It needs to
> be done in some common code, like in PageSetLSN or XLogInsert or something.
>
> Aside from that, isn't this horrible from a performance point of view? The
> patch doubles the buffer manager traffic, because any update to any page
> will also need to modify the LSN map. This code is copied from the
> visibility map code, but we got away with it there because the VM only needs
> to be updated the first time a page is modified. Subsequent updates will
> know the visibility bit is already cleared, and don't need to access the
> visibility map.
>
> Ans scalability: Whether you store one value for every N pages, or the LSN
> of every page, this is going to have a huge effect of focusing contention to
> the LSN pages. Currently, if ten backends operate on ten different heap
> pages, for example, they can run in parallel. There will be some contention
> on the WAL insertions (much less in 9.4 than before). But with this patch,
> they will all fight for the exclusive lock on the single LSN map page.
>
> You'll need to find a way to not update the LSN map on every update. For
> example, only update the LSN page on the first update after a checkpoint
> (although that would still have a big contention focusing effect right after
> a checkpoint).

I think it would make more sense to do this in the background.
Suppose there's a background process that reads the WAL and figures
out which buffers it touched, and then updates the LSN map
accordingly. Then the contention-focusing effect disappears, because
all of the updates to the LSN map are being made by the same process.
You need some way to make sure the WAL sticks around until you've
scanned it for changed blocks - but that is mighty close to what a
physical replication slot does, so it should be manageable.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company