Re: Index use during Hot Standby

On Mon, 2008-10-20 at 18:24 +0400, Teodor Sigaev wrote:
> > 3. Implement an extra indexAM API call that allows indexAM to decide
> > when/if index is valid during recovery. This would also cover the second
> > concern neatly in a single API call.
> >
> > wait until after deadline to implement (2) or (3), in case somebody
> > fixes this up in the next few weeks.
> >
>
> IMHO, Without locking of pages in recovery mode Btree and GIN are not usable
> while incomplete split exists - there is a nonconnected branch in tree.

I think the two things are not necessarily related to each other.

Incomplete splits prevent us from using a checkpoint as a valid
restartpoint from recovery. So that means an archive recovery always
starts from a point where we have all the WAL information required to
complete any incomplete splits.

That has nothing to do with locking requirements for concurrency. Now
you may be right, that we do not have the correct locking for
concurrency in the splitting algorithms. But I haven't seen any issue
yet myself. But I will look again. If you could be more specific that
would help me.

What I would say is that we need not consider the whole index to be
unusable at a point in time. If we emulate the lock sequence in recovery
that was used on the master, then we should have the same concurrency.
So preventing access to the whole index should be unnecessary.

Thinking some more on this, the database is not in a usable state until
all splits which were incomplete at time recovery started have been
completed. This is because the first half of the split is on disk
already, but we haven't reacquired the locks we held while making the
split in the first place. So we do have at least one problem in this
area.

> GiST has similar issue - incomplete insert. One insertion in leaf page can
> produce updating of keys up to the root. During that split pages may occurs.
> So, it's needed to add to gistxlog.c tracking of pages split to get exact
> knowledge about moments of unusability of index.
>
> One more thing about GiST - when database is switched from recovery mode to the
> normal mode then it's needed to complete insertion in GiST and, possibly, vacuum
> index. Dig around GistBulkDeleteResult->needFullVacuum and gistContinueInsert()

OK, will look at those. (Problems++).

Thanks,

--
Simon Riggs www.2ndQuadrant.com
PostgreSQL Training, Services and Support