Re: Yet another fast GiST build

On 21/09/2020 02:06, Tom Lane wrote:
> Justin Pryzby <pryzby(at)telsasoft(dot)com> writes:
>> This also appears to break checksums.

Thanks, I'll go fix it.

> I was wondering about that, because the typical pattern for use of
> smgrextend for indexes seems to be
>
> RelationOpenSmgr(rel);
> PageSetChecksumInplace(page, lastblock);
> smgrextend(rel->rd_smgr, MAIN_FORKNUM, lastblock, zerobuf.data, false);
>
> and gist_indexsortbuild wasn't doing either of the first two things.
>
> gist_indexsortbuild_flush_ready_pages looks like it might be
> a few bricks shy of a load too. But my local CLOBBER_CACHE_ALWAYS
> run hasn't gotten to anything except the pretty-trivial index
> made in point.sql, so I don't have evidence about it.

I don't think a relcache invalidation can happen on the index we're
building. Other similar callers call RelationOpenSmgr(rel) before every
write though (e.g. _bt_blwritepage()), so perhaps it's better to copy
that pattern here too.

> Another interesting point is that all the other index AMs seem to WAL-log
> the new page before the smgrextend call, whereas this code is doing it
> in the other order. I strongly doubt that both patterns are equally
> correct. Could be that the other AMs are in the wrong though.

My thinking was that it's better to call smgrextend() first, so that if
you run out of disk space, you get the error before WAL-logging it. That
reduces the chance that WAL replay will run out of disk space. A lot of
things are different during WAL replay, so it's quite likely that WAL
replay runs out of disk space anyway if you're living on the edge, but
still.

I didn't notice that the other callers are doing it the other way round,
though. I think they need to, so that they can stamp the page with the
LSN of the WAL record. But GiST build is special in that regard, because
it stamps all pages with GistBuildLSN.

- Heikki