Re: Pluggable Storage - Andres's take

From: Heikki Linnakangas <hlinnaka(at)iki(dot)fi>
To: Andres Freund <andres(at)anarazel(dot)de>, Justin Pryzby <pryzby(at)telsasoft(dot)com>
Cc: Haribabu Kommi <kommi(dot)haribabu(at)gmail(dot)com>, Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, David Rowley <david(dot)rowley(at)2ndquadrant(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Asim R P <apraveen(at)pivotal(dot)io>, pgsql-hackers(at)postgresql(dot)org, Ashwin Agrawal <aagrawal(at)pivotal(dot)io>, Dmitry Dolgov <9erthalion6(at)gmail(dot)com>, Alexander Korotkov <a(dot)korotkov(at)postgrespro(dot)ru>
Subject: Re: Pluggable Storage - Andres's take
Date: 2019-04-08 12:34:46
Message-ID: 9a7fb9cc-2419-5db7-8840-ddc10c93f122@iki.fi
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

I wrote a little toy implementation that just returns constant data to
play with this a little. Looks good overall.

There were a bunch of typos in the comments in tableam.h, see attached.
Some of the comments could use more copy-editing and clarification, I
think, but I stuck to fixing just typos and such for now.

index_update_stats() calls RelationGetNumberOfBlocks(<table>). If the AM
doesn't use normal data files, that won't work. I bumped into that with
my toy implementation, which wouldn't need to create any data files, if
it wasn't for this.

The comments for relation_set_new_relfilenode() callback say that the AM
can set *freezeXid and *minmulti to invalid. But when I did that, VACUUM
hits this assertion:

TRAP: FailedAssertion("!(((classForm->relfrozenxid) >= ((TransactionId)
3)))", File: "vacuum.c", Line: 1323)

There's a little bug in index-only scan executor node, where it mixes up
the slots to hold a tuple from the index, and from the table. That
doesn't cause any ill effects if the AM uses TTSOpsHeapTuple, but with
my toy AM, which uses a virtual slot, it caused warnings like this from
index-only scans:

WARNING: problem in alloc set ExecutorState: detected write past chunk
end in block 0x56419b0f88e8, chunk 0x56419b0f8f90

Attached is a patch with the toy implementation I used to test this.
I'm not suggesting we should commit that - although feel free to do that
if you think it's useful - but it shows how I bumped into these issues.
The second patch fixes the index-only-scan slot confusion (untested,
except with my toy AM).

- Heikki

Attachment Content-Type Size
0001-Add-a-toy-table-AM-implementation-to-play-with.patch text/x-patch 19.8 KB
0002-Fix-confusion-on-different-kinds-of-slots-in-IndexOn.patch text/x-patch 2.5 KB
0003-Fix-typos-and-grammar-in-tableam.h-comments.patch text/x-patch 17.9 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Jonathan S. Katz 2019-04-08 12:37:48 Re: change password_encryption default to scram-sha-256?
Previous Message Peter Eisentraut 2019-04-08 12:25:07 Re: initdb recommendations