From b089f95774892d56ad70bfc63acc23e99c245159 Mon Sep 17 00:00:00 2001 From: Nikhil Kumar Veldanda Date: Fri, 25 Sep 2026 01:26:02 -0700 Subject: [PATCH v17 2/5] Allow more than four TOAST compression methods A compressed varlena identifies its compression method in the two high-order bits of the tcinfo word of its header, and an external TOAST pointer does the same in va_extinfo. Two of the four possible values are taken by pglz and lz4, and a third is reserved as TOAST_INVALID_COMPRESSION_ID, so adding even one more method would use up the last one. Introduce a "long" form of both structures, in which the two method bits hold VARLENA_COMPRESS_METHOD_LONG and the actual method ID is stored in a byte of its own following the fixed part: - For compressed-in-line datums the long form is described by a new struct, varattrib_4b_long: the va_compressed layout with a va_cmid byte inserted before the data, giving a header of VARHDRSZ_COMPRESSED_LONG bytes. It is deliberately not a member of the varattrib_4b union, as a member with the extra byte would pad to 12 bytes and so increase sizeof(varattrib_4b) from 8; code all over the place looks at varlena headers of unknown or smaller size through pointers of that type, and compilers use the type's size to check such accesses (-Warray-bounds). Static assertions tie its layout to va_compressed. VARDATA_COMPRESSED_SET_TCINFO_LONG() writes this header, next to VARDATA_COMPRESSED_SET_TCINFO() for the plain one, and VARDATA_COMPRESSED_GET_COMPRESS_METHOD() reads both forms. - Two new TOAST pointer tags, VARTAG_ONDISK_OID_LONG and VARTAG_ONDISK_OID8_LONG, denote a plain varatt_external_oid or varatt_external_oid8 followed by the method ID byte. Using tags rather than a variable-size struct keeps VARTAG_SIZE() a pure function of the tag, which matters for tuple deforming, and avoids a struct with trailing padding that must not reach disk. VARTAG_IS_ONDISK_LONG() tells the two forms apart. Only pglz and lz4 keep using the two bits directly, so existing data is unaffected and neither form changes size for them. Every other method, present or future, uses the long form; which is decided in one place, toast_compression_id_needs_cmid_byte(). This leaves room for IDs up to 255. No method uses the long form yet: the writer side added here is first reached by zstd, in the next commit. TOAST_INVALID_COMPRESSION_ID moves from 2 to 3, so as to be equal to VARLENA_COMPRESS_METHOD_LONG: code that mistakenly reads the raw method bits of a long-form value as a method ID then ends up with an invalid ID rather than a real method. This also frees ID 2 for the next method. A compressed-in-line datum read from disk may be corrupt, and the long form makes its header variable in size, so the method is no longer read blindly. toast_inline_compression_id() checks that the datum is long enough for the header form it claims, and then that the method bits, the method byte if any, and the method agree, and returns TOAST_INVALID_COMPRESSION_ID otherwise; the decompression entry points, pg_column_compression() and toast_save_datum() (which moves an inline value out of line) then report the datum as corrupt instead of reading past its end or decoding from the wrong offset. toast_raw_datum_size() checks for a complete header too. toast_external_info_get() applies the same rule to TOAST pointers; both go through one function, toast_compression_id_resolve(). toast_external_data also keeps the stored method bits and method byte, so that a caller can tell which part of a pointer is wrong. amcheck now verifies that the compression method of an external value is stored in the pointer form appropriate for it, and that only compressed values use the long form, and reports the bits and byte it found rather than the decoded method. A new TAP test forges plain pointers whose method bits claim the long form or a method that needs it, and checks that amcheck reports them and that reading them fails cleanly. An earlier proposal used one new vartag_external per compression method, with the method implied by the tag and the TOAST pointer kept at its plain size. That works for out-of-line values only: an inline compressed datum has no vartag, so it would still have nowhere to record a third method. It also spends a vartag on every method, and there are only 256 tags shared with the in-memory pointer kinds. The long form costs one byte, in the pointer or the inline header, for methods other than pglz and lz4, and needs two tags in total however many methods are added. Reviewed-by: Michael Paquier Reviewed-by: Hannu Krosing Discussion: https://postgr.es/m/CAFAfj_GGY4W81XjtYLvYOhqPtvfZbBNj=YRS6Mq_0SA9AqEwoQ@mail.gmail.com Discussion: https://postgr.es/m/CAOj6k6dy2CRVA6Lsb5N59zE-7KNVKt=oYwWyg8ULK8zOOY8e7A@mail.gmail.com --- contrib/amcheck/Makefile | 2 +- contrib/amcheck/meson.build | 1 + contrib/amcheck/t/007_verify_toast_pointer.pl | 123 +++++++++++++++++ contrib/amcheck/verify_heapam.c | 70 +++++----- doc/src/sgml/storage.sgml | 5 +- src/backend/access/common/detoast.c | 36 ++++- src/backend/access/common/toast_compression.c | 21 ++- src/backend/access/common/toast_internals.c | 69 ++++++++-- src/include/access/detoast.h | 57 ++++++-- src/include/access/toast_compression.h | 106 ++++++++++++++- src/include/varatt.h | 125 +++++++++++++++++- src/tools/pgindent/typedefs.list | 1 + 12 files changed, 535 insertions(+), 81 deletions(-) create mode 100644 contrib/amcheck/t/007_verify_toast_pointer.pl diff --git a/contrib/amcheck/Makefile b/contrib/amcheck/Makefile index 1b7a63cbaa4..49ea1b3076f 100644 --- a/contrib/amcheck/Makefile +++ b/contrib/amcheck/Makefile @@ -15,7 +15,7 @@ PGFILEDESC = "amcheck - function for verifying relation integrity" REGRESS = check check_btree check_gin check_heap -EXTRA_INSTALL = contrib/pg_walinspect +EXTRA_INSTALL = contrib/pageinspect contrib/pg_walinspect TAP_TESTS = 1 ifdef USE_PGXS diff --git a/contrib/amcheck/meson.build b/contrib/amcheck/meson.build index d5137ef691d..fdd94f9df48 100644 --- a/contrib/amcheck/meson.build +++ b/contrib/amcheck/meson.build @@ -50,6 +50,7 @@ tests += { 't/004_verify_nbtree_unique.pl', 't/005_pitr.pl', 't/006_verify_gin.pl', + 't/007_verify_toast_pointer.pl', ], }, } diff --git a/contrib/amcheck/t/007_verify_toast_pointer.pl b/contrib/amcheck/t/007_verify_toast_pointer.pl new file mode 100644 index 00000000000..a7925850066 --- /dev/null +++ b/contrib/amcheck/t/007_verify_toast_pointer.pl @@ -0,0 +1,123 @@ + +# Copyright (c) 2026, PostgreSQL Global Development Group + +# Check that verify_heapam() reports TOAST pointers whose compression method +# is not stored in the form appropriate for it, without crashing, and that a +# well-formed pointer is not reported. +# +# A pointer is forged by rewriting bytes of a heap page on disk: the method +# bits of va_extinfo, and the vartag, which tells the plain form of a pointer +# from the long form that carries the method in a byte of its own. + +use strict; +use warnings FATAL => 'all'; + +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; + +use Test::More; + +my $node = PostgreSQL::Test::Cluster->new('test'); +$node->init(no_data_checksums => 1); +$node->append_conf('postgresql.conf', 'autovacuum=off'); +$node->start; +$node->safe_psql('postgres', q(CREATE EXTENSION amcheck)); +$node->safe_psql('postgres', q(CREATE EXTENSION pageinspect)); + +# One row per case, each with one compressed external value: a low +# toast_tuple_target pushes the compressed values out of line. +my $nrows = 3; +$node->safe_psql( + 'postgres', qq( + CREATE TABLE tp (id int, f text COMPRESSION pglz) + WITH (autovacuum_enabled = false, toast_tuple_target = 128); + INSERT INTO tp + SELECT g, repeat('abcdefgh' || g, 2000) FROM generate_series(1, $nrows) g; + CHECKPOINT; +)); + +# The values must be compressed and stored externally. +is( $node->safe_psql( + 'postgres', q( + SELECT string_agg(pg_column_compression(f) || ':' || + (pg_column_size(f) < octet_length(f)), ',' ORDER BY id) + FROM tp)), + join(',', ('pglz:true') x $nrows), + 'values are compressed with pglz'); +is($node->safe_psql('postgres', q(SELECT * FROM verify_heapam('tp'))), + '', 'no corruption before forging'); + +# Find each tuple's data in the page. A tuple is (int4 id, text f), and f is +# an on-disk TOAST pointer in the plain form: one byte 0x01, one byte vartag, +# then va_rawsize (int4) and va_extinfo (uint32), unaligned. So va_extinfo +# is at offset 4 + 2 + 4 of the tuple data. +my $relpath = $node->data_dir . '/' + . $node->safe_psql('postgres', q(SELECT pg_relation_filepath('tp'))); +my %extinfo_off; +for my $row ( + split /\n/, + $node->safe_psql( + 'postgres', q( + SELECT lp, lp_off + t_hoff, get_byte(t_data, 4), get_byte(t_data, 5) + FROM heap_page_items(get_raw_page('tp', 0)) + ORDER BY lp))) +{ + my ($lp, $dataoff, $b4, $b5) = split /\|/, $row; + is("$b4/$b5", "1/18", "tuple $lp holds a plain on-disk TOAST pointer"); + $extinfo_off{$lp} = $dataoff + 4 + 2 + 4; +} + +$node->stop; +my $fh; +open($fh, '+<', $relpath) or BAIL_OUT("open failed: $!"); +binmode $fh; + +# Rewrite the two method bits (the top bits) of a va_extinfo +sub set_method_bits +{ + my ($off, $bits) = @_; + my $code = (pack('L', 1) eq pack('V', 1)) ? 'V' : 'N'; + my $buf; + + sysseek($fh, $off, 0) or BAIL_OUT("sysseek failed: $!"); + sysread($fh, $buf, 4) == 4 or BAIL_OUT("sysread failed: $!"); + my $extinfo = (unpack($code, $buf) & 0x3FFFFFFF) | ($bits << 30); + sysseek($fh, $off, 0) or BAIL_OUT("sysseek failed: $!"); + syswrite($fh, pack($code, $extinfo)) == 4 + or BAIL_OUT("syswrite failed: $!"); +} + +# Case 1: a plain pointer whose method bits say "long form" +set_method_bits($extinfo_off{1}, 3); + +# Case 2: a plain pointer whose method bits name a method that needs the long +# form (2, zstd's ID once it exists, and in any case not pglz or lz4) +set_method_bits($extinfo_off{2}, 2); + +# Row 3 is left alone. +close($fh) or BAIL_OUT("close failed: $!"); +$node->start; + +my $result = $node->safe_psql('postgres', + q(SELECT offnum, msg FROM verify_heapam('tp') ORDER BY offnum, msg)); +like( + $result, + qr/^1\|toast value \d+ has invalid compression method id 3$/m, + 'plain pointer with long-form method bits is reported'); +like( + $result, + qr/^2\|toast value \d+ has invalid compression method id 2$/m, + 'plain pointer naming a long-form method is reported'); +unlike($result, qr/^3\|/m, 'well-formed pointer is not reported'); + +# Reading the forged values fails cleanly instead of crashing. +for my $id (1, 2) +{ + my ($ret, $stdout, $stderr) = $node->psql('postgres', + "SELECT pg_column_compression(f) FROM tp WHERE id = $id"); + like($stderr, qr/compressed data is corrupt/, + "pg_column_compression() rejects forged pointer $id"); +} +is($node->safe_psql('postgres', q(SELECT 1)), '1', 'server still up'); + +done_testing(); diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c index 289a3764e95..3583155ae0f 100644 --- a/contrib/amcheck/verify_heapam.c +++ b/contrib/amcheck/verify_heapam.c @@ -77,7 +77,7 @@ typedef struct ToastedAttribute { Oid8 va_valueid; /* value ID (works for both Oid and Oid8) */ int32 extsize; /* external saved size */ - vartag_external tag; /* VARTAG_ONDISK_OID or VARTAG_ONDISK_OID8 */ + vartag_external tag; /* one of the VARTAG_ONDISK_* tags */ BlockNumber blkno; /* block in main table */ OffsetNumber offnum; /* offset in main table */ AttrNumber attnum; /* attribute in main table */ @@ -1758,7 +1758,7 @@ check_tuple_attribute(HeapCheckContext *ctx) { uint8 va_tag = VARTAG_EXTERNAL(tp + ctx->offset); - if (va_tag != VARTAG_ONDISK_OID && va_tag != VARTAG_ONDISK_OID8) + if (!VARATT_IS_EXTERNAL_ONDISK(tp + ctx->offset)) { report_corruption(ctx, psprintf("toasted attribute has unexpected TOAST tag %u", @@ -1818,29 +1818,40 @@ check_tuple_attribute(HeapCheckContext *ctx) if (toast_ext_data.is_compressed) { - ToastCompressionId cmid; - bool valid = false; - - /* Compressed attributes should have a valid compression method */ - cmid = toast_ext_data.compress_method; - switch (cmid) + /* + * Compressed attributes should have a valid compression method, and + * it must be stored in the TOAST pointer form appropriate for it: the + * methods that fit in the two method bits of va_extinfo use the plain + * pointer, all others the long form, which flags itself in those bits + * and carries the method in a byte of its own. Report what is stored + * rather than the decoded method, which is just "invalid" for every + * kind of mismatch. + */ + if (toast_ext_data.compress_method == TOAST_INVALID_COMPRESSION_ID) { - /* List of all valid compression method IDs */ - case TOAST_PGLZ_COMPRESSION_ID: - case TOAST_LZ4_COMPRESSION_ID: - valid = true; - break; - - /* Recognized but invalid compression method ID */ - case TOAST_INVALID_COMPRESSION_ID: - break; - - /* Intentionally no default here */ + if (toast_ext_data.method_byte < 0) + report_corruption(ctx, + psprintf("toast value %" PRIu64 " has invalid compression method id %d", + toast_pointer_valueid, + toast_ext_data.method_bits)); + else if (toast_ext_data.method_bits != VARLENA_COMPRESS_METHOD_LONG) + report_corruption(ctx, + psprintf("toast value %" PRIu64 " has a long TOAST pointer but compression method bits %d", + toast_pointer_valueid, + toast_ext_data.method_bits)); + else + report_corruption(ctx, + psprintf("toast value %" PRIu64 " has a long TOAST pointer with invalid compression method id %d", + toast_pointer_valueid, + toast_ext_data.method_byte)); } - if (!valid) - report_corruption(ctx, - psprintf("toast value %" PRIu64 " has invalid compression method id %d", - toast_pointer_valueid, cmid)); + } + else if (VARTAG_IS_ONDISK_LONG(va_tag_value)) + { + /* Only compressed values get the long form of the pointer */ + report_corruption(ctx, + psprintf("toast value %" PRIu64 " is not compressed but has a long TOAST pointer", + toast_pointer_valueid)); } /* The tuple header better claim to contain toasted values */ @@ -1908,7 +1919,6 @@ check_toasted_attribute(HeapCheckContext *ctx, ToastedAttribute *ta) int32 max_chunk_size; Oid8 toast_valueid; Oid toast_typid; - vartag_external expected_tag; toast_valueid = ta->va_valueid; extsize = ta->extsize; @@ -1916,14 +1926,11 @@ check_toasted_attribute(HeapCheckContext *ctx, ToastedAttribute *ta) /* * Take the chunk_id type from the TOAST table's own definition, not from * the vartag in the main table as that pointer is the very thing under - * scrutiny here. The two must agree. + * scrutiny here. The two must agree, whether the pointer is of the plain + * or of the long form. */ toast_typid = TupleDescAttr(ctx->toast_rel->rd_att, 0)->atttypid; - if (toast_typid == OID8OID) - expected_tag = VARTAG_ONDISK_OID8; - else if (toast_typid == OIDOID) - expected_tag = VARTAG_ONDISK_OID; - else + if (toast_typid != OIDOID && toast_typid != OID8OID) { report_toast_corruption(ctx, ta, psprintf("toast value %" PRIu64 " stored in toast table whose chunk_id has unexpected type %u", @@ -1931,7 +1938,8 @@ check_toasted_attribute(HeapCheckContext *ctx, ToastedAttribute *ta) return; } - if (ta->tag != expected_tag) + if ((ta->tag == VARTAG_ONDISK_OID8 || ta->tag == VARTAG_ONDISK_OID8_LONG) != + (toast_typid == OID8OID)) { report_toast_corruption(ctx, ta, psprintf("toast value %" PRIu64 " has TOAST tag %u, but chunk_id of toast table has type %u", diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml index 58f997d5da0..0d8d35b1bbd 100644 --- a/doc/src/sgml/storage.sgml +++ b/doc/src/sgml/storage.sgml @@ -441,7 +441,10 @@ any. Allowing for the varlena header bytes, the total size of an on-disk TOAST pointer datum is 18 bytes when using an OID as chunk_id, or 22 bytes when using an OID8 as chunk_id, regardless of the -actual size of the represented value. +actual size of the represented value. Values compressed with a method other +than pglz or lz4 need one additional +byte to identify the compression method, both in the pointer datum and in +the header of the compressed data itself. diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c index a07703ab9cc..fdf3c8c96ea 100644 --- a/src/backend/access/common/detoast.c +++ b/src/backend/access/common/detoast.c @@ -445,11 +445,15 @@ toast_fetch_datum_slice(varlena *attr, int32 sliceoffset, /* * When fetching a prefix of a compressed external datum, account for the - * space required by va_tcinfo, which is stored at the beginning as an - * int32 value. + * compressed header stored at its beginning: va_tcinfo, followed in the + * long form by the compression method ID byte. (Today only pglz asks for + * a true prefix, see detoast_attr_slice(); other methods ask for the + * whole value, and the request is clamped to it below.) */ if (is_compressed && slicelength > 0) - slicelength = slicelength + sizeof(int32); + slicelength += (VARTAG_IS_ONDISK_LONG(toast_ext_data.tag) ? + VARHDRSZ_COMPRESSED_LONG : VARHDRSZ_COMPRESSED) - + VARHDRSZ; /* * Adjust length request if needed. (Note: our sole caller, @@ -499,7 +503,11 @@ toast_decompress_datum(varlena *attr) * Fetch the compression method id stored in the compression header and * decompress the data using the appropriate decompression routine. */ - cmid = VARDATA_COMPRESSED_GET_COMPRESS_METHOD(attr); + cmid = toast_inline_compression_id(attr); + if (cmid == TOAST_INVALID_COMPRESSION_ID) + ereport(ERROR, + (errcode(ERRCODE_DATA_CORRUPTED), + errmsg_internal("compressed data is corrupt"))); switch (cmid) { case TOAST_PGLZ_COMPRESSION_ID: @@ -527,6 +535,17 @@ toast_decompress_datum_slice(varlena *attr, int32 slicelength) Assert(VARATT_IS_COMPRESSED(attr)); + /* + * Fetch the compression method id stored in the compression header. Do + * it first, as it also checks the header is complete before we read the + * decompressed size from it below. + */ + cmid = toast_inline_compression_id(attr); + if (cmid == TOAST_INVALID_COMPRESSION_ID) + ereport(ERROR, + (errcode(ERRCODE_DATA_CORRUPTED), + errmsg_internal("compressed data is corrupt"))); + /* * Some callers may pass a slicelength that's more than the actual * decompressed size. If so, just decompress normally. This avoids @@ -539,10 +558,8 @@ toast_decompress_datum_slice(varlena *attr, int32 slicelength) return toast_decompress_datum(attr); /* - * Fetch the compression method id stored in the compression header and - * decompress the data slice using the appropriate decompression routine. + * Decompress the data slice using the appropriate decompression routine. */ - cmid = VARDATA_COMPRESSED_GET_COMPRESS_METHOD(attr); switch (cmid) { case TOAST_PGLZ_COMPRESSION_ID: @@ -593,6 +610,11 @@ toast_raw_datum_size(Datum value) } else if (VARATT_IS_COMPRESSED(attr)) { + /* the header must be complete for us to read the size from it */ + if (VARSIZE(attr) < VARHDRSZ_COMPRESSED) + ereport(ERROR, + (errcode(ERRCODE_DATA_CORRUPTED), + errmsg_internal("compressed data is corrupt"))); /* here, va_rawsize is just the payload size */ result = VARDATA_COMPRESSED_GET_EXTSIZE(attr) + VARHDRSZ; } diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c index 8250cf85879..c1cea3ee8f8 100644 --- a/src/backend/access/common/toast_compression.c +++ b/src/backend/access/common/toast_compression.c @@ -89,6 +89,8 @@ pglz_decompress_datum(const varlena *value) varlena *result; int32 rawsize; + Assert(VARDATA_COMPRESSED_GET_COMPRESS_METHOD(value) == TOAST_PGLZ_COMPRESSION_ID); + /* * Allocate memory for the uncompressed data. VARLENA_EXTSIZE_MASK * protects this against overflow. @@ -120,6 +122,8 @@ pglz_decompress_datum_slice(const varlena *value, varlena *result; int32 rawsize; + Assert(VARDATA_COMPRESSED_GET_COMPRESS_METHOD(value) == TOAST_PGLZ_COMPRESSION_ID); + /* * Allocate memory for the uncompressed data. Caller must have checked * the requested size, but verify. @@ -208,6 +212,8 @@ lz4_decompress_datum(const varlena *value) int32 rawsize; varlena *result; + Assert(VARDATA_COMPRESSED_GET_COMPRESS_METHOD(value) == TOAST_LZ4_COMPRESSION_ID); + /* * Allocate memory for the uncompressed data. VARLENA_EXTSIZE_MASK * protects this against overflow. @@ -245,6 +251,8 @@ lz4_decompress_datum_slice(const varlena *value, int32 slicelength) int32 rawsize; varlena *result; + Assert(VARDATA_COMPRESSED_GET_COMPRESS_METHOD(value) == TOAST_LZ4_COMPRESSION_ID); + /* slice decompression not supported prior to 1.8.3 */ if (LZ4_versionNumber() < 10803) return lz4_decompress_datum(value); @@ -296,9 +304,20 @@ toast_get_compression_id(varlena *attr) toast_external_info_get(attr, &toast_ext_data); cmid = toast_ext_data.compress_method; + if (toast_ext_data.is_compressed && + cmid == TOAST_INVALID_COMPRESSION_ID) + ereport(ERROR, + (errcode(ERRCODE_DATA_CORRUPTED), + errmsg_internal("compressed data is corrupt"))); } else if (VARATT_IS_COMPRESSED(attr)) - cmid = VARDATA_COMPRESSED_GET_COMPRESS_METHOD(attr); + { + cmid = toast_inline_compression_id(attr); + if (cmid == TOAST_INVALID_COMPRESSION_ID) + ereport(ERROR, + (errcode(ERRCODE_DATA_CORRUPTED), + errmsg_internal("compressed data is corrupt"))); + } return cmid; } diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c index a42ca7ed76c..6a9a2a85e75 100644 --- a/src/backend/access/common/toast_internals.c +++ b/src/backend/access/common/toast_internals.c @@ -35,18 +35,33 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid8 valueid); * toast_pointer_build - * * Build an on-disk TOAST pointer datum based on the given tag from "ptr", - * itself a varatt_external_*. + * itself a varatt_external_*. For the long-form tags, the compression + * method ID byte is appended to it. * ---------- */ static inline varlena * -toast_pointer_build(vartag_external tag, const void *ptr) +toast_pointer_build(vartag_external tag, const void *ptr, + ToastCompressionId cmid) { varlena *result; + char *data; + Size size = VARTAG_SIZE(tag); - result = (varlena *) palloc(VARHDRSZ_EXTERNAL + VARTAG_SIZE(tag)); + result = (varlena *) palloc(VARHDRSZ_EXTERNAL + size); SET_VARTAG_EXTERNAL(result, tag); Assert(VARATT_IS_EXTERNAL_ONDISK(result)); - memcpy(VARDATA_EXTERNAL(result), ptr, VARTAG_SIZE(tag)); + data = VARDATA_EXTERNAL(result); + + if (VARTAG_IS_ONDISK_LONG(tag)) + { + uint8 cmid_byte = (uint8) cmid; + + Assert(cmid != TOAST_INVALID_COMPRESSION_ID && + toast_compression_id_needs_cmid_byte(cmid)); + size -= VARATT_EXTERNAL_CMID_SIZE; + memcpy(data + size, &cmid_byte, sizeof(cmid_byte)); + } + memcpy(data, ptr, size); return result; } @@ -112,9 +127,15 @@ toast_compress_datum(Datum value, char cmethod) */ if (VARSIZE(tmp) < valsize - 2) { - /* successful compression */ + /* + * Successful compression. The compression routine has laid out the + * header in the form its method requires; fill it in accordingly. + */ Assert(cmid != TOAST_INVALID_COMPRESSION_ID); - VARDATA_COMPRESSED_SET_TCINFO(tmp, valsize, cmid); + if (toast_compression_id_needs_cmid_byte(cmid)) + VARDATA_COMPRESSED_SET_TCINFO_LONG(tmp, valsize, cmid); + else + VARDATA_COMPRESSED_SET_TCINFO(tmp, valsize, cmid); return PointerGetDatum(tmp); } else @@ -210,6 +231,8 @@ toast_save_datum(Relation rel, Datum value, uint32 va_extinfo; Oid8 va_valueid; Oid va_toastrelid; + ToastCompressionId cmid = TOAST_INVALID_COMPRESSION_ID; + bool pointer_long = false; Assert(!VARATT_IS_EXTERNAL(dval)); @@ -237,7 +260,9 @@ toast_save_datum(Relation rel, Datum value, * * va_extinfo stored the actual size of the data payload in the toast * records and the compression method in first 2 bits if data is - * compressed. + * compressed. Methods that don't fit in those bits are flagged there + * with VARLENA_COMPRESS_METHOD_LONG and require the long form of the + * TOAST pointer, which carries the actual method ID in an extra byte. */ if (VARATT_IS_SHORT(dval)) { @@ -248,7 +273,15 @@ toast_save_datum(Relation rel, Datum value, } else if (VARATT_IS_COMPRESSED(dval)) { - uint32 cmid = VARDATA_COMPRESSED_GET_COMPRESS_METHOD(dval); + /* + * The value may be an unchanged one from an existing tuple, read from + * disk, so check its header before we trust it. + */ + cmid = toast_inline_compression_id(dval); + if (cmid == TOAST_INVALID_COMPRESSION_ID) + ereport(ERROR, + (errcode(ERRCODE_DATA_CORRUPTED), + errmsg_internal("compressed data is corrupt"))); data_p = VARDATA(dval); data_todo = VARSIZE(dval) - VARHDRSZ; @@ -256,9 +289,14 @@ toast_save_datum(Relation rel, Datum value, va_rawsize = VARDATA_COMPRESSED_GET_EXTSIZE(dval) + VARHDRSZ; /* set external size and compression method */ - Assert(cmid == TOAST_PGLZ_COMPRESSION_ID || - cmid == TOAST_LZ4_COMPRESSION_ID); - va_extinfo = data_todo | (cmid << VARLENA_EXTSIZE_BITS); + if (toast_compression_id_needs_cmid_byte(cmid)) + { + pointer_long = true; + va_extinfo = data_todo | + ((uint32) VARLENA_COMPRESS_METHOD_LONG << VARLENA_EXTSIZE_BITS); + } + else + va_extinfo = data_todo | ((uint32) cmid << VARLENA_EXTSIZE_BITS); /* Assert that the numbers look like it's compressed */ Assert(VARATT_EXTINFO_IS_COMPRESSED(va_extinfo, va_rawsize)); @@ -403,7 +441,8 @@ toast_save_datum(Relation rel, Datum value, table_close(toastrel, NoLock); /* - * Create the TOAST pointer value that we'll return + * Create the TOAST pointer value that we'll return. The long form is the + * plain one followed by the compression method ID byte. */ if (toast_typid == OID8OID) { @@ -414,7 +453,8 @@ toast_save_datum(Relation rel, Datum value, VARATT_EXTERNAL_OID8_SET_VALUEID(&toast_pointer, va_valueid); toast_pointer.va_toastrelid = va_toastrelid; - result = toast_pointer_build(VARTAG_ONDISK_OID8, &toast_pointer); + result = toast_pointer_build(pointer_long ? VARTAG_ONDISK_OID8_LONG : VARTAG_ONDISK_OID8, + &toast_pointer, cmid); } else { @@ -425,7 +465,8 @@ toast_save_datum(Relation rel, Datum value, toast_pointer.va_valueid = (Oid) va_valueid; toast_pointer.va_toastrelid = va_toastrelid; - result = toast_pointer_build(VARTAG_ONDISK_OID, &toast_pointer); + result = toast_pointer_build(pointer_long ? VARTAG_ONDISK_OID_LONG : VARTAG_ONDISK_OID, + &toast_pointer, cmid); } return PointerGetDatum(result); diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h index fe3444c6739..bb2285a9136 100644 --- a/src/include/access/detoast.h +++ b/src/include/access/detoast.h @@ -43,8 +43,11 @@ do { \ * Decoded contents of an on-disk external TOAST pointer. * * Callers use these fields rather than decoding va_extinfo themselves. - * compress_method is TOAST_INVALID_COMPRESSION_ID when the value is not - * compressed. + * compress_method is the actual method ID whichever pointer form was used. + * It is TOAST_INVALID_COMPRESSION_ID when the value is not compressed, and + * also when is_compressed is set but the pointer is inconsistent (see + * toast_external_info_get()); method_bits and method_byte keep what is + * stored so that callers such as amcheck can tell which. */ typedef struct toast_external_data { @@ -53,6 +56,8 @@ typedef struct toast_external_data int32 extsize; /* external saved size (without header) */ bool is_compressed; /* is the saved data compressed? */ ToastCompressionId compress_method; /* compression method ID */ + uint8 method_bits; /* raw method bits of va_extinfo */ + int16 method_byte; /* trailing method byte, or -1 if plain form */ Oid8 valueid; /* value ID (can be widened from Oid) */ Oid toastrelid; /* OID of the TOAST table containing it */ } toast_external_data; @@ -63,45 +68,73 @@ typedef struct toast_external_data static inline void toast_external_info_get(const struct varlena *attr, toast_external_data *toast_ext_data) { + const char *ptr; + Size fixedsize; uint32 extinfo; Assert(VARATT_IS_EXTERNAL_ONDISK(attr)); + /* + * The long form of a pointer is the plain form followed by the + * compression method ID byte, so decoding the fixed part is the same for + * both. We can't use VARATT_EXTERNAL_GET_POINTER() here because it + * insists on the datum being exactly the size of the struct. + */ + ptr = VARDATA_EXTERNAL(attr); toast_ext_data->tag = VARTAG_EXTERNAL(attr); - if (toast_ext_data->tag == VARTAG_ONDISK_OID8) + if (toast_ext_data->tag == VARTAG_ONDISK_OID8 || + toast_ext_data->tag == VARTAG_ONDISK_OID8_LONG) { varatt_external_oid8 toast_pointer; - VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr); + memcpy(&toast_pointer, ptr, sizeof(toast_pointer)); toast_ext_data->rawsize = toast_pointer.va_rawsize; extinfo = toast_pointer.va_extinfo; toast_ext_data->valueid = VARATT_EXTERNAL_OID8_GET_VALUEID(toast_pointer); toast_ext_data->toastrelid = toast_pointer.va_toastrelid; + fixedsize = sizeof(toast_pointer); } else { varatt_external_oid toast_pointer; - VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr); + memcpy(&toast_pointer, ptr, sizeof(toast_pointer)); toast_ext_data->rawsize = toast_pointer.va_rawsize; extinfo = toast_pointer.va_extinfo; toast_ext_data->valueid = toast_pointer.va_valueid; toast_ext_data->toastrelid = toast_pointer.va_toastrelid; + fixedsize = sizeof(toast_pointer); } toast_ext_data->extsize = VARATT_EXTINFO_GET_EXTSIZE(extinfo); toast_ext_data->is_compressed = VARATT_EXTINFO_IS_COMPRESSED(extinfo, toast_ext_data->rawsize); + toast_ext_data->method_bits = extinfo >> VARLENA_EXTSIZE_BITS; + toast_ext_data->method_byte = -1; + if (VARTAG_IS_ONDISK_LONG(toast_ext_data->tag)) + { + uint8 cmid; + + memcpy(&cmid, ptr + fixedsize, sizeof(cmid)); + toast_ext_data->method_byte = cmid; + } /* - * An external datum that is not compressed has no method; report it as - * the invalid id rather than leaving the raw method bits (which read as - * pglz) in place, so toast_external_data is self-describing. + * Resolve the compression method: it is the two method bits of extinfo, + * except in the long form of a pointer, where those bits only flag the + * form and the method is in the trailing byte. + * + * A long-form pointer whose method bits say otherwise is corrupt, and so + * is one whose trailing byte names a method that uses the plain form or + * no method at all. Likewise, a plain pointer can only name pglz or lz4. + * This is also used on data read from disk, notably by amcheck, so don't + * assert on that: report an invalid method and let the caller deal with + * it. An external datum that is not compressed has no method either. */ - if (toast_ext_data->is_compressed) - toast_ext_data->compress_method = (ToastCompressionId) - (extinfo >> VARLENA_EXTSIZE_BITS); - else + toast_ext_data->compress_method = + toast_compression_id_resolve(toast_ext_data->method_bits, + toast_ext_data->method_byte); + if (!toast_ext_data->is_compressed) toast_ext_data->compress_method = TOAST_INVALID_COMPRESSION_ID; } diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h index 3265f10b734..888269393ba 100644 --- a/src/include/access/toast_compression.h +++ b/src/include/access/toast_compression.h @@ -13,6 +13,8 @@ #ifndef TOAST_COMPRESSION_H #define TOAST_COMPRESSION_H +#include "varatt.h" + /* * GUC support. * @@ -23,24 +25,114 @@ extern PGDLLIMPORT int default_toast_compression; /* - * Built-in compression method ID. The toast compression header will store - * this in the first 2 bits of the raw length. These built-in compression - * method IDs are directly mapped to the built-in compression methods. + * Built-in compression method ID. These built-in compression method IDs are + * directly mapped to the built-in compression methods. + * + * A compressed varlena identifies its method in the two high-order bits of + * its tcinfo/extinfo word. Only pglz and lz4 are stored there directly; all + * other methods use the long form of the header, flagged by + * VARLENA_COMPRESS_METHOD_LONG in those bits, which stores the ID in a byte + * of its own (see varatt.h) and so leaves room for IDs up to 255. + * toast_compression_id_needs_cmid_byte() tells which form a given ID uses. + * + * TOAST_INVALID_COMPRESSION_ID is not a real compression method and is never + * stored on disk; it only serves to report "this value is not compressed". + * It is deliberately equal to VARLENA_COMPRESS_METHOD_LONG, so that code + * that mistakenly interprets the raw two-bit field of a long-form value as a + * method ID ends up with an invalid ID rather than a real method. * * Don't use these values for anything other than understanding the meaning * of the raw bits from a varlena; in particular, if the goal is to identify * a compression method, use the constants TOAST_PGLZ_COMPRESSION, etc. - * below. We might someday support more than 4 compression methods, but - * we can never have more than 4 values in this enum, because there are - * only 2 bits available in the places where this is stored. + * below. */ typedef enum ToastCompressionId { TOAST_PGLZ_COMPRESSION_ID = 0, TOAST_LZ4_COMPRESSION_ID = 1, - TOAST_INVALID_COMPRESSION_ID = 2, + TOAST_INVALID_COMPRESSION_ID = 3, } ToastCompressionId; +StaticAssertDecl(TOAST_INVALID_COMPRESSION_ID == VARLENA_COMPRESS_METHOD_LONG, + "TOAST_INVALID_COMPRESSION_ID must match VARLENA_COMPRESS_METHOD_LONG"); + +/* Largest compression method ID in use; IDs above it are not methods. */ +#define TOAST_LAST_COMPRESSION_ID TOAST_LZ4_COMPRESSION_ID + +/* + * Does this compression method ID need a byte of its own? + * + * Only the two original methods fit in the two method bits of the header; + * every other method needs the long form of the compressed-in-line header + * and of the TOAST pointer, which carry the ID in a separate byte. Not + * meaningful for TOAST_INVALID_COMPRESSION_ID. + */ +static inline bool +toast_compression_id_needs_cmid_byte(ToastCompressionId cmid) +{ + Assert(cmid != TOAST_INVALID_COMPRESSION_ID); + return (cmid != TOAST_PGLZ_COMPRESSION_ID && + cmid != TOAST_LZ4_COMPRESSION_ID); +} + +/* + * Resolve the compression method ID recorded in a compressed header or TOAST + * pointer, or TOAST_INVALID_COMPRESSION_ID if the two disagree. + * + * method_bits are the two method bits; method_byte is the separate method + * byte of the long form, or -1 if the header or pointer is in the plain + * form. The method bits must say "long form" exactly when there is a method + * byte, and the byte must name a method that uses the long form: pglz and + * lz4 use the plain form, every other method the long one. + */ +static inline ToastCompressionId +toast_compression_id_resolve(uint32 method_bits, int method_byte) +{ + if (method_byte < 0) + { + if (method_bits == VARLENA_COMPRESS_METHOD_LONG || + toast_compression_id_needs_cmid_byte((ToastCompressionId) method_bits)) + return TOAST_INVALID_COMPRESSION_ID; + return (ToastCompressionId) method_bits; + } + if (method_bits != VARLENA_COMPRESS_METHOD_LONG || + method_byte == TOAST_INVALID_COMPRESSION_ID || + method_byte > TOAST_LAST_COMPRESSION_ID || + !toast_compression_id_needs_cmid_byte((ToastCompressionId) method_byte)) + return TOAST_INVALID_COMPRESSION_ID; + return (ToastCompressionId) method_byte; +} + +/* + * Compression method ID of a compressed-in-line datum, or + * TOAST_INVALID_COMPRESSION_ID if its header is malformed: the datum is too + * short for the header form it claims, or the form does not match the method + * (see toast_compression_id_resolve()). + * + * Use this rather than VARDATA_COMPRESSED_GET_COMPRESS_METHOD() on a datum + * that may come from disk. A corrupt one could otherwise make us read past + * its end, or dispatch to a method that decodes it from the wrong offset. + */ +static inline ToastCompressionId +toast_inline_compression_id(const void *PTR) +{ + uint32 method_bits; + int method_byte = -1; + + Assert(VARATT_IS_COMPRESSED(PTR)); + + if (VARSIZE(PTR) < VARHDRSZ_COMPRESSED) + return TOAST_INVALID_COMPRESSION_ID; + method_bits = VARDATA_COMPRESSED_GET_RAW_METHOD(PTR); + if (method_bits == VARLENA_COMPRESS_METHOD_LONG) + { + if (VARSIZE(PTR) < VARHDRSZ_COMPRESSED_LONG) + return TOAST_INVALID_COMPRESSION_ID; + method_byte = ((const varattrib_4b_long *) PTR)->va_cmid; + } + return toast_compression_id_resolve(method_bits, method_byte); +} + /* * Built-in compression methods. pg_attribute will store these in the * attcompression column. In attcompression, InvalidCompressionMethod diff --git a/src/include/varatt.h b/src/include/varatt.h index 602f45fa627..3689049a8ce 100644 --- a/src/include/varatt.h +++ b/src/include/varatt.h @@ -84,9 +84,27 @@ VARATT_EXTERNAL_OID8_SET_VALUEID(varatt_external_oid8 *toast_pointer, Oid8 id) * These macros define the "saved size" portion of va_extinfo. This value has * been chosen to avoid integer wraparound; see detoast_external_attr(). The * remaining two high-order bits identify the compression method. + * + * Only the two original compression methods (pglz and lz4) are identified + * directly by those two bits. Any other method uses the "long" form of the + * header, which is flagged by the value VARLENA_COMPRESS_METHOD_LONG in those + * bits and carries the actual method ID in a byte of its own following the + * fixed part; see varattrib_4b_long and the VARTAG_ONDISK_*_LONG TOAST + * pointers below. */ #define VARLENA_EXTSIZE_BITS 30 #define VARLENA_EXTSIZE_MASK ((1U << VARLENA_EXTSIZE_BITS) - 1) +#define VARLENA_COMPRESS_METHOD_LONG 3 + +/* + * Size of the compression method ID byte that the VARTAG_ONDISK_*_LONG TOAST + * pointers append to a varatt_external_oid or varatt_external_oid8. No + * struct is declared for those pointers, because a struct containing that + * trailing byte would carry padding that we must not store on disk; decode + * the fixed part as usual and then read the extra byte (see + * toast_external_info_get()). + */ +#define VARATT_EXTERNAL_CMID_SIZE sizeof(uint8) /* * varatt_indirect is a "TOAST pointer" representing an out-of-line @@ -124,6 +142,13 @@ typedef struct varatt_expanded * value for VARTAG_ONDISK_OID comes from a requirement for on-disk * compatibility with a previous notion that the tag field was the pointer * datum's length. + * + * The _LONG variants of the on-disk tags denote the long form of a TOAST + * pointer, which appends a compression method ID byte to the corresponding + * plain form (see VARATT_EXTERNAL_CMID_SIZE). It is used only for values + * whose compression method does not fit in the two method bits of + * va_extinfo. Each _LONG tag is numbered next to its plain counterpart, + * but nothing relies on that. */ typedef enum vartag_external { @@ -131,7 +156,9 @@ typedef enum vartag_external VARTAG_EXPANDED_RO = 2, VARTAG_EXPANDED_RW = 3, VARTAG_ONDISK_OID8 = 4, - VARTAG_ONDISK_OID = 18 + VARTAG_ONDISK_OID8_LONG = 5, + VARTAG_ONDISK_OID = 18, + VARTAG_ONDISK_OID_LONG = 19 } vartag_external; /* Is a TOAST pointer either type of expanded-object pointer? */ @@ -142,6 +169,13 @@ VARTAG_IS_EXPANDED(vartag_external tag) return ((tag & ~1) == VARTAG_EXPANDED_RO); } +/* Is a TOAST pointer the long form of an on-disk pointer? */ +static inline bool +VARTAG_IS_ONDISK_LONG(vartag_external tag) +{ + return (tag == VARTAG_ONDISK_OID_LONG || tag == VARTAG_ONDISK_OID8_LONG); +} + /* Size of the data part of a "TOAST pointer" datum */ static inline Size VARTAG_SIZE(vartag_external tag) @@ -154,6 +188,10 @@ VARTAG_SIZE(vartag_external tag) return sizeof(varatt_external_oid); else if (tag == VARTAG_ONDISK_OID8) return sizeof(varatt_external_oid8); + else if (tag == VARTAG_ONDISK_OID_LONG) + return sizeof(varatt_external_oid) + VARATT_EXTERNAL_CMID_SIZE; + else if (tag == VARTAG_ONDISK_OID8_LONG) + return sizeof(varatt_external_oid8) + VARATT_EXTERNAL_CMID_SIZE; else { Assert(false); @@ -186,6 +224,33 @@ typedef union } va_compressed; } varattrib_4b; +/* + * Long form of the compressed-in-line format, used when the method bits of + * va_tcinfo hold VARLENA_COMPRESS_METHOD_LONG. It is the va_compressed + * layout above with the compression method ID inserted before the data. + * + * This is deliberately not a member of the varattrib_4b union: a member with + * the extra byte would pad to 12 bytes and so increase sizeof(varattrib_4b) + * from 8, and code all over the place inspects varlena headers of unknown or + * smaller size through pointers of that type. Only ever use offsetof() on + * this struct, never sizeof(), as it has trailing padding. + */ +typedef struct +{ + uint32 va_header; + uint32 va_tcinfo; /* As in va_compressed, with the method bits + * set to VARLENA_COMPRESS_METHOD_LONG */ + uint8 va_cmid; /* Compression method ID */ + char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */ +} varattrib_4b_long; + +StaticAssertDecl(offsetof(varattrib_4b_long, va_tcinfo) == + offsetof(varattrib_4b, va_compressed.va_tcinfo), + "varattrib_4b_long must extend va_compressed"); +StaticAssertDecl(offsetof(varattrib_4b_long, va_cmid) == + offsetof(varattrib_4b, va_compressed.va_data), + "varattrib_4b_long must extend va_compressed"); + typedef struct { uint8 va_header; @@ -322,6 +387,7 @@ typedef struct #define VARHDRSZ_EXTERNAL offsetof(varattrib_1b_e, va_data) #define VARHDRSZ_COMPRESSED offsetof(varattrib_4b, va_compressed.va_data) +#define VARHDRSZ_COMPRESSED_LONG offsetof(varattrib_4b_long, va_data) #define VARHDRSZ_SHORT offsetof(varattrib_1b, va_data) #define VARATT_SHORT_MAX 0x7F @@ -412,7 +478,8 @@ VARATT_IS_EXTERNAL_ONDISK(const void *PTR) if (!VARATT_IS_EXTERNAL(PTR)) return false; tag = VARTAG_EXTERNAL(PTR); - return (tag == VARTAG_ONDISK_OID || tag == VARTAG_ONDISK_OID8); + return (tag == VARTAG_ONDISK_OID || tag == VARTAG_ONDISK_OID8 || + VARTAG_IS_ONDISK_LONG(tag)); } /* Is varlena datum an indirect pointer? */ @@ -547,33 +614,77 @@ VARDATA_COMPRESSED_GET_EXTSIZE(const void *PTR) return ((const varattrib_4b *) PTR)->va_compressed.va_tcinfo & VARLENA_EXTSIZE_MASK; } -/* Compression method of a compressed-in-line varlena datum */ +/* + * Raw two-bit method field of a compressed-in-line varlena, without reading + * the long-form method byte. A value of VARLENA_COMPRESS_METHOD_LONG means + * the real method is in va_cmid, which is only present (and safe to read) + * when the datum is at least VARHDRSZ_COMPRESSED_LONG bytes. + */ +static inline uint32 +VARDATA_COMPRESSED_GET_RAW_METHOD(const void *PTR) +{ + const varattrib_4b *va = (const varattrib_4b *) PTR; + + return va->va_compressed.va_tcinfo >> VARLENA_EXTSIZE_BITS; +} + +/* + * Compression method of a compressed-in-line varlena datum. This handles + * both header forms, so it always returns the actual method ID. + */ static inline uint32 VARDATA_COMPRESSED_GET_COMPRESS_METHOD(const void *PTR) { - return ((const varattrib_4b *) PTR)->va_compressed.va_tcinfo >> VARLENA_EXTSIZE_BITS; + const varattrib_4b *va = (const varattrib_4b *) PTR; + uint32 method = va->va_compressed.va_tcinfo >> VARLENA_EXTSIZE_BITS; + + if (method == VARLENA_COMPRESS_METHOD_LONG) + method = ((const varattrib_4b_long *) PTR)->va_cmid; + return method; } /* * Set the decompressed size and the compression method of a * compressed-in-line varlena datum. The length word is not touched here; the * caller is expected to have set it with SET_VARSIZE_COMPRESSED(). + * + * This writes the plain form of the header, so the method must be one that + * fits in the two method bits; VARLENA_COMPRESS_METHOD_LONG is not a method. */ static inline void VARDATA_COMPRESSED_SET_TCINFO(void *PTR, uint32 extsize, uint32 cmethod) { Assert(extsize > 0 && extsize <= VARLENA_EXTSIZE_MASK); - Assert(cmethod < (1U << (32 - VARLENA_EXTSIZE_BITS))); + Assert(cmethod < VARLENA_COMPRESS_METHOD_LONG); ((varattrib_4b *) PTR)->va_compressed.va_tcinfo = extsize | (cmethod << VARLENA_EXTSIZE_BITS); } +/* + * Same for the long form of the header: the method bits are set to + * VARLENA_COMPRESS_METHOD_LONG and the method goes into va_cmid. The datum + * must have been laid out with VARHDRSZ_COMPRESSED_LONG bytes of header. + */ +static inline void +VARDATA_COMPRESSED_SET_TCINFO_LONG(void *PTR, uint32 extsize, uint8 cmethod) +{ + varattrib_4b_long *va = (varattrib_4b_long *) PTR; + + Assert(extsize > 0 && extsize <= VARLENA_EXTSIZE_MASK); + + va->va_tcinfo = extsize | + ((uint32) VARLENA_COMPRESS_METHOD_LONG << VARLENA_EXTSIZE_BITS); + va->va_cmid = cmethod; +} + /* * Same for the saved size of external Datums, stored in va_extinfo. * - * There is deliberately no equivalent for the compression method: decode the - * pointer with toast_external_info_get() and use its compress_method field. + * There is deliberately no equivalent for the compression method: in the long + * form of a TOAST pointer it lives in a separate byte, so it cannot be + * derived from va_extinfo alone. Decode the pointer with + * toast_external_info_get() and use its compress_method field instead. */ static inline Size VARATT_EXTINFO_GET_EXTSIZE(uint32 extinfo) diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index f10eb469266..0acf231f66d 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -4424,6 +4424,7 @@ varatt_indirect varattrib_1b varattrib_1b_e varattrib_4b +varattrib_4b_long varlena vartag_external vbits -- 2.50.1