Re: Reduce build times of pg_trgm GIN indexes

From: Kirill Reshke <reshkekirill(at)gmail(dot)com>
To: David Geier <geidav(dot)pg(at)gmail(dot)com>
Cc: Heikki Linnakangas <hlinnaka(at)iki(dot)fi>, Matthias van de Meent <boekewurm+postgres(at)gmail(dot)com>, Japin Li <japinli(at)hotmail(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Reduce build times of pg_trgm GIN indexes
Date: 2026-11-10 03:01:07
Message-ID: CALdSSPjKLKxz3_QAJJwgtSLCC6oRFcdis2bP-EXcNqoT=bcZQg@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, 10 Nov 2026 at 02:10, Kirill Reshke <reshkekirill(at)gmail(dot)com> wrote:
>
> On Mon, 9 Nov 2026 at 20:34, David Geier <geidav(dot)pg(at)gmail(dot)com> wrote:
> >
> > > Committed
> > > v11-0001-Use-branchless-comparisons-in-btint4cmp-and-btin.patch, since
> > > that one seems uncontroversial. Thanks!
> >
> > Thanks!
> >
> > >> In the case of btint4cmp(), it's only ever invoked from the function
> > >> manager, where it cannot be inlined.
> > >>
> > >> Or are there ways to invoke btint4cmp() that can be inlined, which I'm
> > >> unaware of?
> > >
> > > JIT compilation, maybe? Unless someone shows that there's a measurable
> > > problem though, I wouldn't worry about it.
> >
> > Yes, but I reasoned that when inlined via JIT it's not used by a
> > sort_template.h
> >
> > --
> > David Geier
> >
> >
>
>
> Hi! I think there is a regression with numeric[]. Equal numeric values
> can have different scales, so e.g. 1, 1.0 have different byte images.
> The RB-tree accumulator combines them by comparator equality while the
> hash keeps a separate entry for each image.
>
> example (no pg_trgm extension required):
>
> SET max_parallel_maintenance_workers = 0;
> SET maintenance_work_mem = '256MB';
> CREATE TABLE numeric_many AS
> SELECT i, ARRAY[round((i%100)::numeric, ((i/100)%256)::int)] AS a
> FROM generate_series(1,1000000) AS i;
> VACUUM (FREEZE, ANALYZE) numeric_many;
> SELECT count(*), count(DISTINCT a[1]),
> count(DISTINCT encode(numeric_send(a[1]),'hex')) FROM numeric_many;
>
>
> in my run size of patch index is 7.3532338308457711x bigger and build
> time is 4.0799353308331456x longer
>
> --
> Best regards,
> Kirill Reshke

Another 0001 regression is for long text keys:

SET maintenance_work_mem = '256MB';
SET max_parallel_maintenance_workers = 0;
CREATE TABLE long_key AS
SELECT i, ARRAY[repeat('a',6000)] a FROM generate_series(1,1000000) i;
VACUUM (FREEZE,ANALYZE) long_key;
CREATE INDEX long_key_gin ON long_key USING gin(a);

in my run, slowdown is 1605.768 -> 705.046 ms

CPU profiles show datum_image_hash() -> hash_bytes() on top. The RB
tree use the fast memcmp, but hashing bytes is much more expensive:
each 12 bytes from input we perform mix marco, which is 6 calls of
`pg_rotate_left32`

0001 perf:

# Event count (approx.): 3621242456
#
# Overhead Symbol IPC [IPC Coverage]
# ........ ...................................... ....................
#
49.36% [.] hash_bytes - -
28.83% [.] __memmove_avx512_unaligned_erms - -
3.65% [.] pglz_decompress - -
2.82% [.] AssertBufferLocksPermitCatalogRead - -
2.16% [.] __memset_avx512_unaligned_erms - -
1.88% [.] __memcmp_evex_movbe - -
1.00% [.] AllocSetAllocFromNewBlock - -
0.77% [.] AllocSetCheck - -
0.77% [.] _int_malloc - -
0.55% [.] BufferLockAcquire - -
0.55% [.] verify_compact_attribute - -

master

# Samples: 851 of event 'cpu-clock:u'
# Event count (approx.): 1705410808
#
# Overhead Symbol IPC [IPC Coverage]
# ........ ...................................... ....................
#
54.29% [.] __memmove_avx512_unaligned_erms - -
8.11% [.] AssertBufferLocksPermitCatalogRead - -
7.05% [.] pglz_decompress - -
5.52% [.] __memcmp_evex_movbe - -
4.23% [.] __memset_avx512_unaligned_erms - -
2.47% [.] AllocSetCheck - -
1.06% [.] _int_malloc - -
0.82% [.] AllocSetAlloc - -
0.82% [.] AllocSetReset - -
0.82% [.] BufferLockAcquire - -
0.82% [.] _int_free_merge_chunk - -
0.82% [.] heapgettup - -
0.71% [.] AllocSetAllocFromNewBlock - -
0.71% [.] ginExtractEntries - -
0.71% [.] memcpy(at)plt - -
0.71% [.] tts_buffer_heap_getsomeattrs - -
0.59% [.] AssertNotCatalogBufferLock - -
0.59% [.] ginInsertBAEntries - -
0.59% [.] unlink_chunk.isra.0 - -

--
Best regards,
Kirill Reshke

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message shihao zhong 2026-11-10 03:23:22 Stored generated columns are NULL in transition tables of partitioned tables
Previous Message Amit Kapila 2026-11-10 02:57:29 Re: Proposal: Conflict log history table for Logical Replication