Detoast iterators -take 2

From: Nikita Malakhov <hukutoc(at)gmail(dot)com>
To: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Detoast iterators -take 2
Date: 2025-08-26 07:41:53
Message-ID: CAN-LCVPgFFBFVjATy-guOkHevgeDuKpwTdMzR=AY6cgFDEp1Aw@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi hackers!

Some time ago there was a discussion on detoast iterators [1].
The original thread stalled, but we took it, reviewed and refactored,
and used it for some of our ideas.

I've refactored it onto the current master and made some tests,
iterative detoast provides significant improvement over fully detoasting
a value in cases like position() function. Please check out a POC patch
attached.

Below is simple test with fixed value (storage set to external to eliminate
compression influence):
postgres(at)postgres=# create table t (id int, t text);
CREATE TABLE
Time: 7.609 ms
postgres(at)postgres=# alter table t alter column t set storage external;
ALTER TABLE
Time: 8.496 ms
postgres(at)postgres=# insert into t (select i, i::text || 'abc' ||
repeat('a', 10000000) from generate_series(1, 1000) as i);
INSERT 0 1000
Time: 153096.834 ms (02:33.097)

master:
postgres(at)postgres=# select position('abc' in t) from t where id=599;
position
----------
4
(1 row)

Time: 47.346 ms

patched:
postgres(at)postgres=# select position('abc' in t) from t where id=599;
position
----------
4
(1 row)

Time: 7.607 ms

7.6 ms over 47.3 seems quite good.

[1]
https://www.postgresql.org/message-id/flat/CAL-OGks_onzpc9M9bXPCztMofWULcFkyeCeKiAgXzwRL8kXiag%40mail.gmail.com
--
Regards,
Nikita Malakhov
Postgres Professional
The Russian Postgres Company
https://postgrespro.ru/

Attachment Content-Type Size
v1-0001-detoast-iterator.patch application/octet-stream 34.3 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Daniel Gustafsson 2025-08-26 07:42:13 Re: Support getrandom() for pg_strong_random() source
Previous Message Nazir Bilal Yavuz 2025-08-26 07:27:27 Re: Explicitly enable meson features in CI