Detoast iterators - take 2

From: Nikita Malakhov <hukutoc(at)gmail(dot)com>
To: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Detoast iterators - take 2
Date: 2025-08-13 15:01:27
Message-ID: CAN-LCVNL_oqnMmBbbVsKynjOYtMa+7D8JqaskPPaMOtesdih9A@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi hackers!

Some time ago there was a discussion on detoast iterators [1].
The original thread stalled, but we took it, reviewed and refactored,
and used it for some of our ideas.

I've refactored it onto the current master and made some tests,
iterative detoast provides significant improvement over fully detoasting
a value in cases like position() function. Please check out a POC patch
attached.

Below is simple test with fixed value (storage set to external to eliminate
compression influence):
postgres(at)postgres=# create table t (id int, t text);
CREATE TABLE
Time: 7.609 ms
postgres(at)postgres=# alter table t alter column t set storage external;
ALTER TABLE
Time: 8.496 ms
postgres(at)postgres=# insert into t (select i, i::text || 'abc' ||
repeat('a', 10000000) from generate_series(1, 1000) as i);
INSERT 0 1000
Time: 153096.834 ms (02:33.097)

master:
postgres(at)postgres=# select position('abc' in t) from t where id=599;
position
----------
4
(1 row)

Time: 47.346 ms

patched:
postgres(at)postgres=# select position('abc' in t) from t where id=599;
position
----------
4
(1 row)

Time: 7.607 ms

7.6 ms over 47.3 seems quite good.

[1]
https://www.postgresql.org/message-id/flat/CAL-OGks_onzpc9M9bXPCztMofWULcFkyeCeKiAgXzwRL8kXiag%40mail.gmail.com
--
Regards,
Nikita Malakhov
Postgres Professional
The Russian Postgres Company
https://postgrespro.ru/

Attachment Content-Type Size
v1-0001-detoast-iterator.patch application/octet-stream 34.3 KB

Browse pgsql-hackers by date

  From Date Subject
Next Message Andres Freund 2025-08-13 15:16:24 Re: Adding basic NUMA awareness
Previous Message Andres Freund 2025-08-13 14:44:33 Re: index prefetching