[PG19] Hash join writes null-keyed outer rows to temp files

From: shihao zhong <zhong950419(at)gmail(dot)com>
To: pgsql-hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Subject: [PG19] Hash join writes null-keyed outer rows to temp files
Date: 2026-10-19 01:48:44
Message-ID: CAGRkXqTgugmYrtnP_11NjL_GCXwAfzVfm5WKUNFsfZOzxDF2rA@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi hackers,

Since 1811f1af98f, a hash join keeps outer rows with a NULL join key in
a tuplestore and returns them at the end of the batch. Once they pass
work_mem / 16 they go to disk, even for a single-batch join. 18 returned
them right away and never stored them.

With default settings:

CREATE TABLE b (k int PRIMARY KEY);
INSERT INTO b SELECT generate_series(1, 1000);
CREATE TABLE a AS SELECT NULL::int AS k, repeat('x', 100) AS pad
FROM generate_series(1, 1000000);
VACUUM ANALYZE a, b;
EXPLAIN (ANALYZE, BUFFERS, COSTS OFF, TIMING OFF)
SELECT count(b.k) FROM a LEFT JOIN b ON a.k = b.k;
SET temp_file_limit = '10MB';
SELECT count(b.k) FROM a LEFT JOIN b ON a.k = b.k;

On release builds 18 takes 22 ms with no temp files. 19 takes 110 ms
and writes 106 MB of temp files.

The original mail said returning them right away would need a big
rewrite [1]. The patch gives such a row a dummy hash value of 0 and
lets it probe as usual. That value always maps to the current batch,
and the row cannot match, so it comes out null-extended right away,
like in 18. Only the multi-batch parallel case still uses the
tuplestore, since the outer side is partitioned before probing.

Another way is to return such a row with a flag and jump straight to
HJ_FILL_OUTER_TUPLE, skipping the probe. That is twice the size, so I
went with the smaller one in the hope it can go into 19. I can post the
other one if people prefer it.

[1] https://postgr.es/m/3061845.1746486714@sss.pgh.pa.us

Thanks,
Shihao

Attachment Content-Type Size
v1-0001-Emit-null-keyed-outer-hash-join-tuples-immediatel.patch application/octet-stream 4.0 KB

Browse pgsql-hackers by date

  From Date Subject
Next Message Scott Ray 2026-10-19 02:02:18 Re: pg_xmin_horizon: a system view of everything pinning the xmin horizon
Previous Message Fujii Masao 2026-10-19 00:59:26 Re: [PG19] ILIKE index scan misses rows with a libc UTF-8 collation on macOS