From 661830be323389e1be427269e718544cfea486c6 Mon Sep 17 00:00:00 2001 From: Tomas Vondra Date: Fri, 29 May 2026 16:29:07 +0200 Subject: [PATCH v1] PoC: hashjoin bloom filter pushdown When construction hashjoin plans, try to pushdown a Bloom filter built on the hashtable to a scan node in the outer side of the join. This has multiple significant benefits: a) Probing a bloom filter is cheaper than probing a hash table, so if a tuple gets eliminated using the bloom filter, we save cycles. b) The Bloom filter is more compact, and so more cache efficient. The hash table may not even split into memory, and the hashjoin has to spill data to files. The Bloom filter may still fit into memory, and eliminate many tuples on the outer side (which reduces the amount of data spilled to disk). c) The Bloom filter is pushed to a scan node, which may be multiple steps before the join. This increases the benefit, because the eliminated tuples don't need to pass through any of the nodes in between. This futehr amplifies the difference between probing a filter and probing the hashtable. d) If a table joins with multiple other tables (e.g. in a starjoin), the scan node may receive multiple Bloom filters. The selectivity of the filters multiply, once again amplifying the benefits. If a scan gets two filters, each discarding 90% of tuples, the scan will discard 99% of tuples, i.e. 2 orders of magnitude fewer tuples. This patch performs Bloom filter pushdown when constructing the plan, after path selection. That means the filters are not considered when estimating and costing the paths, and we only have a chance to do the pushdown if we happen to a pick a plan with a hashjoin. If the pushdown is what makes the plan fast (faster than plans without hashjoins), we may not pick it. With our bottom-up planning it's hard to do better. The decision which Bloom filters to build (and which scan nodes should evaluate them) happens in create_hashjoin_plan. This registers the filters in both the hashjoin and the recipient scan node, etc. Then at execution time, the Hash node builds the filter with the hashtable, and the scan node probes the Bloom filter similarly to evaluating the regular quals. How effective this is depends on how many tuples the filter eliminates. A highly selective filter (e.g. discarding >90% tuples) is going to be a win no matter what. But even a "poor" filter (e.g. discarding only 10% tuples) may still be a win, if the hashjoin has to perform batching, and thus spill data to disk. It's hard to know in advance which filters are selective. The patch has a simple adaptive logic, that disables filters that are not selective enough (too many probes find a match), and the enables the filter when the filter gets more selective. There's a number of open questions to solve: - Pushdown after path construction means we can't consider Bloom filters when costing the paths, and the cost are as if no tuples were eliminated by the scan node. Solving this with the bottom-up planning is unlikely, or would have disadvantages (e.g. would increase the number of paths we have to condsider). - The EXPLAIN ANALYZE output can be somewhat confusing/misleading. The path estimates don't consider how many rows may be eliminated by the filter. This may lead to huge differences between estimated and actual "rows" in the EXPLAIN output, even with perfect estimates. The EXPLAIN now includes information about Bloom filters (number of probes and number of discarded rows), but it's still hard to interpret. - We have little control over the Bloom filter parameters. The library picks most of the attributes on our behalf. Works reasonably well, but we may need to know e.g. false positive rate (if it gets too high, the filter becomes useless, and we should stop using it). - We only push filters to scan nodes, and only through some other nodes (e.g. through joins, sort, ...). We could expand this to also pushdown through aggregations, etc. - The patch does not support parallel queries. This can be addressed later, it's certainly doable. - Similarly, there's no support for partitioned tables (fixing this should be simpler than supporting parallel queries). - It might be interesting to allow the scan nodes to use the Bloom filters in other ways. E.g. it might push the filter to storage, or perhaps to remote node (with a ForeignScan), and let it do smart things with it. The storage might prefilter data, foreign server could filter data on the remote end. That'd require using some well defined and portable library for the filter. - The cost model determining which filters are effective is a bit crude and based on empirical observations. For example the thresholds used in the adaptive logic are somewhat arbitrary and need more thought. - We could push filters into other nodes, not just scans. Might be useful for more complex joins. --- .../pg_plan_advice/expected/join_order.out | 52 +- .../pg_plan_advice/expected/join_strategy.out | 20 +- .../pg_plan_advice/expected/partitionwise.out | 40 +- contrib/pg_plan_advice/expected/semijoin.out | 24 +- .../expected/pg_stash_advice.out | 54 ++- .../expected/level_tracking.out | 24 +- .../postgres_fdw/expected/postgres_fdw.out | 8 +- src/backend/commands/explain.c | 189 ++++++++ src/backend/executor/execUtils.c | 2 + src/backend/executor/nodeBitmapHeapscan.c | 3 + src/backend/executor/nodeHash.c | 60 +++ src/backend/executor/nodeHashjoin.c | 455 ++++++++++++++++++ src/backend/executor/nodeIndexonlyscan.c | 3 + src/backend/executor/nodeIndexscan.c | 3 + src/backend/executor/nodeSamplescan.c | 3 + src/backend/executor/nodeSeqscan.c | 3 + src/backend/executor/nodeTidrangescan.c | 3 + src/backend/executor/nodeTidscan.c | 3 + src/backend/lib/bloomfilter.c | 19 + src/backend/optimizer/path/costsize.c | 1 + src/backend/optimizer/plan/createplan.c | 299 ++++++++++++ src/backend/optimizer/plan/planner.c | 1 + src/backend/optimizer/plan/setrefs.c | 63 +++ src/backend/utils/misc/guc_parameters.dat | 7 + src/backend/utils/misc/postgresql.conf.sample | 1 + src/include/executor/execScan.h | 22 +- src/include/executor/executor.h | 12 + src/include/executor/nodeHashjoin.h | 9 + src/include/lib/bloomfilter.h | 2 + src/include/nodes/execnodes.h | 91 ++++ src/include/nodes/pathnodes.h | 3 + src/include/nodes/plannodes.h | 51 ++ src/include/optimizer/cost.h | 1 + src/test/regress/expected/aggregates.out | 8 +- src/test/regress/expected/eager_aggregate.out | 166 ++++++- src/test/regress/expected/join.out | 176 +++++-- src/test/regress/expected/join_hash.out | 28 +- src/test/regress/expected/merge.out | 20 +- src/test/regress/expected/misc_functions.out | 4 +- .../regress/expected/partition_aggregate.out | 34 +- src/test/regress/expected/partition_join.out | 452 ++++++++++++++--- src/test/regress/expected/predicate.out | 8 +- src/test/regress/expected/privileges.out | 4 +- src/test/regress/expected/returning.out | 10 +- src/test/regress/expected/rowsecurity.out | 2 + src/test/regress/expected/select_views.out | 2 + src/test/regress/expected/stats_ext.out | 4 +- src/test/regress/expected/subselect.out | 64 ++- src/test/regress/expected/sysviews.out | 3 +- src/test/regress/expected/updatable_views.out | 12 +- src/test/regress/expected/window.out | 4 +- src/test/regress/expected/with.out | 16 +- src/test/regress/sql/rowsecurity.sql | 3 + src/test/regress/sql/select_views.sql | 3 + 54 files changed, 2313 insertions(+), 241 deletions(-) diff --git a/contrib/pg_plan_advice/expected/join_order.out b/contrib/pg_plan_advice/expected/join_order.out index a5a9728e3fd..0e5f93a046f 100644 --- a/contrib/pg_plan_advice/expected/join_order.out +++ b/contrib/pg_plan_advice/expected/join_order.out @@ -27,17 +27,21 @@ SELECT * FROM jo_fact f LEFT JOIN jo_dim1 d1 ON f.dim1_id = d1.id LEFT JOIN jo_dim2 d2 ON f.dim2_id = d2.id WHERE val1 = 1 AND val2 = 1; - QUERY PLAN ------------------------------------------- + QUERY PLAN +---------------------------------------------- Hash Join Hash Cond: (f.dim1_id = d1.id) -> Hash Join Hash Cond: (f.dim2_id = d2.id) -> Seq Scan on jo_fact f + Bloom Filter 1: keys=(dim2_id) + Bloom Filter 2: keys=(dim1_id) -> Hash + Bloom Filter 1 -> Seq Scan on jo_dim2 d2 Filter: (val2 = 1) -> Hash + Bloom Filter 2 -> Seq Scan on jo_dim1 d1 Filter: (val1 = 1) Generated Plan Advice: @@ -45,7 +49,7 @@ SELECT * FROM jo_fact f HASH_JOIN(d2 d1) SEQ_SCAN(f d2 d1) NO_GATHER(f d1 d2) -(16 rows) +(20 rows) -- Force a few different join orders. Some of these are very inefficient, -- but the planner considers them all viable. @@ -56,17 +60,21 @@ SELECT * FROM jo_fact f LEFT JOIN jo_dim1 d1 ON f.dim1_id = d1.id LEFT JOIN jo_dim2 d2 ON f.dim2_id = d2.id WHERE val1 = 1 AND val2 = 1; - QUERY PLAN ------------------------------------------- + QUERY PLAN +---------------------------------------------- Hash Join Hash Cond: (f.dim2_id = d2.id) -> Hash Join Hash Cond: (f.dim1_id = d1.id) -> Seq Scan on jo_fact f + Bloom Filter 1: keys=(dim1_id) + Bloom Filter 2: keys=(dim2_id) -> Hash + Bloom Filter 1 -> Seq Scan on jo_dim1 d1 Filter: (val1 = 1) -> Hash + Bloom Filter 2 -> Seq Scan on jo_dim2 d2 Filter: (val2 = 1) Supplied Plan Advice: @@ -76,7 +84,7 @@ SELECT * FROM jo_fact f HASH_JOIN(d1 d2) SEQ_SCAN(f d1 d2) NO_GATHER(f d1 d2) -(18 rows) +(22 rows) SET LOCAL pg_plan_advice.advice = 'join_order(f d2 d1)'; EXPLAIN (COSTS OFF, PLAN_ADVICE) @@ -84,17 +92,21 @@ SELECT * FROM jo_fact f LEFT JOIN jo_dim1 d1 ON f.dim1_id = d1.id LEFT JOIN jo_dim2 d2 ON f.dim2_id = d2.id WHERE val1 = 1 AND val2 = 1; - QUERY PLAN ------------------------------------------- + QUERY PLAN +---------------------------------------------- Hash Join Hash Cond: (f.dim1_id = d1.id) -> Hash Join Hash Cond: (f.dim2_id = d2.id) -> Seq Scan on jo_fact f + Bloom Filter 1: keys=(dim2_id) + Bloom Filter 2: keys=(dim1_id) -> Hash + Bloom Filter 1 -> Seq Scan on jo_dim2 d2 Filter: (val2 = 1) -> Hash + Bloom Filter 2 -> Seq Scan on jo_dim1 d1 Filter: (val1 = 1) Supplied Plan Advice: @@ -104,7 +116,7 @@ SELECT * FROM jo_fact f HASH_JOIN(d2 d1) SEQ_SCAN(f d2 d1) NO_GATHER(f d1 d2) -(18 rows) +(22 rows) SET LOCAL pg_plan_advice.advice = 'join_order(d1 f d2)'; EXPLAIN (COSTS OFF, PLAN_ADVICE) @@ -120,7 +132,9 @@ SELECT * FROM jo_fact f Hash Cond: (d1.id = f.dim1_id) -> Seq Scan on jo_dim1 d1 Filter: (val1 = 1) + Bloom Filter 1: keys=(id) -> Hash + Bloom Filter 1 -> Seq Scan on jo_fact f -> Hash -> Seq Scan on jo_dim2 d2 @@ -132,7 +146,7 @@ SELECT * FROM jo_fact f HASH_JOIN(f d2) SEQ_SCAN(d1 f d2) NO_GATHER(f d1 d2) -(18 rows) +(20 rows) SET LOCAL pg_plan_advice.advice = 'join_order(f (d1 d2))'; EXPLAIN (COSTS OFF, PLAN_ADVICE) @@ -145,7 +159,9 @@ SELECT * FROM jo_fact f Hash Join Hash Cond: ((f.dim1_id = d1.id) AND (f.dim2_id = d2.id)) -> Seq Scan on jo_fact f + Bloom Filter 1: keys=(dim1_id, dim2_id) -> Hash + Bloom Filter 1 -> Nested Loop -> Seq Scan on jo_dim1 d1 Filter: (val1 = 1) @@ -160,7 +176,7 @@ SELECT * FROM jo_fact f HASH_JOIN((d1 d2)) SEQ_SCAN(f d1 d2) NO_GATHER(f d1 d2) -(18 rows) +(20 rows) SET LOCAL pg_plan_advice.advice = 'join_order(f {d1 d2})'; EXPLAIN (COSTS OFF, PLAN_ADVICE) @@ -173,7 +189,9 @@ SELECT * FROM jo_fact f Hash Join Hash Cond: ((f.dim1_id = d1.id) AND (f.dim2_id = d2.id)) -> Seq Scan on jo_fact f + Bloom Filter 1: keys=(dim1_id, dim2_id) -> Hash + Bloom Filter 1 -> Nested Loop -> Seq Scan on jo_dim1 d1 Filter: (val1 = 1) @@ -188,7 +206,7 @@ SELECT * FROM jo_fact f HASH_JOIN((d1 d2)) SEQ_SCAN(f d1 d2) NO_GATHER(f d1 d2) -(18 rows) +(20 rows) COMMIT; -- Force a join order by mentioning just a prefix of the join list. @@ -199,17 +217,21 @@ SELECT * FROM jo_fact f LEFT JOIN jo_dim1 d1 ON f.dim1_id = d1.id LEFT JOIN jo_dim2 d2 ON f.dim2_id = d2.id WHERE val1 = 1 AND val2 = 1; - QUERY PLAN ------------------------------------------------- + QUERY PLAN +---------------------------------------------------- Hash Join Hash Cond: (d2.id = f.dim2_id) -> Seq Scan on jo_dim2 d2 Filter: (val2 = 1) + Bloom Filter 2: keys=(id) -> Hash + Bloom Filter 2 -> Hash Join Hash Cond: (f.dim1_id = d1.id) -> Seq Scan on jo_fact f + Bloom Filter 1: keys=(dim1_id) -> Hash + Bloom Filter 1 -> Seq Scan on jo_dim1 d1 Filter: (val1 = 1) Supplied Plan Advice: @@ -219,7 +241,7 @@ SELECT * FROM jo_fact f HASH_JOIN(d1 (f d1)) SEQ_SCAN(d2 f d1) NO_GATHER(f d1 d2) -(18 rows) +(22 rows) SET LOCAL pg_plan_advice.advice = 'join_order(d2 d1)'; EXPLAIN (COSTS OFF, PLAN_ADVICE) diff --git a/contrib/pg_plan_advice/expected/join_strategy.out b/contrib/pg_plan_advice/expected/join_strategy.out index 0f9db692190..ce105856fda 100644 --- a/contrib/pg_plan_advice/expected/join_strategy.out +++ b/contrib/pg_plan_advice/expected/join_strategy.out @@ -15,19 +15,21 @@ VACUUM ANALYZE join_fact; -- We expect a hash join by default. EXPLAIN (COSTS OFF, PLAN_ADVICE) SELECT * FROM join_fact f JOIN join_dim d ON f.dim_id = d.id; - QUERY PLAN ------------------------------------- + QUERY PLAN +--------------------------------------- Hash Join Hash Cond: (f.dim_id = d.id) -> Seq Scan on join_fact f + Bloom Filter 1: keys=(dim_id) -> Hash + Bloom Filter 1 -> Seq Scan on join_dim d Generated Plan Advice: JOIN_ORDER(f d) HASH_JOIN(d) SEQ_SCAN(f d) NO_GATHER(f d) -(10 rows) +(12 rows) -- Try forcing each join method in turn with join_dim as the inner table. -- All of these should work except for MERGE_JOIN_MATERIALIZE; that will @@ -37,12 +39,14 @@ BEGIN; SET LOCAL pg_plan_advice.advice = 'HASH_JOIN(d)'; EXPLAIN (COSTS OFF, PLAN_ADVICE) SELECT * FROM join_fact f JOIN join_dim d ON f.dim_id = d.id; - QUERY PLAN ------------------------------------- + QUERY PLAN +--------------------------------------- Hash Join Hash Cond: (f.dim_id = d.id) -> Seq Scan on join_fact f + Bloom Filter 1: keys=(dim_id) -> Hash + Bloom Filter 1 -> Seq Scan on join_dim d Supplied Plan Advice: HASH_JOIN(d) /* matched */ @@ -51,7 +55,7 @@ EXPLAIN (COSTS OFF, PLAN_ADVICE) HASH_JOIN(d) SEQ_SCAN(f d) NO_GATHER(f d) -(12 rows) +(14 rows) SET LOCAL pg_plan_advice.advice = 'MERGE_JOIN_MATERIALIZE(d)'; EXPLAIN (COSTS OFF, PLAN_ADVICE) @@ -162,7 +166,9 @@ EXPLAIN (COSTS OFF, PLAN_ADVICE) Hash Join Hash Cond: (d.id = f.dim_id) -> Seq Scan on join_dim d + Bloom Filter 1: keys=(id) -> Hash + Bloom Filter 1 -> Seq Scan on join_fact f Supplied Plan Advice: HASH_JOIN(f) /* matched */ @@ -171,7 +177,7 @@ EXPLAIN (COSTS OFF, PLAN_ADVICE) HASH_JOIN(f) SEQ_SCAN(d f) NO_GATHER(f d) -(12 rows) +(14 rows) SET LOCAL pg_plan_advice.advice = 'MERGE_JOIN_MATERIALIZE(f)'; EXPLAIN (COSTS OFF, PLAN_ADVICE) diff --git a/contrib/pg_plan_advice/expected/partitionwise.out b/contrib/pg_plan_advice/expected/partitionwise.out index 2b3d0a82443..3b003a927ac 100644 --- a/contrib/pg_plan_advice/expected/partitionwise.out +++ b/contrib/pg_plan_advice/expected/partitionwise.out @@ -60,7 +60,9 @@ SELECT * FROM pt1, pt2, pt3 WHERE pt1.id = pt2.id AND pt2.id = pt3.id Hash Cond: (pt2_1.id = pt3_1.id) -> Seq Scan on pt2a pt2_1 Filter: (val2 = 1) + Bloom Filter 1: keys=(id) -> Hash + Bloom Filter 1 -> Seq Scan on pt3a pt3_1 Filter: (val3 = 1) -> Index Scan using pt1a_pkey on pt1a pt1_1 @@ -71,7 +73,9 @@ SELECT * FROM pt1, pt2, pt3 WHERE pt1.id = pt2.id AND pt2.id = pt3.id Hash Cond: (pt2_2.id = pt3_2.id) -> Seq Scan on pt2b pt2_2 Filter: (val2 = 1) + Bloom Filter 2: keys=(id) -> Hash + Bloom Filter 2 -> Seq Scan on pt3b pt3_2 Filter: (val3 = 1) -> Index Scan using pt1b_pkey on pt1b pt1_2 @@ -82,7 +86,9 @@ SELECT * FROM pt1, pt2, pt3 WHERE pt1.id = pt2.id AND pt2.id = pt3.id Hash Cond: (pt2_3.id = pt3_3.id) -> Seq Scan on pt2c pt2_3 Filter: (val2 = 1) + Bloom Filter 3: keys=(id) -> Hash + Bloom Filter 3 -> Seq Scan on pt3c pt3_3 Filter: (val3 = 1) -> Index Scan using pt1c_pkey on pt1c pt1_3 @@ -101,7 +107,7 @@ SELECT * FROM pt1, pt2, pt3 WHERE pt1.id = pt2.id AND pt2.id = pt3.id PARTITIONWISE((pt1 pt2 pt3)) NO_GATHER(pt1/public.pt1a pt1/public.pt1b pt1/public.pt1c pt2/public.pt2a pt2/public.pt2b pt2/public.pt2c pt3/public.pt3a pt3/public.pt3b pt3/public.pt3c) -(47 rows) +(53 rows) -- Suppress partitionwise join, or do it just partially. BEGIN; @@ -169,21 +175,27 @@ SELECT * FROM pt1, pt2, pt3 WHERE pt1.id = pt2.id AND pt2.id = pt3.id Hash Cond: (pt1_1.id = pt2_1.id) -> Seq Scan on pt1a pt1_1 Filter: (val1 = 1) + Bloom Filter 1: keys=(id) -> Hash + Bloom Filter 1 -> Seq Scan on pt2a pt2_1 Filter: (val2 = 1) -> Hash Join Hash Cond: (pt1_2.id = pt2_2.id) -> Seq Scan on pt1b pt1_2 Filter: (val1 = 1) + Bloom Filter 2: keys=(id) -> Hash + Bloom Filter 2 -> Seq Scan on pt2b pt2_2 Filter: (val2 = 1) -> Hash Join Hash Cond: (pt1_3.id = pt2_3.id) -> Seq Scan on pt1c pt1_3 Filter: (val1 = 1) + Bloom Filter 3: keys=(id) -> Hash + Bloom Filter 3 -> Seq Scan on pt2c pt2_3 Filter: (val2 = 1) -> Hash @@ -209,7 +221,7 @@ SELECT * FROM pt1, pt2, pt3 WHERE pt1.id = pt2.id AND pt2.id = pt3.id PARTITIONWISE((pt1 pt2) pt3) NO_GATHER(pt1/public.pt1a pt1/public.pt1b pt1/public.pt1c pt2/public.pt2a pt2/public.pt2b pt2/public.pt2c pt3/public.pt3a pt3/public.pt3b pt3/public.pt3c) -(47 rows) +(53 rows) COMMIT; -- Test conflicting advice. @@ -227,7 +239,9 @@ SELECT * FROM pt1, pt2, pt3 WHERE pt1.id = pt2.id AND pt2.id = pt3.id Hash Cond: (pt2_1.id = pt3_1.id) -> Seq Scan on pt2a pt2_1 Filter: (val2 = 1) + Bloom Filter 1: keys=(id) -> Hash + Bloom Filter 1 -> Seq Scan on pt3a pt3_1 Filter: (val3 = 1) -> Index Scan using pt1a_pkey on pt1a pt1_1 @@ -238,7 +252,9 @@ SELECT * FROM pt1, pt2, pt3 WHERE pt1.id = pt2.id AND pt2.id = pt3.id Hash Cond: (pt2_2.id = pt3_2.id) -> Seq Scan on pt2b pt2_2 Filter: (val2 = 1) + Bloom Filter 2: keys=(id) -> Hash + Bloom Filter 2 -> Seq Scan on pt3b pt3_2 Filter: (val3 = 1) -> Index Scan using pt1b_pkey on pt1b pt1_2 @@ -249,7 +265,9 @@ SELECT * FROM pt1, pt2, pt3 WHERE pt1.id = pt2.id AND pt2.id = pt3.id Hash Cond: (pt2_3.id = pt3_3.id) -> Seq Scan on pt2c pt2_3 Filter: (val2 = 1) + Bloom Filter 3: keys=(id) -> Hash + Bloom Filter 3 -> Seq Scan on pt3c pt3_3 Filter: (val3 = 1) -> Index Scan using pt1c_pkey on pt1c pt1_3 @@ -271,7 +289,7 @@ SELECT * FROM pt1, pt2, pt3 WHERE pt1.id = pt2.id AND pt2.id = pt3.id PARTITIONWISE((pt1 pt2 pt3)) NO_GATHER(pt1/public.pt1a pt1/public.pt1b pt1/public.pt1c pt2/public.pt2a pt2/public.pt2b pt2/public.pt2c pt3/public.pt3a pt3/public.pt3b pt3/public.pt3c) -(51 rows) +(57 rows) COMMIT; -- Can't force a partitionwise join with a mismatched table. @@ -321,7 +339,9 @@ SELECT * FROM pt1, pt2, pt3 WHERE pt1.id = pt2.id AND pt2.id = pt3.id Hash Cond: (pt3_1.id = pt2_1.id) -> Seq Scan on pt3a pt3_1 Filter: (val3 = 1) + Bloom Filter 1: keys=(id) -> Hash + Bloom Filter 1 -> Seq Scan on pt2a pt2_1 Filter: (val2 = 1) -> Index Scan using pt1a_pkey on pt1a pt1_1 @@ -332,7 +352,9 @@ SELECT * FROM pt1, pt2, pt3 WHERE pt1.id = pt2.id AND pt2.id = pt3.id Hash Cond: (pt2_2.id = pt3_2.id) -> Seq Scan on pt2b pt2_2 Filter: (val2 = 1) + Bloom Filter 2: keys=(id) -> Hash + Bloom Filter 2 -> Seq Scan on pt3b pt3_2 Filter: (val3 = 1) -> Index Scan using pt1b_pkey on pt1b pt1_2 @@ -343,7 +365,9 @@ SELECT * FROM pt1, pt2, pt3 WHERE pt1.id = pt2.id AND pt2.id = pt3.id Hash Cond: (pt2_3.id = pt3_3.id) -> Seq Scan on pt2c pt2_3 Filter: (val2 = 1) + Bloom Filter 3: keys=(id) -> Hash + Bloom Filter 3 -> Seq Scan on pt3c pt3_3 Filter: (val3 = 1) -> Index Scan using pt1c_pkey on pt1c pt1_3 @@ -364,7 +388,7 @@ SELECT * FROM pt1, pt2, pt3 WHERE pt1.id = pt2.id AND pt2.id = pt3.id PARTITIONWISE((pt1 pt2 pt3)) NO_GATHER(pt1/public.pt1a pt1/public.pt1b pt1/public.pt1c pt2/public.pt2a pt2/public.pt2b pt2/public.pt2c pt3/public.pt3a pt3/public.pt3b pt3/public.pt3c) -(49 rows) +(55 rows) SET LOCAL pg_plan_advice.advice = 'JOIN_ORDER(pt3/pt3a pt2/pt2a pt1/pt1a)'; EXPLAIN (PLAN_ADVICE, COSTS OFF) @@ -378,7 +402,9 @@ SELECT * FROM pt1, pt2, pt3 WHERE pt1.id = pt2.id AND pt2.id = pt3.id Hash Cond: (pt3_1.id = pt2_1.id) -> Seq Scan on pt3a pt3_1 Filter: (val3 = 1) + Bloom Filter 1: keys=(id) -> Hash + Bloom Filter 1 -> Seq Scan on pt2a pt2_1 Filter: (val2 = 1) -> Index Scan using pt1a_pkey on pt1a pt1_1 @@ -389,7 +415,9 @@ SELECT * FROM pt1, pt2, pt3 WHERE pt1.id = pt2.id AND pt2.id = pt3.id Hash Cond: (pt2_2.id = pt3_2.id) -> Seq Scan on pt2b pt2_2 Filter: (val2 = 1) + Bloom Filter 2: keys=(id) -> Hash + Bloom Filter 2 -> Seq Scan on pt3b pt3_2 Filter: (val3 = 1) -> Index Scan using pt1b_pkey on pt1b pt1_2 @@ -400,7 +428,9 @@ SELECT * FROM pt1, pt2, pt3 WHERE pt1.id = pt2.id AND pt2.id = pt3.id Hash Cond: (pt2_3.id = pt3_3.id) -> Seq Scan on pt2c pt2_3 Filter: (val2 = 1) + Bloom Filter 3: keys=(id) -> Hash + Bloom Filter 3 -> Seq Scan on pt3c pt3_3 Filter: (val3 = 1) -> Index Scan using pt1c_pkey on pt1c pt1_3 @@ -421,6 +451,6 @@ SELECT * FROM pt1, pt2, pt3 WHERE pt1.id = pt2.id AND pt2.id = pt3.id PARTITIONWISE((pt1 pt2 pt3)) NO_GATHER(pt1/public.pt1a pt1/public.pt1b pt1/public.pt1c pt2/public.pt2a pt2/public.pt2b pt2/public.pt2c pt3/public.pt3a pt3/public.pt3b pt3/public.pt3c) -(49 rows) +(55 rows) COMMIT; diff --git a/contrib/pg_plan_advice/expected/semijoin.out b/contrib/pg_plan_advice/expected/semijoin.out index db6b069ec8e..f60778d0d38 100644 --- a/contrib/pg_plan_advice/expected/semijoin.out +++ b/contrib/pg_plan_advice/expected/semijoin.out @@ -75,7 +75,9 @@ SELECT * FROM sj_wide Hash Semi Join Hash Cond: ((sj_wide.id = "*VALUES*".column1) AND (sj_wide.val1 = "*VALUES*".column2)) -> Seq Scan on sj_wide + Bloom Filter 1: keys=(id, val1) -> Hash + Bloom Filter 1 -> Values Scan on "*VALUES*" Supplied Plan Advice: SEMIJOIN_NON_UNIQUE("*VALUES*") /* matched */ @@ -85,7 +87,7 @@ SELECT * FROM sj_wide SEQ_SCAN(sj_wide) SEMIJOIN_NON_UNIQUE("*VALUES*") NO_GATHER(sj_wide "*VALUES*") -(13 rows) +(15 rows) COMMIT; -- Because this table is narrower than the previous one, a sequential scan @@ -100,7 +102,9 @@ SELECT * FROM sj_narrow Hash Semi Join Hash Cond: ((sj_narrow.id = "*VALUES*".column1) AND (sj_narrow.val1 = "*VALUES*".column2)) -> Seq Scan on sj_narrow + Bloom Filter 1: keys=(id, val1) -> Hash + Bloom Filter 1 -> Values Scan on "*VALUES*" Generated Plan Advice: JOIN_ORDER(sj_narrow "*VALUES*") @@ -108,7 +112,7 @@ SELECT * FROM sj_narrow SEQ_SCAN(sj_narrow) SEMIJOIN_NON_UNIQUE("*VALUES*") NO_GATHER(sj_narrow "*VALUES*") -(11 rows) +(13 rows) -- Here, we expect advising a unique semijoin to swith to the same plan that -- we got with sj_wide, and advising a non-unique semijoin should not change @@ -123,7 +127,9 @@ SELECT * FROM sj_narrow Hash Join Hash Cond: ((sj_narrow.id = "*VALUES*".column1) AND (sj_narrow.val1 = "*VALUES*".column2)) -> Seq Scan on sj_narrow + Bloom Filter 1: keys=(id, val1) -> Hash + Bloom Filter 1 -> HashAggregate Group Key: "*VALUES*".column1, "*VALUES*".column2 -> Values Scan on "*VALUES*" @@ -135,7 +141,7 @@ SELECT * FROM sj_narrow SEQ_SCAN(sj_narrow) SEMIJOIN_UNIQUE("*VALUES*") NO_GATHER(sj_narrow "*VALUES*") -(15 rows) +(17 rows) SET LOCAL pg_plan_advice.advice = 'semijoin_non_unique("*VALUES*")'; EXPLAIN (COSTS OFF, PLAN_ADVICE) @@ -146,7 +152,9 @@ SELECT * FROM sj_narrow Hash Semi Join Hash Cond: ((sj_narrow.id = "*VALUES*".column1) AND (sj_narrow.val1 = "*VALUES*".column2)) -> Seq Scan on sj_narrow + Bloom Filter 1: keys=(id, val1) -> Hash + Bloom Filter 1 -> Values Scan on "*VALUES*" Supplied Plan Advice: SEMIJOIN_NON_UNIQUE("*VALUES*") /* matched */ @@ -156,7 +164,7 @@ SELECT * FROM sj_narrow SEQ_SCAN(sj_narrow) SEMIJOIN_NON_UNIQUE("*VALUES*") NO_GATHER(sj_narrow "*VALUES*") -(13 rows) +(15 rows) COMMIT; -- In the above example, we made the outer side of the join unique, but here, @@ -261,7 +269,9 @@ SELECT * FROM generate_series(1,1000) g Hash Right Semi Join Hash Cond: (sj_narrow.val1 = g.g) -> Seq Scan on sj_narrow + Bloom Filter 1: keys=(val1) -> Hash + Bloom Filter 1 -> Function Scan on generate_series g Supplied Plan Advice: SEMIJOIN_NON_UNIQUE(sj_narrow) /* matched */ @@ -272,7 +282,7 @@ SELECT * FROM generate_series(1,1000) g SEQ_SCAN(sj_narrow) SEMIJOIN_NON_UNIQUE(sj_narrow) NO_GATHER(g sj_narrow) -(14 rows) +(16 rows) COMMIT; -- However, mentioning the wrong side of the join should result in an advice @@ -407,11 +417,13 @@ SELECT 1 FROM generate_series(1, 1000) g WHERE EXISTS -> Unique -> Nested Loop -> Index Only Scan using sj_narrow_pkey on sj_narrow t2 + Bloom Filter 1: keys=(id) -> Materialize -> Nested Loop Left Join -> Result -> Seq Scan on sj_narrow -> Hash + Bloom Filter 1 -> Function Scan on generate_series g Generated Plan Advice: JOIN_ORDER(t2 ("*RESULT*" sj_narrow) g) @@ -422,5 +434,5 @@ SELECT 1 FROM generate_series(1, 1000) g WHERE EXISTS INDEX_ONLY_SCAN(t2 public.sj_narrow_pkey) SEMIJOIN_UNIQUE((t2 sj_narrow "*RESULT*")) NO_GATHER(g t2 sj_narrow "*RESULT*") -(20 rows) +(22 rows) diff --git a/contrib/pg_stash_advice/expected/pg_stash_advice.out b/contrib/pg_stash_advice/expected/pg_stash_advice.out index 788da854aa7..d62afaa6651 100644 --- a/contrib/pg_stash_advice/expected/pg_stash_advice.out +++ b/contrib/pg_stash_advice/expected/pg_stash_advice.out @@ -57,20 +57,24 @@ EXPLAIN (COSTS OFF) SELECT * FROM aa_fact f LEFT JOIN aa_dim1 d1 ON f.dim1_id = d1.id LEFT JOIN aa_dim2 d2 ON f.dim2_id = d2.id WHERE val1 = 1 AND val2 = 1; - QUERY PLAN ------------------------------------------- + QUERY PLAN +---------------------------------------------- Hash Join Hash Cond: (f.dim1_id = d1.id) -> Hash Join Hash Cond: (f.dim2_id = d2.id) -> Seq Scan on aa_fact f + Bloom Filter 1: keys=(dim2_id) + Bloom Filter 2: keys=(dim1_id) -> Hash + Bloom Filter 1 -> Seq Scan on aa_dim2 d2 Filter: (val2 = 1) -> Hash + Bloom Filter 2 -> Seq Scan on aa_dim1 d1 Filter: (val1 = 1) -(11 rows) +(15 rows) -- Force an index scan on dim1 SELECT pg_set_stashed_advice('regress_stash', :'qid', @@ -91,15 +95,19 @@ EXPLAIN (COSTS OFF) SELECT * FROM aa_fact f -> Hash Join Hash Cond: (f.dim2_id = d2.id) -> Seq Scan on aa_fact f + Bloom Filter 1: keys=(dim2_id) + Bloom Filter 2: keys=(dim1_id) -> Hash + Bloom Filter 1 -> Seq Scan on aa_dim2 d2 Filter: (val2 = 1) -> Hash + Bloom Filter 2 -> Index Scan using aa_dim1_pkey on aa_dim1 d1 Filter: (val1 = 1) Supplied Plan Advice: INDEX_SCAN(d1 aa_dim1_pkey) /* matched */ -(13 rows) +(17 rows) -- Force an alternative join order SELECT pg_set_stashed_advice('regress_stash', :'qid', @@ -113,22 +121,26 @@ EXPLAIN (COSTS OFF) SELECT * FROM aa_fact f LEFT JOIN aa_dim1 d1 ON f.dim1_id = d1.id LEFT JOIN aa_dim2 d2 ON f.dim2_id = d2.id WHERE val1 = 1 AND val2 = 1; - QUERY PLAN ------------------------------------------- + QUERY PLAN +---------------------------------------------- Hash Join Hash Cond: (f.dim2_id = d2.id) -> Hash Join Hash Cond: (f.dim1_id = d1.id) -> Seq Scan on aa_fact f + Bloom Filter 1: keys=(dim1_id) + Bloom Filter 2: keys=(dim2_id) -> Hash + Bloom Filter 1 -> Seq Scan on aa_dim1 d1 Filter: (val1 = 1) -> Hash + Bloom Filter 2 -> Seq Scan on aa_dim2 d2 Filter: (val2 = 1) Supplied Plan Advice: JOIN_ORDER(f d1 d2) /* matched */ -(13 rows) +(17 rows) -- Force an alternative join strategy SELECT pg_set_stashed_advice('regress_stash', :'qid', @@ -148,7 +160,9 @@ EXPLAIN (COSTS OFF) SELECT * FROM aa_fact f -> Hash Join Hash Cond: (f.dim2_id = d2.id) -> Seq Scan on aa_fact f + Bloom Filter 1: keys=(dim2_id) -> Hash + Bloom Filter 1 -> Seq Scan on aa_dim2 d2 Filter: (val2 = 1) -> Index Scan using aa_dim1_pkey on aa_dim1 d1 @@ -156,7 +170,7 @@ EXPLAIN (COSTS OFF) SELECT * FROM aa_fact f Filter: (val1 = 1) Supplied Plan Advice: NESTED_LOOP_PLAIN(d1) /* matched */ -(12 rows) +(14 rows) -- Add a useless extra entry to our test stash. Shouldn't change the result -- from the previous test. @@ -178,7 +192,9 @@ EXPLAIN (COSTS OFF) SELECT * FROM aa_fact f -> Hash Join Hash Cond: (f.dim2_id = d2.id) -> Seq Scan on aa_fact f + Bloom Filter 1: keys=(dim2_id) -> Hash + Bloom Filter 1 -> Seq Scan on aa_dim2 d2 Filter: (val2 = 1) -> Index Scan using aa_dim1_pkey on aa_dim1 d1 @@ -186,7 +202,7 @@ EXPLAIN (COSTS OFF) SELECT * FROM aa_fact f Filter: (val1 = 1) Supplied Plan Advice: NESTED_LOOP_PLAIN(d1) /* matched */ -(12 rows) +(14 rows) -- Try an empty stash to be sure it does nothing SELECT pg_create_advice_stash('regress_empty_stash'); @@ -200,20 +216,24 @@ EXPLAIN (COSTS OFF) SELECT * FROM aa_fact f LEFT JOIN aa_dim1 d1 ON f.dim1_id = d1.id LEFT JOIN aa_dim2 d2 ON f.dim2_id = d2.id WHERE val1 = 1 AND val2 = 1; - QUERY PLAN ------------------------------------------- + QUERY PLAN +---------------------------------------------- Hash Join Hash Cond: (f.dim1_id = d1.id) -> Hash Join Hash Cond: (f.dim2_id = d2.id) -> Seq Scan on aa_fact f + Bloom Filter 1: keys=(dim2_id) + Bloom Filter 2: keys=(dim1_id) -> Hash + Bloom Filter 1 -> Seq Scan on aa_dim2 d2 Filter: (val2 = 1) -> Hash + Bloom Filter 2 -> Seq Scan on aa_dim1 d1 Filter: (val1 = 1) -(11 rows) +(15 rows) -- Test that we can list each stash individually and all of them together, -- but not a nonexistent stash. @@ -263,20 +283,24 @@ EXPLAIN (COSTS OFF) SELECT * FROM aa_fact f LEFT JOIN aa_dim1 d1 ON f.dim1_id = d1.id LEFT JOIN aa_dim2 d2 ON f.dim2_id = d2.id WHERE val1 = 1 AND val2 = 1; - QUERY PLAN ------------------------------------------- + QUERY PLAN +---------------------------------------------- Hash Join Hash Cond: (f.dim1_id = d1.id) -> Hash Join Hash Cond: (f.dim2_id = d2.id) -> Seq Scan on aa_fact f + Bloom Filter 1: keys=(dim2_id) + Bloom Filter 2: keys=(dim1_id) -> Hash + Bloom Filter 1 -> Seq Scan on aa_dim2 d2 Filter: (val2 = 1) -> Hash + Bloom Filter 2 -> Seq Scan on aa_dim1 d1 Filter: (val1 = 1) -(11 rows) +(15 rows) SELECT * FROM pg_get_advice_stashes() ORDER BY stash_name; stash_name | num_entries diff --git a/contrib/pg_stat_statements/expected/level_tracking.out b/contrib/pg_stat_statements/expected/level_tracking.out index 832d65e97ca..db84cc6af01 100644 --- a/contrib/pg_stat_statements/expected/level_tracking.out +++ b/contrib/pg_stat_statements/expected/level_tracking.out @@ -189,9 +189,11 @@ EXPLAIN (COSTS OFF) MERGE INTO stats_track_tab -> Hash Right Join Hash Cond: (stats_track_tab.x = id.id) -> Seq Scan on stats_track_tab + Bloom Filter 1: keys=(x) -> Hash + Bloom Filter 1 -> Function Scan on generate_series id -(6 rows) +(8 rows) EXPLAIN (COSTS OFF) SELECT 1 UNION SELECT 2; QUERY PLAN @@ -315,9 +317,11 @@ EXPLAIN (COSTS OFF) MERGE INTO stats_track_tab -> Hash Right Join Hash Cond: (stats_track_tab.x = id.id) -> Seq Scan on stats_track_tab + Bloom Filter 1: keys=(x) -> Hash + Bloom Filter 1 -> Function Scan on generate_series id -(6 rows) +(8 rows) EXPLAIN (COSTS OFF) SELECT 1 UNION SELECT 2; QUERY PLAN @@ -536,9 +540,11 @@ EXPLAIN (COSTS OFF) MERGE INTO stats_track_tab -> Hash Right Join Hash Cond: (stats_track_tab.x = id.id) -> Seq Scan on stats_track_tab + Bloom Filter 1: keys=(x) -> Hash + Bloom Filter 1 -> Function Scan on generate_series id -(6 rows) +(8 rows) QUERY PLAN ------------ @@ -664,9 +670,11 @@ EXPLAIN (COSTS OFF) MERGE INTO stats_track_tab USING (SELECT id FROM generate_se -> Hash Right Join Hash Cond: (stats_track_tab.x = id.id) -> Seq Scan on stats_track_tab + Bloom Filter 1: keys=(x) -> Hash + Bloom Filter 1 -> Function Scan on generate_series id -(6 rows) +(8 rows) QUERY PLAN ------------ @@ -772,9 +780,11 @@ EXPLAIN (COSTS OFF) WITH a AS (SELECT 4) MERGE INTO stats_track_tab -> Hash Right Join Hash Cond: (stats_track_tab.x = id.id) -> Seq Scan on stats_track_tab + Bloom Filter 1: keys=(x) -> Hash + Bloom Filter 1 -> Function Scan on generate_series id -(6 rows) +(8 rows) EXPLAIN (COSTS OFF) WITH a AS (select 4) SELECT 1 UNION SELECT 2; QUERY PLAN @@ -866,9 +876,11 @@ EXPLAIN (COSTS OFF) WITH a AS (SELECT 4) MERGE INTO stats_track_tab -> Hash Right Join Hash Cond: (stats_track_tab.x = id.id) -> Seq Scan on stats_track_tab + Bloom Filter 1: keys=(x) -> Hash + Bloom Filter 1 -> Function Scan on generate_series id -(6 rows) +(8 rows) EXPLAIN (COSTS OFF) WITH a AS (select 4) SELECT 1 UNION SELECT 2; QUERY PLAN diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out index e90289e4ab1..d24b3a7c375 100644 --- a/contrib/postgres_fdw/expected/postgres_fdw.out +++ b/contrib/postgres_fdw/expected/postgres_fdw.out @@ -11723,12 +11723,14 @@ INSERT INTO join_tbl SELECT * FROM async_pt t1, async_pt t2 WHERE t1.a = t2.a AN Hash Cond: ((t2_3.a = t1_3.a) AND (t2_3.b = t1_3.b)) -> Seq Scan on public.async_p3 t2_3 Output: t2_3.a, t2_3.b, t2_3.c + Bloom Filter 1: keys=(t2_3.a, t2_3.b) -> Hash Output: t1_3.a, t1_3.b, t1_3.c + Bloom Filter 1 -> Seq Scan on public.async_p3 t1_3 Output: t1_3.a, t1_3.b, t1_3.c Filter: ((t1_3.b % 100) = 0) -(20 rows) +(22 rows) INSERT INTO join_tbl SELECT * FROM async_pt t1, async_pt t2 WHERE t1.a = t2.a AND t1.b = t2.b AND t1.b % 100 = 0; SELECT * FROM join_tbl ORDER BY a1; @@ -11786,12 +11788,14 @@ INSERT INTO join_tbl SELECT t1.a, t1.b, 'AAA' || t1.c, t2.a, t2.b, 'AAA' || t2.c Hash Cond: ((t2_3.a = t1_3.a) AND (t2_3.b = t1_3.b)) -> Seq Scan on public.async_p3 t2_3 Output: t2_3.a, t2_3.b, t2_3.c + Bloom Filter 1: keys=(t2_3.a, t2_3.b) -> Hash Output: t1_3.a, t1_3.b, t1_3.c + Bloom Filter 1 -> Seq Scan on public.async_p3 t1_3 Output: t1_3.a, t1_3.b, t1_3.c Filter: ((t1_3.b % 100) = 0) -(20 rows) +(22 rows) INSERT INTO join_tbl SELECT t1.a, t1.b, 'AAA' || t1.c, t2.a, t2.b, 'AAA' || t2.c FROM async_pt t1, async_pt t2 WHERE t1.a = t2.a AND t1.b = t2.b AND t1.b % 100 = 0; SELECT * FROM join_tbl ORDER BY a1; diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c index 112c17b0d64..5ae6de5b6fa 100644 --- a/src/backend/commands/explain.c +++ b/src/backend/commands/explain.c @@ -25,6 +25,7 @@ #include "commands/prepare.h" #include "foreign/fdwapi.h" #include "jit/jit.h" +#include "lib/bloomfilter.h" #include "libpq/pqformat.h" #include "libpq/protocol.h" #include "nodes/extensible.h" @@ -92,6 +93,8 @@ static void show_scan_qual(List *qual, const char *qlabel, static void show_upper_qual(List *qual, const char *qlabel, PlanState *planstate, List *ancestors, ExplainState *es); +static void show_bloom_filter_info(PlanState *planstate, List *ancestors, + ExplainState *es); static void show_sort_keys(SortState *sortstate, List *ancestors, ExplainState *es); static void show_incremental_sort_keys(IncrementalSortState *incrsortstate, @@ -1978,6 +1981,7 @@ ExplainNode(PlanState *planstate, List *ancestors, if (plan->qual) show_instrumentation_count("Rows Removed by Filter", 1, planstate, es); + show_bloom_filter_info(planstate, ancestors, es); show_indexsearches_info(planstate, es); break; case T_IndexOnlyScan: @@ -1995,6 +1999,7 @@ ExplainNode(PlanState *planstate, List *ancestors, if (es->analyze) ExplainPropertyFloat("Heap Fetches", NULL, planstate->instrument->ntuples2, 0, es); + show_bloom_filter_info(planstate, ancestors, es); show_indexsearches_info(planstate, es); break; case T_BitmapIndexScan: @@ -2012,6 +2017,7 @@ ExplainNode(PlanState *planstate, List *ancestors, if (plan->qual) show_instrumentation_count("Rows Removed by Filter", 1, planstate, es); + show_bloom_filter_info(planstate, ancestors, es); show_tidbitmap_info((BitmapHeapScanState *) planstate, es); show_scan_io_usage((ScanState *) planstate, es); break; @@ -2030,6 +2036,7 @@ ExplainNode(PlanState *planstate, List *ancestors, if (plan->qual) show_instrumentation_count("Rows Removed by Filter", 1, planstate, es); + show_bloom_filter_info(planstate, ancestors, es); if (IsA(plan, CteScan)) show_ctescan_info(castNode(CteScanState, planstate), es); show_scan_io_usage((ScanState *) planstate, es); @@ -2132,6 +2139,7 @@ ExplainNode(PlanState *planstate, List *ancestors, if (plan->qual) show_instrumentation_count("Rows Removed by Filter", 1, planstate, es); + show_bloom_filter_info(planstate, ancestors, es); } break; case T_TidRangeScan: @@ -2149,6 +2157,7 @@ ExplainNode(PlanState *planstate, List *ancestors, if (plan->qual) show_instrumentation_count("Rows Removed by Filter", 1, planstate, es); + show_bloom_filter_info(planstate, ancestors, es); show_scan_io_usage((ScanState *) planstate, es); } break; @@ -2578,6 +2587,120 @@ show_upper_qual(List *qual, const char *qlabel, show_qual(qual, qlabel, planstate, ancestors, useprefix, es); } +/* + * show_bloom_filter_info + * Show info about every bloom filter pushed down to a scan node. + * + * In TEXT format each filter is rendered on a single line, e.g. + * + * Bloom Filter N: (a, b) producer=3 checked=99999 rejected=99990 + * + * The checked/rejected fields are omitted outside of ANALYZE). In + * structured formats we emit a group per filter with the same fields + * broken out as properties. + * + * Called from the per-recipient cases in ExplainNode; the recipient is + * expected to be a scan node in the current PoC, but nothing here + * depends on that. + */ +static void +show_bloom_filter_info(PlanState *planstate, List *ancestors, + ExplainState *es) +{ + Plan *plan = planstate->plan; + ListCell *lc1, + *lc2; + List *deparse_cxt = NIL; + bool useprefix = false; + + if (plan->bloom_filters == NIL) + return; + + deparse_cxt = set_deparse_context_plan(es->deparse_cxt, + plan, ancestors); + useprefix = (IsA(plan, SubqueryScan) || es->verbose); + + if (es->format != EXPLAIN_FORMAT_TEXT) + ExplainOpenGroup("Bloom Filters", "Bloom Filters", false, es); + + /* print info about all the bloom filters */ + forboth(lc1, plan->bloom_filters, + lc2, planstate->bloom_filters) + { + BloomFilter *bf = lfirst_node(BloomFilter, lc1); + BloomFilterState *bfs = (BloomFilterState *) lfirst(lc2); + ListCell *lc; + StringInfoData keys; + bool first = true; + + initStringInfo(&keys); + + /* deparse the filter expressions */ + appendStringInfoChar(&keys, '('); + foreach(lc, bf->filter_exprs) + { + char *key; + Node *expr = (Node *) lfirst(lc); + + if (!first) + appendStringInfoString(&keys, ", "); + key = deparse_expression(expr, deparse_cxt, + useprefix, false); + appendStringInfoString(&keys, key); + first = false; + } + appendStringInfoChar(&keys, ')'); + + if (es->format == EXPLAIN_FORMAT_TEXT) + { + StringInfoData buf; + + initStringInfo(&buf); + + /* show the filter ID only when there are more filters */ + appendStringInfo(&buf, "Bloom Filter %d: keys=%s", + bf->producer_id, keys.data); + + /* include the counts only during ANALYZE */ + if (es->analyze && bfs != NULL) + { + /* rejected fraction */ + double frac = 100.0 * bfs->rejected / Max(1, bfs->checked); + + appendStringInfo(&buf, + " checked=" UINT64_FORMAT + " rejected=" UINT64_FORMAT + " (%.1f%%)", + bfs->checked, bfs->rejected, frac); + } + + ExplainIndentText(es); + appendStringInfoString(es->str, buf.data); + appendStringInfoChar(es->str, '\n'); + pfree(buf.data); + } + else /* non-text format */ + { + ExplainOpenGroup("Bloom Filter", NULL, true, es); + ExplainPropertyText("Keys", keys.data, es); + ExplainPropertyInteger("ID", NULL, bf->producer_id, es); + if (es->analyze && bfs != NULL) + { + ExplainPropertyFloat("Checked", NULL, + (double) bfs->checked, 0, es); + ExplainPropertyFloat("Rejected", NULL, + (double) bfs->rejected, 0, es); + } + ExplainCloseGroup("Bloom Filter", NULL, true, es); + } + + pfree(keys.data); + } + + if (es->format != EXPLAIN_FORMAT_TEXT) + ExplainCloseGroup("Bloom Filters", "Bloom Filters", false, es); +} + /* * Show the sort keys for a Sort node. */ @@ -3474,6 +3597,72 @@ show_hash_info(HashState *hashstate, ExplainState *es) spacePeakKb); } } + + /* + * Show infromation about the bloom filter produced by this Hash node + * (if any). For plain EXPLAIN, the filter is not initialized / built, + * but we still show available plan-time metadata (at least the ID). + */ + if (hashstate->bloom_filter_id > 0) + { + int producer_id = hashstate->bloom_filter_id; + uint64 nbits = 0; + int nhashfns = 0; + uint64 bytes = 0; + + if (hashstate->bloom_filter != NULL) + { + nbits = bloom_total_bits(hashstate->bloom_filter); + nhashfns = bloom_hash_funcs(hashstate->bloom_filter); + bytes = nbits / 8; + } + + if (es->format == EXPLAIN_FORMAT_TEXT) + { + ExplainIndentText(es); + if (hashstate->bloom_filter != NULL) + appendStringInfo(es->str, + "Bloom Filter %d: bits=" UINT64_FORMAT + " hashes=%d memory=" UINT64_FORMAT "kB" + " checked=" UINT64_FORMAT " rejected=" UINT64_FORMAT "\n", + producer_id, + nbits, + nhashfns, + BYTES_TO_KILOBYTES(bytes), + hashstate->bloomFilterChecked, + hashstate->bloomFilterRejected); + else if (es->analyze) + appendStringInfo(es->str, + "Bloom Filter %d: (not initialized)\n", + producer_id); + else + appendStringInfo(es->str, + "Bloom Filter %d\n", + producer_id); + } + else + { + /* there can be just one bloom filter per fproducer (for now) */ + ExplainOpenGroup("Bloom Filter", "Bloom Filter", true, es); + + ExplainPropertyInteger("Producer", NULL, producer_id, es); + ExplainPropertyBool("Initialized", + (hashstate->bloom_filter != NULL), es); + + if (hashstate->bloom_filter != NULL) + { + ExplainPropertyUInteger("Bits", NULL, nbits, es); + ExplainPropertyInteger("Hash Functions", NULL, nhashfns, es); + ExplainPropertyUInteger("Memory Usage", "kB", + BYTES_TO_KILOBYTES(bytes), es); + ExplainPropertyFloat("Checked", NULL, + (double) hashstate->bloomFilterChecked, 0, es); + ExplainPropertyFloat("Rejected", NULL, + (double) hashstate->bloomFilterRejected, 0, es); + } + ExplainCloseGroup("Bloom Filter", "Bloom Filter", true, es); + } + } } /* diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c index 1eb6b9f1f40..3c32a61dddd 100644 --- a/src/backend/executor/execUtils.c +++ b/src/backend/executor/execUtils.c @@ -159,6 +159,8 @@ CreateExecutorState(void) estate->es_auxmodifytables = NIL; + estate->es_bloom_producers = NIL; + estate->es_per_tuple_exprcontext = NULL; estate->es_sourceText = NULL; diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c index 83d6478bc2b..e71a47b6205 100644 --- a/src/backend/executor/nodeBitmapHeapscan.c +++ b/src/backend/executor/nodeBitmapHeapscan.c @@ -456,6 +456,9 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags) scanstate->bitmapqualorig = ExecInitQual(node->bitmapqualorig, (PlanState *) scanstate); + ExecInitBloomFilters((PlanState *) scanstate, + scanstate->ss.ss_ScanTupleSlot); + scanstate->ss.ss_currentRelation = currentRelation; /* diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c index 8825bb6fa23..37224324bce 100644 --- a/src/backend/executor/nodeHash.c +++ b/src/backend/executor/nodeHash.c @@ -35,6 +35,7 @@ #include "executor/instrument.h" #include "executor/nodeHash.h" #include "executor/nodeHashjoin.h" +#include "lib/bloomfilter.h" #include "miscadmin.h" #include "port/pg_bitutils.h" #include "utils/lsyscache.h" @@ -184,6 +185,18 @@ MultiExecPrivateHash(HashState *node) uint32 hashvalue = DatumGetUInt32(hashdatum); int bucketNumber; + /* + * Add the tuple to the pushed-down bloom filter (if any). Do + * it here (rather than in ExecHashTableInsert) so that each + * tuple is added exactly once, even if it later gets shuffled + * between batches by ExecHashIncreaseNumBatches. The filter + * would still produce the same matches, but it costs CPU. + */ + if (node->bloom_filter != NULL) + bloom_add_element(node->bloom_filter, + (unsigned char *) &hashvalue, + sizeof(hashvalue)); + bucketNumber = ExecHashGetSkewBucket(hashtable, hashvalue); if (bucketNumber != INVALID_SKEW_BUCKET_NO) { @@ -665,6 +678,53 @@ ExecHashTableCreate(HashState *state) MemoryContextSwitchTo(oldcxt); } + /* + * If we managed to push down a bloom filter to the outer side of the + * hash join, allocate it with the hash table. + * + * Whether we build the filter is decided by try_push_bloom_filter at + * plan time. If there's no recipient node, or when the GUC is set to + * off, state->want_bloom_filter is false. + * + * XXX We don't do this for parallel hash joins, to keep the PoC simple. + * The filter would need to live in shared memory, and the workers would + * need to coordinate to build it. But it's doable. + * + * The filter lives in the HashState, in the hashCtx memory context. + * That means it gets destroyed along with the hashtable, and it follows + * the same lifecycle (during rescans, etc.). + * + * The size of the filter is bounded by both the estimated inner row + * count and a fixed fraction of work_mem. bloom_create() will round + * down to the next power-of-two bitset and enforces a 1MB minimum. + * + * XXX This may need more thought. If we limit bloom_work_mem too much, + * the false positive rate will get too bad, and we won't filter enough + * tuples for the filter to pay for itself. The adaptive behavior will + * eventually skip the filter, but we could just not build it at all? + * Or do we want to take the chance, sometimes? + */ + if (state->want_bloom_filter) + { + MemoryContext oldctx; + int bloom_work_mem; + + /* only serial hashjoins for now, init only once */ + Assert(hashtable->parallel_state == NULL); + Assert(state->bloom_filter == NULL); + + state->bloomFilterChecked = 0; + state->bloomFilterRejected = 0; + + /* Cap bloom filter at ~1/8 of work_mem, but not less than 1MB. */ + bloom_work_mem = Max(1024, work_mem / 8); + + oldctx = MemoryContextSwitchTo(hashtable->hashCxt); + state->bloom_filter = bloom_create((int64) Max(rows, 1.0), + bloom_work_mem, 0); + MemoryContextSwitchTo(oldctx); + } + return hashtable; } diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c index 0b365d5b475..8fa7af4cfef 100644 --- a/src/backend/executor/nodeHashjoin.c +++ b/src/backend/executor/nodeHashjoin.c @@ -169,7 +169,9 @@ #include "executor/instrument.h" #include "executor/nodeHash.h" #include "executor/nodeHashjoin.h" +#include "lib/bloomfilter.h" #include "miscadmin.h" +#include "port/pg_bitutils.h" #include "utils/lsyscache.h" #include "utils/sharedtuplestore.h" #include "utils/tuplestore.h" @@ -834,6 +836,7 @@ HashJoinState * ExecInitHashJoin(HashJoin *node, EState *estate, int eflags) { HashJoinState *hjstate; + HashState *hashState; Plan *outerNode; Hash *hashNode; TupleDesc outerDesc, @@ -875,11 +878,37 @@ ExecInitHashJoin(HashJoin *node, EState *estate, int eflags) outerNode = outerPlan(node); hashNode = (Hash *) innerPlan(node); + /* + * Register ourselves as a bloom-filter producer in the EState before + * recursing into the outer subtree, so the scan node (we pushed the + * filter to) can find us. We do this only if we actually managed to + * push down the filter to a scan node. + */ + if (node->bloom_consumer_count > 0) + ExecRegisterBloomFilterProducer(hjstate); + outerPlanState(hjstate) = ExecInitNode(outerNode, estate, eflags); outerDesc = ExecGetResultType(outerPlanState(hjstate)); innerPlanState(hjstate) = ExecInitNode((Plan *) hashNode, estate, eflags); innerDesc = ExecGetResultType(innerPlanState(hjstate)); + /* + * Tell the Hash child to actually build the bloom filter, and the + * ID assigned to the filter. + * + * XXX Seems a bit ugly to manipulate the inner plan state like this. + * Surely there's a better way. OTOH the two nodes are pretty tightly + * coupled already, so maybe it's fine. + * + * XXX Also, this assumes the hash table is not built by ExecInitNode(), + * which is true for now. But maybe we will relax that in the future + * (e.g. so that the scan can push the filter to storage / to remote FDW + * node / ...)? + */ + hashState = castNode(HashState, innerPlanState(hjstate)); + hashState->want_bloom_filter = (node->bloom_consumer_count > 0); + hashState->bloom_filter_id = node->bloom_filter_id; + /* * Initialize result slot, type and projection. */ @@ -1080,11 +1109,15 @@ ExecEndHashJoin(HashJoinState *node) /* * Free hash table + * + * Clear the bloom_filter pointer. It lives in hashCxt, so it gets freed by + * the ExecHashTableDestroy call. */ if (node->hj_HashTable) { ExecHashTableDestroy(node->hj_HashTable); node->hj_HashTable = NULL; + hashNode->bloom_filter = NULL; } /* @@ -1737,6 +1770,12 @@ ExecReScanHashJoin(HashJoinState *node) node->hj_HashTable = NULL; node->hj_JoinState = HJ_BUILD_HASHTABLE; + /* + * Clear the bloom_filter pointer. It lives in hashCxt, so it gets + * freed by the ExecHashTableDestroy call. + */ + hashNode->bloom_filter = NULL; + /* * if chgParam of subnode is not null then plan will be re-scanned * by first ExecProcNode. @@ -1975,3 +2014,419 @@ ExecHashJoinInitializeWorker(HashJoinState *state, ExecSetExecProcNode(&state->js.ps, ExecParallelHashJoin); } + +/* + * BLOOM FILTER PUSHDOWN + * + * The pushdown decision is done in try_push_bloom_filter, when constructing + * the plan from the selected paths (see createplan.c). It decides which scan + * node should receive the bloom filter (if any), and what expressions it + * should use to calculate the hash value. + * + * Then at execution time: + * + * - ExecInitHashJoin registers itself in EState.es_bloom_producers + * before recursing into child plans, so by the time a recipient's + * ExecInit runs, the producer is already discoverable by plan_node_id. + * This registration only happens when there's at least one consumer. + * It also sets want_bloom_filter for the Hash node. + * + * - ExecHashTableCreate (in nodeHash.c) builds the actual bloom_filter + * when HashState.want_bloom_filter is set (so no work happens when + * nobody will probe). + * + * - Nodes with non-NIL plan->bloom_filters (and supporting bloom + * filters) call ExecInitBloomFilters() during its own ExecInit, + * which looks up the producer node (in the EState), compiles + * ExprStates for the hash expressions, etc. The filter state + * (BloomFilterState) gets added to ps->bloom_filters (a node may + * have multiple bloom filters from different hash joins). + * + * - The per-tuple loop of the scan node calls ExecBloomFilters() (much + * like ExecQual) to test the tuple against every attached filter, + * dropping it on the first filter that excludes it. For scan nodes + * this call happens in ExecScanExtended. + * + * The scan nodes reach the bloom filter via the HashJoinState pointer + * added to EState.es_bloom_producers, so that the rescans etc. (filter + * freed + recreated when the hash table is destroyed and rebuilt) are + * transparent to the consumer. The bloom filter gets reallocated after + * a rescan, so the pointer to it may change. + * + * XXX It's possible the bloom filter gets pushed down to a node that + * fails to initialize/use it. It'll be added to the bloom_filters list, + * but if the node does not call ExecInitBloomFilters, the filter will + * be unused. + * ---------------------------------------------------------------- + */ + +/* + * Lookup the HashJoinState for a producer by plan_node_id in the + * EState's es_bloom_producers list. Returns NULL if no matching + * producer has registered yet (which can happen for filters attached to + * recipients in trees where the producer hasn't ExecInit'd yet -- in + * normal execution we always register first). + */ +static HashJoinState * +LookupBloomFilterProducer(EState *estate, int bloom_filter_id) +{ + ListCell *lc; + + foreach(lc, estate->es_bloom_producers) + { + HashJoinState *hjstate = (HashJoinState *) lfirst(lc); + HashJoin *plan = (HashJoin *) hjstate->js.ps.plan; + + if (plan->bloom_filter_id == bloom_filter_id) + return hjstate; + } + return NULL; +} + +/* + * ExecBloomFilterHash + * Calculate the hash value for a tuple in the recipient node. + * + * Uses the per-key ExprStates initialized by ExecInitBloomFilters. Mirrors the + * scheme used by the Hash node, so that it matches what was inserted into the + * hashtable (and filter). + * + * Returns false if a strict key is NULL: such a tuple can't match anything in + * the bloom filter, but we still must let it pass through to the upstream join + * so the join (rather than us) decides what to do with it (e.g. emit + * NULL-extended for an outer join). + * + * XXX I'm not sure about this strict/NULL business. + */ +static inline bool +ExecBloomFilterHash(BloomFilterState *bfs, ExprContext *econtext, + uint32 *hashvalue) +{ + bool isnull; + uint32 hash; + + hash = DatumGetUInt32(ExecEvalExpr(bfs->keys, econtext, &isnull)); + + if (isnull) + return 0; /* XXX correct? do we care about NULL values?*/ + + *hashvalue = hash; + return true; +} + +/* + * ADAPTIVE BEHAVIOR + * + * If the bloom filter lets through most (or all) tuples, it becomes somewhat + * useless - we're just wasting CPU cycles, getting nothing in return. We could + * simply stop using such filter. But we've already paid quite a bit to build + * it, and maybe the data set is not uniform and we'll get into a part where + * fewer tuples pass. + * + * So we're evaluating the match rate for windows of 1000 probes. If more than + * 90% match, we start sampling 1% of the probes (i.e. 99% it treated as a + * match without looking at the filter). And if the match rate drops below 80%, + * we stop the sampling and all probes go to the filter. + * + * XXX These are empirical values, picked based on experiments. "Perfect" + * values depend on hardware, number of keys, data types, ... and maybe even + * on how many hash joins / pushed-down filters there are, and how deep (the + * deeper the bigger the benefit). + * + * XXX Maybe we should sample more probes, or maybe the window should be a bit + * smaller? With 1% and 1000 probes per window, it'll take 100k probes to + * enable the filter again. That seems like a lot. + * + * XXX We should probably track the number of times we "disabled" the filter, + * and what fraction of entries were "let through" during sampling periods. + * + * XXX There's an intentional gap between low/high thresholds, to add a bit + * of hysteresis into the behavior, so it does not flap all the time. + */ +#define BLOOM_ADAPTIVE_WINDOW_SIZE 1000 +#define BLOOM_ADAPTIVE_HIGH_MATCH_PERCENT 90 +#define BLOOM_ADAPTIVE_LOW_MATCH_PERCENT 80 +#define BLOOM_ADAPTIVE_SAMPLE_RATE 100 + +/* + * ExecBloomFilterShouldProbe + * Decide if the next tuple should probe the bloom filter. + * + * Returns true if the next value should actually probe the bloom filter + * We sample 1/100 (1/BLOOM_ADAPTIVE_SAMPLE_RATE) probes, i.e. 1%. + */ +static inline bool +ExecBloomFilterShouldProbe(BloomFilterState *bfs) +{ + if (!bfs->adaptiveSampling) + return true; + + bfs->adaptiveSampleCounter++; + + if (bfs->adaptiveSampleCounter >= BLOOM_ADAPTIVE_SAMPLE_RATE) + { + bfs->adaptiveSampleCounter = 0; + return true; + } + + return false; +} + +/* + * ExecBloomFilterUpdateAdaptiveState + * Update the adaptive state for sampling the probes. + * + * Adjust the adaptive behavior every 1000 probes. If too many probes match, + * stop using the filter (and just sample 1% of probes instead). If we were + * sampling, and the fraction of matches drops enough, stop the sampling. + */ +static inline void +ExecBloomFilterUpdateAdaptiveState(BloomFilterState *bfs, bool match) +{ + bfs->adaptiveWindowProbes++; + if (match) + bfs->adaptiveWindowMatches++; + + /* have we done enough probes in this window? */ + if (bfs->adaptiveWindowProbes >= BLOOM_ADAPTIVE_WINDOW_SIZE) + { + uint64 match_percent; + + /* fraction of matches */ + match_percent = (bfs->adaptiveWindowMatches * 100) / + bfs->adaptiveWindowProbes; + + if (!bfs->adaptiveSampling && + match_percent > BLOOM_ADAPTIVE_HIGH_MATCH_PERCENT) + { + /* Too many matches - start sampling. */ + bfs->adaptiveSampling = true; + bfs->adaptiveSampleCounter = 0; + } + else if (bfs->adaptiveSampling && + match_percent < BLOOM_ADAPTIVE_LOW_MATCH_PERCENT) + { + /* Stop sampling if the match fraction got low enough. */ + bfs->adaptiveSampling = false; + bfs->adaptiveSampleCounter = 0; + } + + /* in any case, start a new window of probes */ + bfs->adaptiveWindowProbes = 0; + bfs->adaptiveWindowMatches = 0; + } +} + +/* + * ExecBloomFilters + * Probe bloom filters for the current slot. + * + * Test the slot in the expression context (set by the scan node) against + * every bloom filter attached to the node. Returns true if the tuple matches + * all filters (some of which may have been skipped, because the hash table + * isn't built yet); false if at least one filter conclusively excludes it. + * + * 'filters' is a list of BloomFilterState for each filter, pushed to the + * node (stored in planstate->bloom_filters). It may be NIL, which means + * are no filters, and the function simply returns NULL. + * + * The caller is responsible for having set econtext->ecxt_scantuple to + * 'slot' first. We do not reset the per-tuple context here (it's up to the + * scan node). + * + * Designed to be called like ExecQual from the recipient's per-tuple + * loop. See ExecScanExtended for the scan-node integration point. + * + * XXX We're pushing filters to scan nodes, which set the scan slot. And + * setrefs.c is currently wired to do fix_scan_bloom_filters, called from + * set_plan_refs. If we decide to push filters to other nodes (e.g. joins), + * this may need some rework. + */ +bool +ExecBloomFilters(List *filters, ExprContext *econtext) +{ + ListCell *lc; + + /* bail out if no filters */ + if (filters == NIL) + return true; + + foreach(lc, filters) + { + BloomFilterState *bfs = (BloomFilterState *) lfirst(lc); + HashJoinState *producer = bfs->producer; + HashState *hashNode; + bloom_filter *bf; + uint32 hashvalue; + + /* Producer should always exist (resolved at init time). */ + Assert(producer != NULL); + + /* + * The hashtable (and the bloom filter) is built lazily the first + * time it needs to do a lookup. Until then, assume everything + * matches everything through. Once the filter is in place, start + * probing it. + * + * XXX It should only take a couple tuples (maybe just a single one) + * from the scan node before the filter is available. + */ + hashNode = castNode(HashState, innerPlanState(&producer->js.ps)); + bf = hashNode->bloom_filter; + if (bf == NULL) + continue; + + /* + * When recent bloom probes mostly pass through, probe only a sample of + * values to avoid spending work on an ineffective filter. Sampled + * probes keep updating the recent match fraction, so filtering resumes + * for every value once the filter becomes selective again. + */ + if (!ExecBloomFilterShouldProbe(bfs)) + continue; + + /* NULL strict key: tuple cannot be in the filter, pass through. */ + if (!ExecBloomFilterHash(bfs, econtext, &hashvalue)) + continue; + + /* + * XXX It's a bit silly the counters are in two places. We should + * keep just the hashNode counters, and get rid of bfs counters. + */ + bfs->checked++; + hashNode->bloomFilterChecked++; + + /* If not matching, we're done - reject the tuple. */ + if (bloom_lacks_element(bf, + (unsigned char *) &hashvalue, + sizeof(hashvalue))) + { + bfs->rejected++; + hashNode->bloomFilterRejected++; + ExecBloomFilterUpdateAdaptiveState(bfs, false); + return false; + } + + ExecBloomFilterUpdateAdaptiveState(bfs, true); + } + + return true; +} + +/* + * ExecInitBloomFilters + * Initialize state for pushed-down bloom filters. + * + * Called by nodes that want to act as a recipient of pushed-down filters, + * after the node's projection / scan-tuple slot are set up, just like + * for regular quals. + * + * Walks the plan's bloom_filters list and produces a list of BloomFilterState + * nodes, stored in planstate->bloom_filters. The producer HashJoinState node + * is resolved here, once, via EState.es_bloom_producers; so that no lookup is + * needed at probe time (the bloom_filter pointer may change on rescan, but + * that's not what we store). + * + * 'output_slot' is the slot whose values the filter expressions will be + * evaluated against (i.e. the same slot the surrounding qual evaluates + * against, post-setrefs). + * + * XXX For now this has to be the scan slot. See the comment about setrefs + * a bit earlier. Could be relaxed later, if we support to pushdown to + * other node types. + * + * XXX The filter states are initialized in es_query_cxt, but maybe that's + * too long-lived. The states live only as long as the recipient node. + */ +void +ExecInitBloomFilters(PlanState *planstate, TupleTableSlot *output_slot) +{ + Plan *plan = planstate->plan; + EState *estate = planstate->state; + List *result = NIL; + ListCell *lc; + MemoryContext oldctx; + + /* bail out if there are no pushed-down filters */ + if (plan->bloom_filters == NIL) + return; + + oldctx = MemoryContextSwitchTo(estate->es_query_cxt); + + foreach(lc, plan->bloom_filters) + { + BloomFilter *bf = lfirst_node(BloomFilter, lc); + BloomFilterState *bfs; + int nkeys; + + nkeys = list_length(bf->filter_exprs); + Assert(nkeys > 0); + Assert(nkeys == list_length(bf->hashops)); + Assert(nkeys == list_length(bf->hashcollations)); + + bfs = makeNode(BloomFilterState); + + /* XXX some of this is redundant */ + bfs->filter = bf; + bfs->producer_id = bf->producer_id; + bfs->producer = LookupBloomFilterProducer(estate, bf->producer_id); + + /* initialize the expression state for the hashvalue calculation */ + { + Oid *outer_hashfuncid = palloc_array(Oid, nkeys); + Oid *inner_hashfuncid = palloc_array(Oid, nkeys); + bool *hash_strict = palloc_array(bool, nkeys); + ListCell *lc2; + + /* + * Determine the hash function for each side of the join for the given + * join operator, and detect whether the join operator is strict. + */ + foreach(lc2, bf->hashops) + { + Oid hashop = lfirst_oid(lc2); + int i = foreach_current_index(lc2); + + if (!get_op_hash_functions(hashop, + &outer_hashfuncid[i], + &inner_hashfuncid[i])) + elog(ERROR, + "could not find hash function for hash operator %u", + hashop); + hash_strict[i] = op_strict(hashop); + } + + /* state for the hash value calculation */ + bfs->keys = ExecBuildHash32Expr(output_slot->tts_tupleDescriptor, + output_slot->tts_ops, + outer_hashfuncid, + bf->hashcollations, + bf->filter_exprs, + hash_strict, + planstate, + 0); + } + + result = lappend(result, bfs); + } + + planstate->bloom_filters = result; + + MemoryContextSwitchTo(oldctx); +} + +/* + * ExecRegisterBloomFilterProducer + * Register the pushed downn bloom filter. + * + * Called by ExecInitHashJoin (before recursing into the outer subtree) + * to register this HashJoinState as a producer for the pushed-down filter. + * Recipients in the outer subtree will look us up here by plan_node_id. + */ +void +ExecRegisterBloomFilterProducer(HashJoinState *hjstate) +{ + EState *estate = hjstate->js.ps.state; + + estate->es_bloom_producers = lappend(estate->es_bloom_producers, hjstate); +} diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c index d52012e8a69..bd7e5966017 100644 --- a/src/backend/executor/nodeIndexonlyscan.c +++ b/src/backend/executor/nodeIndexonlyscan.c @@ -600,6 +600,9 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags) indexstate->recheckqual = ExecInitQual(node->recheckqual, (PlanState *) indexstate); + ExecInitBloomFilters((PlanState *) indexstate, + indexstate->ss.ss_ScanTupleSlot); + /* * If we are just doing EXPLAIN (ie, aren't going to run the plan), stop * here. This allows an index-advisor plugin to EXPLAIN a plan containing diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c index 39f6691ee35..ef56522fbc5 100644 --- a/src/backend/executor/nodeIndexscan.c +++ b/src/backend/executor/nodeIndexscan.c @@ -970,6 +970,9 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags) indexstate->indexorderbyorig = ExecInitExprList(node->indexorderbyorig, (PlanState *) indexstate); + ExecInitBloomFilters((PlanState *) indexstate, + indexstate->ss.ss_ScanTupleSlot); + /* * If we are just doing EXPLAIN (ie, aren't going to run the plan), stop * here. This allows an index-advisor plugin to EXPLAIN a plan containing diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c index f3d273e1c5e..738502433b4 100644 --- a/src/backend/executor/nodeSamplescan.c +++ b/src/backend/executor/nodeSamplescan.c @@ -147,6 +147,9 @@ ExecInitSampleScan(SampleScan *node, EState *estate, int eflags) scanstate->repeatable = ExecInitExpr(tsc->repeatable, (PlanState *) scanstate); + ExecInitBloomFilters((PlanState *) scanstate, + scanstate->ss.ss_ScanTupleSlot); + /* * If we don't have a REPEATABLE clause, select a random seed. We want to * do this just once, since the seed shouldn't change over rescans. diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c index 5bcb0a861d7..4d3d7ba10a9 100644 --- a/src/backend/executor/nodeSeqscan.c +++ b/src/backend/executor/nodeSeqscan.c @@ -268,6 +268,9 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags) scanstate->ss.ps.qual = ExecInitQual(node->scan.plan.qual, (PlanState *) scanstate); + ExecInitBloomFilters((PlanState *) scanstate, + scanstate->ss.ss_ScanTupleSlot); + /* * When EvalPlanQual() is not in use, assign ExecProcNode for this node * based on the presence of qual and projection. Each ExecSeqScan*() diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c index b387ed6c308..b00a3736b99 100644 --- a/src/backend/executor/nodeTidrangescan.c +++ b/src/backend/executor/nodeTidrangescan.c @@ -434,6 +434,9 @@ ExecInitTidRangeScan(TidRangeScan *node, EState *estate, int eflags) tidrangestate->ss.ps.qual = ExecInitQual(node->scan.plan.qual, (PlanState *) tidrangestate); + ExecInitBloomFilters((PlanState *) tidrangestate, + tidrangestate->ss.ss_ScanTupleSlot); + TidExprListCreate(tidrangestate); /* diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c index 6641df10999..f7ba78a63fc 100644 --- a/src/backend/executor/nodeTidscan.c +++ b/src/backend/executor/nodeTidscan.c @@ -551,6 +551,9 @@ ExecInitTidScan(TidScan *node, EState *estate, int eflags) tidstate->ss.ps.qual = ExecInitQual(node->scan.plan.qual, (PlanState *) tidstate); + ExecInitBloomFilters((PlanState *) tidstate, + tidstate->ss.ss_ScanTupleSlot); + TidExprListCreate(tidstate); /* diff --git a/src/backend/lib/bloomfilter.c b/src/backend/lib/bloomfilter.c index 73b3768a172..fbcf788e271 100644 --- a/src/backend/lib/bloomfilter.c +++ b/src/backend/lib/bloomfilter.c @@ -192,6 +192,25 @@ bloom_prop_bits_set(bloom_filter *filter) return bits_set / (double) filter->m; } +/* + * Total bitset size, in bits. Useful for EXPLAIN instrumentation: + * divide by 8 to get the bitset's memory footprint in bytes. + */ +uint64 +bloom_total_bits(bloom_filter *filter) +{ + return filter->m; +} + +/* + * Number of hash functions in use. + */ +int +bloom_hash_funcs(bloom_filter *filter) +{ + return filter->k_hash_funcs; +} + /* * Which element in the sequence of powers of two is less than or equal to * target_bitset_bits? diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c index 1c575e56ff6..c3072a29ccc 100644 --- a/src/backend/optimizer/path/costsize.c +++ b/src/backend/optimizer/path/costsize.c @@ -156,6 +156,7 @@ bool enable_material = true; bool enable_memoize = true; bool enable_mergejoin = true; bool enable_hashjoin = true; +bool enable_hashjoin_bloom = true; bool enable_gathermerge = true; bool enable_partitionwise_join = false; bool enable_partitionwise_aggregate = false; diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c index de6a183da79..7ecb551aae6 100644 --- a/src/backend/optimizer/plan/createplan.c +++ b/src/backend/optimizer/plan/createplan.c @@ -4689,6 +4689,298 @@ create_mergejoin_plan(PlannerInfo *root, return join_plan; } +/* + * BLOOM FILTER PUSHDOWN + * + * When creating a hash join plan, consider building a bloom filter and + * pushing it down to the outer subtree. For now we only push filters to + * scan nodes containing all the join keys. When we find such scan node, + * we append the BloomFilter ID to the node's bloom_filters list, and + * increment the bloom_consumer_count for the hashjoin. + * + * As setrefs hashn't run yet, the join keys are still the raw Vars. + * So it's safe to compare var->varno against the scanrelid, and copy + * the keys verbatim onto the recipient. setrefs will rewrite the Vars + * later as usual, just like for the recipient's qual. + * + * XXX In most cases there'll be only a single consumer node. To get + * multiple consumers, we'd need either joins on the same keys, or + * ability to produce filters for subsets of the join keys (for cases + * where the join is more complex, and does not map to a single scan + * node directly). Seems like a possible future improvement. + * + * XXX Actually, we could have multiple consumer nodes for partitioned + * tables, where each partition gets a separate scan. Which seems like + * something we should support. + * + * XXX For simplicity, all outer join keys have to be bare Vars (from + * the same RTE). We could relax this later, and allow joins on more + * complex expressions. Not sure if that'll erase some of the benefits, + * which relies on filter probes being much cheaper hashtable probes. + * It also doesn't seem like a very common case. + * + * XXX The recipient node must be one of a small set of scan nodes. We + * could relax this, and allow pushing to other nodes (e.g. joins or + * aggregates). Future improvement. + * + * XXX We don't currently push the same HashJoin to multiple recipients, + * but multiple HashJoins may attach a filter to the same scan node. + * -------------------------------------------------------------------------- + */ + +/* + * bloom_join_side_preserved + * Decide if we can push filter to inner/outer side of a join. + * + * Outer joins that emit unmatched outer tuples (LEFT/ANTI/FULL) are + * skipped: dropping outer tuples there would be incorrect. + */ +static bool +bloom_join_side_preserved(JoinType jointype, bool to_outer) +{ + switch (jointype) + { + case JOIN_INNER: + return true; + case JOIN_LEFT: + case JOIN_SEMI: + case JOIN_ANTI: + return to_outer; + case JOIN_RIGHT: + case JOIN_RIGHT_SEMI: + case JOIN_RIGHT_ANTI: + return !to_outer; + case JOIN_FULL: + return false; + default: + return false; + } +} + +/* + * find_bloom_filter_recipient + * Try to find a scan node to push filter to. + * + * We support pushing filter to a subset of scan nodes (could be extended + * later). We support pushing filters through intermediate nodes (joins, + * sorts, ...). See bloom_join_side_preserved for joins. + * + * XXX We could push filters through more nodes - e.g. aggregates if the + * hash keys match GROUP BY keys (are a subset of). + * + * XXX We could do pushdown to parallel parts of a query. But we'd need + * a different way to communicate if a filter is built etc. (the worker + * won't have access to the hashjoin state). + */ +static Plan * +find_bloom_filter_recipient(Plan *plan, Index target_relid) +{ + /* XXX shouldn't really happen, I think */ + if (plan == NULL) + return NULL; + + /* XXX no pushdown to parallel parts of a query */ + if (plan->parallel_aware) + return NULL; + + switch (nodeTag(plan)) + { + case T_SeqScan: + case T_SampleScan: + case T_IndexScan: + case T_IndexOnlyScan: + case T_BitmapHeapScan: + case T_TidScan: + case T_TidRangeScan: + { + Scan *scan = (Scan *) plan; + + if (scan->scanrelid == target_relid) + return plan; + return NULL; + } + case T_Sort: + case T_IncrementalSort: + case T_Material: + case T_Memoize: + case T_Unique: + case T_Limit: + return find_bloom_filter_recipient(outerPlan(plan), target_relid); + case T_HashJoin: + case T_NestLoop: + case T_MergeJoin: + { + JoinType jt = ((Join *) plan)->jointype; + Plan *res; + + /* + * We can only push to one node, but we don't know on which + * side of the join it is (e.g. for inner joins it can be on + * both sides). So make sure to check both, if compatible + * with the join type (per bloom_join_side_preserved). + */ + if (bloom_join_side_preserved(jt, true)) + { + res = find_bloom_filter_recipient(outerPlan(plan), + target_relid); + if (res != NULL) + return res; + } + if (bloom_join_side_preserved(jt, false)) + { + res = find_bloom_filter_recipient(innerPlan(plan), + target_relid); + if (res != NULL) + return res; + } + return NULL; + } + default: + return NULL; + } +} + +/* + * try_push_bloom_filter + * Attempt to pushdown a bloom filter for the current hashjoin. + * + * The filter pushdown happens during plan creation, i.e. after the plan was + * already selected. That is not entirely optimal, and it has a couple of + * annoying consequences. + * + * The main disadvantage is that injecting the filter to a scan node may + * significantly alter the number of tuples produced by that scan node. If a + * filter eliminates 99% of the rows, the scan produces 1/100 of the rows it + * was planned with. It would not affect the scan itself, but if there are + * other nodes (between the scan and the join), maybe we'd have planned them + * differently if we knew about the lower cardinality? + * + * Similarly, it's confusing in the explain. That is, we'll get "rows=N" + * with the planner cardinality (before the filter was pushed down), but + * then in EXPLAIN ANALYZE it'll get much lower values. It'd be easy to + * confuse with inaccurate estimates. + * + * It'd be better to know about the filter earlier, when constructing the scan + * path. But that's not quite feasible with our bottom-up planner. When planing + * the scan, we don't know which of the joins above it will be hashjoins, or + * if it can pushdown the filter. We'd have to speculate, or maybe build more + * paths with/without expectation of the bloom filter pushdown. But that seems + * not great, as it'd add overhead for everyone. + */ +static void +try_push_bloom_filter(PlannerInfo *root, HashJoin *hj, Plan *outer_plan) +{ + List *hashkeys = hj->hashkeys; + List *hashops = hj->hashoperators; + List *hashcolls = hj->hashcollations; + ListCell *lc; + Index target_relid = 0; + Plan *recipient; + BloomFilter *bf; + + /* bail out if feature disabled. */ + if (!enable_hashjoin_bloom) + return; + + /* XXX shouldn't really happen, I think */ + if (hashkeys == NIL) + return; + + Assert(list_length(hashkeys) == list_length(hashops)); + Assert(list_length(hashkeys) == list_length(hashcolls)); + + /* + * Pushdown is unsafe for join types that emit unmatched outer tuples + * (LEFT/ANTI/FULL): we'd risk dropping outer tuples the join would + * otherwise have emitted (possibly NULL-extended). + */ + switch (hj->join.jointype) + { + case JOIN_INNER: + case JOIN_RIGHT: + case JOIN_SEMI: + case JOIN_RIGHT_SEMI: + case JOIN_RIGHT_ANTI: + /* these join types are OK */ + break; + default: + return; + } + + /* + * All hashkeys must be bare Vars referencing the same base RTE. + * + * XXX We could be a bit less strict, and check that at least some of the + * hashkeys are bare Vars, not all of them. So with joins on multiple + * expressions we'd have better chance to push a filter down. Doesn't + * seem worth it, at least for now. + */ + foreach(lc, hashkeys) + { + Node *k = (Node *) lfirst(lc); + Var *var; + + /* not a plain Var */ + if (!IsA(k, Var)) + return; + + var = (Var *) k; + + /* + * Reject outer references, whole-row or system columns, and + * special varnos (not sure we can get them here, though). + */ + if ((var->varlevelsup != 0) || + (var->varattno <= 0) || + IS_SPECIAL_VARNO(var->varno)) + return; + + /* make sure all the vars are for the same relid */ + if (target_relid == 0) + target_relid = var->varno; + else if (var->varno != target_relid) + return; + } + + /* should have found at least one var */ + Assert(target_relid != 0); + + /* + * See if we can find the scan node for target_relid. It certainly is + * in the plan somewhere, but it may not be able to pushdown the filter + * to it (because of a join or so). + */ + recipient = find_bloom_filter_recipient(outer_plan, target_relid); + if (recipient == NULL) + return; + + /* + * If we found a recipient, assign the filter an ID. We'll use it to + * register the filter in ExecRegisterBloomFilterProducer, and then + * for lookups in LookupBloomFilterProducer during execution. + * + * XXX We can't use plan_node_id, as it's not assigned yet, that only + * happens in set_plan_refs. Also, if we ever allow multiple filters + * per hashtable (e.g. for different subsets of keys), it's not work. + */ + hj->bloom_filter_id = ++root->glob->lastBloomFilterId; + + bf = makeNode(BloomFilter); + bf->filter_exprs = (List *) copyObject(hashkeys); + bf->hashops = list_copy(hashops); + bf->hashcollations = list_copy(hashcolls); + bf->producer_id = hj->bloom_filter_id; + + recipient->bloom_filters = lappend(recipient->bloom_filters, bf); + + /* + * XXX We've manged to push the filter to the scan node, but maybe + * we should wait with updating bloom_consumer_count when it actually + * initializes the filters in ExecInit()? + */ + hj->bloom_consumer_count++; +} + static HashJoin * create_hashjoin_plan(PlannerInfo *root, HashPath *best_path) @@ -4859,6 +5151,13 @@ create_hashjoin_plan(PlannerInfo *root, copy_generic_path_info(&join_plan->join.plan, &best_path->jpath.path); + /* + * Try to push the bloom filter for the hashtable down to nodes in the outer + * subtree. If a suitable scan node exists, add the filter to bloom_filters, + * and bump our bloom_consumer_count. + */ + try_push_bloom_filter(root, join_plan, outer_plan); + return join_plan; } diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c index f4689e7c9f8..60bd853bba7 100644 --- a/src/backend/optimizer/plan/planner.c +++ b/src/backend/optimizer/plan/planner.c @@ -389,6 +389,7 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions, glob->lastPHId = 0; glob->lastRowMarkId = 0; glob->lastPlanNodeId = 0; + glob->lastBloomFilterId = 0; glob->transientPlan = false; glob->dependsOnRole = false; glob->partition_directory = NULL; diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c index ff0e875f2a2..0059acfccbe 100644 --- a/src/backend/optimizer/plan/setrefs.c +++ b/src/backend/optimizer/plan/setrefs.c @@ -161,6 +161,7 @@ static Node *fix_scan_expr(PlannerInfo *root, Node *node, int rtoffset, double num_exec); static Node *fix_scan_expr_mutator(Node *node, fix_scan_expr_context *context); static bool fix_scan_expr_walker(Node *node, fix_scan_expr_context *context); +static void fix_scan_bloom_filters(PlannerInfo *root, Plan *plan, int rtoffset); static void set_join_references(PlannerInfo *root, Join *join, int rtoffset); static void set_upper_references(PlannerInfo *root, Plan *plan, int rtoffset); static void set_param_references(PlannerInfo *root, Plan *plan); @@ -665,6 +666,9 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset) splan->scan.plan.qual = fix_scan_list(root, splan->scan.plan.qual, rtoffset, NUM_EXEC_QUAL(plan)); + + /* pushed-down bloom filters */ + fix_scan_bloom_filters(root, plan, rtoffset); } break; case T_SampleScan: @@ -681,6 +685,9 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset) splan->tablesample = (TableSampleClause *) fix_scan_expr(root, (Node *) splan->tablesample, rtoffset, 1); + + /* pushed-down bloom filters */ + fix_scan_bloom_filters(root, plan, rtoffset); } break; case T_IndexScan: @@ -706,6 +713,9 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset) splan->indexorderbyorig = fix_scan_list(root, splan->indexorderbyorig, rtoffset, NUM_EXEC_QUAL(plan)); + + /* pushed-down bloom filters */ + fix_scan_bloom_filters(root, plan, rtoffset); } break; case T_IndexOnlyScan: @@ -744,6 +754,9 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset) splan->bitmapqualorig = fix_scan_list(root, splan->bitmapqualorig, rtoffset, NUM_EXEC_QUAL(plan)); + + /* pushed-down bloom filters */ + fix_scan_bloom_filters(root, plan, rtoffset); } break; case T_TidScan: @@ -760,6 +773,9 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset) splan->tidquals = fix_scan_list(root, splan->tidquals, rtoffset, 1); + + /* pushed-down bloom filters */ + fix_scan_bloom_filters(root, plan, rtoffset); } break; case T_TidRangeScan: @@ -776,6 +792,9 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset) splan->tidrangequals = fix_scan_list(root, splan->tidrangequals, rtoffset, 1); + + /* pushed-down bloom filters */ + fix_scan_bloom_filters(root, plan, rtoffset); } break; case T_SubqueryScan: @@ -1426,6 +1445,30 @@ set_indexonlyscan_references(PlannerInfo *root, rtoffset, NRM_EQUAL, NUM_EXEC_QUAL((Plan *) plan)); + /* + * Bloom filter pushdown: any BloomFilter recipient lists also need + * their key expressions rewritten to reference the index tuple, since + * that's what the recipient scan returns and what ExecBloomFilters + * will evaluate against. + */ + if (plan->scan.plan.bloom_filters != NIL) + { + ListCell *lc2; + + foreach(lc2, plan->scan.plan.bloom_filters) + { + BloomFilter *bf = lfirst_node(BloomFilter, lc2); + + bf->filter_exprs = (List *) + fix_upper_expr(root, + (Node *) bf->filter_exprs, + index_itlist, + INDEX_VAR, + rtoffset, + NRM_EQUAL, + NUM_EXEC_QUAL((Plan *) plan)); + } + } /* indexqual is already transformed to reference index columns */ plan->indexqual = fix_scan_list(root, plan->indexqual, rtoffset, 1); @@ -2398,6 +2441,26 @@ fix_scan_expr_walker(Node *node, fix_scan_expr_context *context) return expression_tree_walker(node, fix_scan_expr_walker, context); } +/* + * Fix references in hashkey expressions of bloom filters pushed down. + */ +static void +fix_scan_bloom_filters(PlannerInfo *root, Plan *plan, int rtoffset) +{ + ListCell *lc; + + if (plan->bloom_filters == NIL) + return; + + foreach(lc, plan->bloom_filters) + { + BloomFilter *bf = lfirst_node(BloomFilter, lc); + + bf->filter_exprs = fix_scan_list(root, bf->filter_exprs, rtoffset, + NUM_EXEC_QUAL(plan)); + } +} + /* * set_join_references * Modify the target list and quals of a join node to reference its diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index afaa058b046..1134b5187ac 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -920,6 +920,13 @@ boot_val => 'true', }, +{ name => 'enable_hashjoin_bloom', type => 'bool', context => 'PGC_USERSET', group => 'QUERY_TUNING_METHOD', + short_desc => 'Enables hash join bloom filter pushdown to the outer side.', + flags => 'GUC_EXPLAIN', + variable => 'enable_hashjoin_bloom', + boot_val => 'true', +}, + { name => 'enable_incremental_sort', type => 'bool', context => 'PGC_USERSET', group => 'QUERY_TUNING_METHOD', short_desc => 'Enables the planner\'s use of incremental sort steps.', flags => 'GUC_EXPLAIN', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index ac38cddaaf9..f8020170506 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -448,6 +448,7 @@ #enable_distinct_reordering = on #enable_self_join_elimination = on #enable_eager_aggregate = on +#enable_hashjoin_bloom = on # - Planner Cost Constants - diff --git a/src/include/executor/execScan.h b/src/include/executor/execScan.h index 18b03235c3c..8cfe9b03c3e 100644 --- a/src/include/executor/execScan.h +++ b/src/include/executor/execScan.h @@ -170,10 +170,11 @@ ExecScanExtended(ScanState *node, /* interrupt checks are in ExecScanFetch */ /* - * If we have neither a qual to check nor a projection to do, just skip - * all the overhead and return the raw scan tuple. + * If we have neither a qual to check nor a projection to do nor any + * pushed-down bloom filter to probe, just skip all the overhead and + * return the raw scan tuple. */ - if (!qual && !projInfo) + if (!qual && !projInfo && node->ps.bloom_filters == NIL) { ResetExprContext(econtext); return ExecScanFetch(node, epqstate, accessMtd, recheckMtd); @@ -214,6 +215,21 @@ ExecScanExtended(ScanState *node, */ econtext->ecxt_scantuple = slot; + /* + * If runtime bloom filters have been pushed down to this scan, + * check them first. A rejected tuple is dropped silently (no + * "Rows Removed by Filter" instrumentation -- the per-filter + * counters in BloomFilterState already capture this). + * + * XXX Maybe we should include this in "Rows Removed by Filter"? + */ + if (node->ps.bloom_filters != NIL && + !ExecBloomFilters(node->ps.bloom_filters, econtext)) + { + ResetExprContext(econtext); + continue; + } + /* * check that the current tuple satisfies the qual-clause * diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h index 33bbdbfeffb..01010a20cd6 100644 --- a/src/include/executor/executor.h +++ b/src/include/executor/executor.h @@ -516,6 +516,18 @@ ExecProject(ProjectionInfo *projInfo) } #endif +/* + * Bloom filter pushdown (see nodeHashjoin.c). Declared here so that + * scan nodes that act as recipients don't need to pull in hashjoin + * internals just to call these helpers from their ExecInit. + * + * XXX There's probably a better place for this. It should live in + * the executor somewhere, not in nodeHashjoin.c? + */ +extern bool ExecBloomFilters(List *filters, ExprContext *econtext); +extern void ExecInitBloomFilters(PlanState *planstate, + TupleTableSlot *output_slot); + /* * ExecQual - evaluate a qual prepared with ExecInitQual (possibly via * ExecPrepareQual). Returns true if qual is satisfied, else false. diff --git a/src/include/executor/nodeHashjoin.h b/src/include/executor/nodeHashjoin.h index aebd39be8b5..08efefae209 100644 --- a/src/include/executor/nodeHashjoin.h +++ b/src/include/executor/nodeHashjoin.h @@ -31,4 +31,13 @@ extern void ExecHashJoinInitializeWorker(HashJoinState *state, extern void ExecHashJoinSaveTuple(MinimalTuple tuple, uint32 hashvalue, BufFile **fileptr, HashJoinTable hashtable); +/* + * Bloom filter pushdown producer-side helper (see nodeHashjoin.c). + * + * ExecBloomFilters and ExecInitBloomFilters live in executor.h so that + * scan nodes can call them from ExecInit without having to pull in + * hashjoin internals. + */ +extern void ExecRegisterBloomFilterProducer(HashJoinState *hjstate); + #endif /* NODEHASHJOIN_H */ diff --git a/src/include/lib/bloomfilter.h b/src/include/lib/bloomfilter.h index 860ee9bdc72..6b0026c8c45 100644 --- a/src/include/lib/bloomfilter.h +++ b/src/include/lib/bloomfilter.h @@ -23,5 +23,7 @@ extern void bloom_add_element(bloom_filter *filter, unsigned char *elem, extern bool bloom_lacks_element(bloom_filter *filter, unsigned char *elem, size_t len); extern double bloom_prop_bits_set(bloom_filter *filter); +extern uint64 bloom_total_bits(bloom_filter *filter); +extern int bloom_hash_funcs(bloom_filter *filter); #endif /* BLOOMFILTER_H */ diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h index 13359180d25..04333f1a4d0 100644 --- a/src/include/nodes/execnodes.h +++ b/src/include/nodes/execnodes.h @@ -762,6 +762,14 @@ typedef struct EState List *es_auxmodifytables; /* List of secondary ModifyTableStates */ + /* + * List of nodes producing pushed-down bloom filters. Each list element + * is a HashJoinState with at least one filter pushed down to a scan node. + * Scans look up their producer in this list (by plan_node_id) lazily on + * first probe (and cache the result). + */ + List *es_bloom_producers; + /* * this ExprContext is for per-output-tuple operations, such as constraint * checks and index-value computations. It will be reset for each output @@ -1286,8 +1294,58 @@ typedef struct PlanState bool outeropsset; bool inneropsset; bool resultopsset; + + /* + * Bloom filters (BloomFilterState) pushed to this node (NIL if no + * filters were pushed down to this node). Right now only some scan + * nodes expect filters, but the list is in PlanState so that we can + * expand the feature to more nodes in the future. + */ + List *bloom_filters; } PlanState; +/* + * BloomFilterState + * + * State for executing (evaluating) a BloomFilter, pushed to a scan node. + * + * The producer is the HashJoinState this bloom filter is for. It's resolved + * at executor-init time using EState.es_bloom_producers (the producer always + * runs ExecInit before its recipients). + * + * 'checked' and 'rejected' are per-recipient counters reported by EXPLAIN + * ANALYZE. It's a bit redundant with the fields we have in the producer + * (see HashState). But if we ever end up with multiple consumers per filter, + * it'd be useful. + * + * The adaptive fields track the match fraction in recently checked probes. + * When most checks are passing through, the executor starts probing less + * frequently (and sampling probes instead), until the fraction of matches + * drops below some threshold. + */ +typedef struct BloomFilterState +{ + NodeTag type; + + /* producer HJ node and the filter */ + int producer_id; + struct HashJoinState *producer; + BloomFilter *filter; + + int nkeys; /* number of hash keys */ + ExprState *keys; /* ExprState for the hash calculation */ + + /* probe counters */ + uint64 checked; + uint64 rejected; + + /* adaptive probing */ + uint64 adaptiveWindowProbes; + uint64 adaptiveWindowMatches; + uint64 adaptiveSampleCounter; + bool adaptiveSampling; +} BloomFilterState; + /* ---------------- * these are defined to avoid confusion problems with "left" * and "right" and "inner" and "outer". The convention is that @@ -2700,6 +2758,39 @@ typedef struct HashState Tuplestorestate *null_tuple_store; /* where to put null-keyed tuples */ bool keep_null_tuples; /* do we need to save such tuples? */ + /* + * True iff at we managed to push down the bloom filter to at least one + * node on the HashJoin's outer side. It tells the Hash node to also build + * a bloom filter next to the hash table. + */ + bool want_bloom_filter; + + /* + * Mirror of the parent HashJoin's bloom_filter_id, copied here at + * ExecInitHashJoin time so EXPLAIN's show_hash_info can label the + * filter without traversing back up the plan-state tree (which is + * not easy, as there's no parent in PlanState). + */ + int bloom_filter_id; + + /* + * Bloom filter on hash values during the build phase. Probed by recipient + * nodes (typically scans in the outer subtree). NULL when pushdown is + * disabled or no recipient was identified, and (transiently) before + * ExecHashTableCreate runs (on the first outer tuple). + * + * Allocated in hashCxt, just like the hashtable itself. Reset on rescans. + */ + struct bloom_filter *bloom_filter; + + /* + * Counters with total per-filter instrumentation. Separate from the + * per-recipient counters in BloomFilterState. Redundant, but will be + * needed if we end up allowing multiple recipients. + */ + uint64 bloomFilterChecked; + uint64 bloomFilterRejected; + /* * In a parallelized hash join, the leader retains a pointer to the * shared-memory stats area in its shared_info field, and then copies the diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h index 27a2c6815b7..69f9ad2d5e3 100644 --- a/src/include/nodes/pathnodes.h +++ b/src/include/nodes/pathnodes.h @@ -244,6 +244,9 @@ typedef struct PlannerGlobal /* highest plan node ID assigned */ int lastPlanNodeId; + /* highest bloom-filter-producer ID assigned (see plannodes.h) */ + int lastBloomFilterId; + /* redo plan when TransactionXmin changes? */ bool transientPlan; diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h index 14a1dfed2b9..4e35d77cc49 100644 --- a/src/include/nodes/plannodes.h +++ b/src/include/nodes/plannodes.h @@ -254,8 +254,40 @@ typedef struct Plan */ Bitmapset *extParam; Bitmapset *allParam; + + /* + * List of BloomFilter nodes (or NIL). When non-empty, this plan node is + * a recipient of one or more runtime bloom filters pushed down at plan + * time by some HashJoin ancestor; see nodeHashjoin.c. Living on Plan + * (rather than on a specific subtype like Scan) allows pushdown to any + * node type that's prepared to call ExecBloomFilters on its output. + */ + List *bloom_filters; } Plan; +/* + * BloomFilter - + * One pushed-down bloom filter, attached to a recipient Plan node. + * + * 'filter_exprs', 'hashops' and 'hashcollations' are parallel lists, one + * entry per join key: the expression to hash, its hash operator (OID), + * and its input collation (OID). + * + * 'producer_id' is the bloom_filter_id of the producing HashJoin (resolved at + * executor init time via EState.es_bloom_producers). + * ---------------- + */ +typedef struct BloomFilter +{ + pg_node_attr(no_query_jumble) + + NodeTag type; + List *filter_exprs; + List *hashops; + List *hashcollations; + int producer_id; +} BloomFilter; + /* ---------------- * these are defined to avoid confusion problems with "left" * and "right" and "inner" and "outer". The convention is that @@ -1073,6 +1105,25 @@ typedef struct HashJoin * perform lookups in the hashtable over the inner plan. */ List *hashkeys; + + /* + * Number of plan nodes that consume this HashJoin's bloom filter via + * pushdown. Set at plan time by the bloom filter pushdown pass. + * + * At execution time, the HashJoin builds the bloom filter only when this + * is non-zero (and the enable_hashjoin_bloom GUC is on). + */ + int bloom_consumer_count; + + /* + * Identifier used by recipient nodes to find this producer at execution + * time, via EState.es_bloom_producers. Assigned during create_hashjoin_plan + * from PlannerGlobal.lastBloomFilterId. Each BloomFilter on a recipient stores + * a copy in its producer_id field for convenience. + * + * Zero when this HashJoin has no consumers. + */ + int bloom_filter_id; } HashJoin; /* ---------------- diff --git a/src/include/optimizer/cost.h b/src/include/optimizer/cost.h index f2fd5d31507..7339979c008 100644 --- a/src/include/optimizer/cost.h +++ b/src/include/optimizer/cost.h @@ -62,6 +62,7 @@ extern PGDLLIMPORT bool enable_material; extern PGDLLIMPORT bool enable_memoize; extern PGDLLIMPORT bool enable_mergejoin; extern PGDLLIMPORT bool enable_hashjoin; +extern PGDLLIMPORT bool enable_hashjoin_bloom; extern PGDLLIMPORT bool enable_gathermerge; extern PGDLLIMPORT bool enable_partitionwise_join; extern PGDLLIMPORT bool enable_partitionwise_aggregate; diff --git a/src/test/regress/expected/aggregates.out b/src/test/regress/expected/aggregates.out index fbda0e3bbc2..a419799a64e 100644 --- a/src/test/regress/expected/aggregates.out +++ b/src/test/regress/expected/aggregates.out @@ -1509,9 +1509,11 @@ group by t1.a,t1.b,t1.c,t1.d,t2.x,t2.y,t2.z; -> Hash Join Hash Cond: ((t2.x = t1.a) AND (t2.y = t1.b)) -> Seq Scan on t2 + Bloom Filter 1: keys=(x, y) -> Hash + Bloom Filter 1 -> Seq Scan on t1 -(7 rows) +(9 rows) -- Test case where t1 can be optimized but not t2 explain (costs off) select t1.*,t2.x,t2.z @@ -1524,9 +1526,11 @@ group by t1.a,t1.b,t1.c,t1.d,t2.x,t2.z; -> Hash Join Hash Cond: ((t2.x = t1.a) AND (t2.y = t1.b)) -> Seq Scan on t2 + Bloom Filter 1: keys=(x, y) -> Hash + Bloom Filter 1 -> Seq Scan on t1 -(7 rows) +(9 rows) -- Cannot optimize when PK is deferrable explain (costs off) select * from t3 group by a,b,c; diff --git a/src/test/regress/expected/eager_aggregate.out b/src/test/regress/expected/eager_aggregate.out index 456d32eb13d..df748f0592a 100644 --- a/src/test/regress/expected/eager_aggregate.out +++ b/src/test/regress/expected/eager_aggregate.out @@ -34,14 +34,16 @@ GROUP BY t1.a ORDER BY t1.a; Hash Cond: (t1.b = t2.b) -> Seq Scan on public.eager_agg_t1 t1 Output: t1.a, t1.b, t1.c + Bloom Filter 1: keys=(t1.b) -> Hash Output: t2.b, (PARTIAL avg(t2.c)) + Bloom Filter 1 -> Partial HashAggregate Output: t2.b, PARTIAL avg(t2.c) Group Key: t2.b -> Seq Scan on public.eager_agg_t2 t2 Output: t2.a, t2.b, t2.c -(18 rows) +(20 rows) SELECT t1.a, avg(t2.c) FROM eager_agg_t1 t1 @@ -80,8 +82,10 @@ GROUP BY t1.a ORDER BY t1.a; Hash Cond: (t1.b = t2.b) -> Seq Scan on public.eager_agg_t1 t1 Output: t1.a, t1.b, t1.c + Bloom Filter 1: keys=(t1.b) -> Hash Output: t2.b, (PARTIAL avg(t2.c)) + Bloom Filter 1 -> Partial GroupAggregate Output: t2.b, PARTIAL avg(t2.c) Group Key: t2.b @@ -90,7 +94,7 @@ GROUP BY t1.a ORDER BY t1.a; Sort Key: t2.b -> Seq Scan on public.eager_agg_t2 t2 Output: t2.c, t2.b -(21 rows) +(23 rows) SELECT t1.a, avg(t2.c) FROM eager_agg_t1 t1 @@ -134,8 +138,10 @@ GROUP BY t1.a ORDER BY t1.a; Hash Cond: (t1.b = t2.b) -> Seq Scan on public.eager_agg_t1 t1 Output: t1.a, t1.b, t1.c + Bloom Filter 2: keys=(t1.b) -> Hash Output: t2.b, (PARTIAL avg((t2.c + t3.c))) + Bloom Filter 2 -> Partial HashAggregate Output: t2.b, PARTIAL avg((t2.c + t3.c)) Group Key: t2.b @@ -144,11 +150,13 @@ GROUP BY t1.a ORDER BY t1.a; Hash Cond: (t3.a = t2.a) -> Seq Scan on public.eager_agg_t3 t3 Output: t3.a, t3.b, t3.c + Bloom Filter 1: keys=(t3.a) -> Hash Output: t2.c, t2.b, t2.a + Bloom Filter 1 -> Seq Scan on public.eager_agg_t2 t2 Output: t2.c, t2.b, t2.a -(25 rows) +(29 rows) SELECT t1.a, avg(t2.c + t3.c) FROM eager_agg_t1 t1 @@ -189,8 +197,10 @@ GROUP BY t1.a ORDER BY t1.a; Hash Cond: (t1.b = t2.b) -> Seq Scan on public.eager_agg_t1 t1 Output: t1.a, t1.b, t1.c + Bloom Filter 2: keys=(t1.b) -> Hash Output: t2.b, (PARTIAL avg((t2.c + t3.c))) + Bloom Filter 2 -> Partial GroupAggregate Output: t2.b, PARTIAL avg((t2.c + t3.c)) Group Key: t2.b @@ -202,11 +212,13 @@ GROUP BY t1.a ORDER BY t1.a; Hash Cond: (t3.a = t2.a) -> Seq Scan on public.eager_agg_t3 t3 Output: t3.a, t3.b, t3.c + Bloom Filter 1: keys=(t3.a) -> Hash Output: t2.c, t2.b, t2.a + Bloom Filter 1 -> Seq Scan on public.eager_agg_t2 t2 Output: t2.c, t2.b, t2.a -(28 rows) +(32 rows) SELECT t1.a, avg(t2.c + t3.c) FROM eager_agg_t1 t1 @@ -249,14 +261,16 @@ GROUP BY t1.a ORDER BY t1.a; Hash Cond: (t1.b = t2.b) -> Seq Scan on public.eager_agg_t1 t1 Output: t1.a, t1.b, t1.c + Bloom Filter 1: keys=(t1.b) -> Hash Output: t2.b, (PARTIAL avg(t2.c)) + Bloom Filter 1 -> Partial HashAggregate Output: t2.b, PARTIAL avg(t2.c) Group Key: t2.b -> Seq Scan on public.eager_agg_t2 t2 Output: t2.a, t2.b, t2.c -(18 rows) +(20 rows) SELECT t1.a, avg(t2.c) FROM eager_agg_t1 t1 @@ -295,11 +309,13 @@ GROUP BY t2.b ORDER BY t2.b; Hash Cond: (t2.b = t1.b) -> Seq Scan on public.eager_agg_t2 t2 Output: t2.a, t2.b, t2.c + Bloom Filter 1: keys=(t2.b) -> Hash Output: t1.b + Bloom Filter 1 -> Seq Scan on public.eager_agg_t1 t1 Output: t1.b -(15 rows) +(17 rows) SELECT t2.b, avg(t2.c) FROM eager_agg_t1 t1 @@ -400,14 +416,16 @@ GROUP BY t1.a ORDER BY t1.a; Hash Cond: (t1.b = t2.b) -> Seq Scan on public.eager_agg_t1 t1 Output: t1.a, t1.b, t1.c + Bloom Filter 1: keys=(t1.b) -> Hash Output: t2.b, (PARTIAL avg(t2.c)) + Bloom Filter 1 -> Partial HashAggregate Output: t2.b, PARTIAL avg(t2.c) Group Key: t2.b -> Seq Scan on public.eager_agg_t2 t2 Output: t2.a, t2.b, t2.c -(18 rows) +(20 rows) SELECT t1.a, avg(t2.c) FROM eager_agg_t1 t1 @@ -444,9 +462,11 @@ GROUP BY t1.a ORDER BY t1.a; -> Hash Join Hash Cond: (t2.b = t1.b) -> Seq Scan on eager_agg_t2 t2 + Bloom Filter 1: keys=(b) -> Hash + Bloom Filter 1 -> Seq Scan on eager_agg_t1 t1 -(9 rows) +(11 rows) EXPLAIN (COSTS OFF) SELECT t1.a, avg(t2.c) FILTER (WHERE random() > 0.5) @@ -462,9 +482,11 @@ GROUP BY t1.a ORDER BY t1.a; -> Hash Join Hash Cond: (t2.b = t1.b) -> Seq Scan on eager_agg_t2 t2 + Bloom Filter 1: keys=(b) -> Hash + Bloom Filter 1 -> Seq Scan on eager_agg_t1 t1 -(9 rows) +(11 rows) DROP TABLE eager_agg_t1; DROP TABLE eager_agg_t2; @@ -509,8 +531,10 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t2.y = t1.x) -> Seq Scan on public.eager_agg_tab2_p1 t2 Output: t2.y + Bloom Filter 1: keys=(t2.y) -> Hash Output: t1.x, (PARTIAL sum(t1.y)), (PARTIAL count(*)) + Bloom Filter 1 -> Partial HashAggregate Output: t1.x, PARTIAL sum(t1.y), PARTIAL count(*) Group Key: t1.x @@ -524,8 +548,10 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t2_1.y = t1_1.x) -> Seq Scan on public.eager_agg_tab2_p2 t2_1 Output: t2_1.y + Bloom Filter 2: keys=(t2_1.y) -> Hash Output: t1_1.x, (PARTIAL sum(t1_1.y)), (PARTIAL count(*)) + Bloom Filter 2 -> Partial HashAggregate Output: t1_1.x, PARTIAL sum(t1_1.y), PARTIAL count(*) Group Key: t1_1.x @@ -539,14 +565,16 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t2_2.y = t1_2.x) -> Seq Scan on public.eager_agg_tab2_p3 t2_2 Output: t2_2.y + Bloom Filter 3: keys=(t2_2.y) -> Hash Output: t1_2.x, (PARTIAL sum(t1_2.y)), (PARTIAL count(*)) + Bloom Filter 3 -> Partial HashAggregate Output: t1_2.x, PARTIAL sum(t1_2.y), PARTIAL count(*) Group Key: t1_2.x -> Seq Scan on public.eager_agg_tab1_p3 t1_2 Output: t1_2.x, t1_2.y -(49 rows) +(55 rows) SELECT t1.x, sum(t1.y), count(*) FROM eager_agg_tab1 t1 @@ -591,8 +619,10 @@ GROUP BY t2.y ORDER BY t2.y; Hash Cond: (t2.y = t1.x) -> Seq Scan on public.eager_agg_tab2_p1 t2 Output: t2.y + Bloom Filter 1: keys=(t2.y) -> Hash Output: t1.x, (PARTIAL sum(t1.y)), (PARTIAL count(*)) + Bloom Filter 1 -> Partial HashAggregate Output: t1.x, PARTIAL sum(t1.y), PARTIAL count(*) Group Key: t1.x @@ -606,8 +636,10 @@ GROUP BY t2.y ORDER BY t2.y; Hash Cond: (t2_1.y = t1_1.x) -> Seq Scan on public.eager_agg_tab2_p2 t2_1 Output: t2_1.y + Bloom Filter 2: keys=(t2_1.y) -> Hash Output: t1_1.x, (PARTIAL sum(t1_1.y)), (PARTIAL count(*)) + Bloom Filter 2 -> Partial HashAggregate Output: t1_1.x, PARTIAL sum(t1_1.y), PARTIAL count(*) Group Key: t1_1.x @@ -621,14 +653,16 @@ GROUP BY t2.y ORDER BY t2.y; Hash Cond: (t2_2.y = t1_2.x) -> Seq Scan on public.eager_agg_tab2_p3 t2_2 Output: t2_2.y + Bloom Filter 3: keys=(t2_2.y) -> Hash Output: t1_2.x, (PARTIAL sum(t1_2.y)), (PARTIAL count(*)) + Bloom Filter 3 -> Partial HashAggregate Output: t1_2.x, PARTIAL sum(t1_2.y), PARTIAL count(*) Group Key: t1_2.x -> Seq Scan on public.eager_agg_tab1_p3 t1_2 Output: t1_2.y, t1_2.x -(49 rows) +(55 rows) SELECT t2.y, sum(t1.y), count(*) FROM eager_agg_tab1 t1 @@ -675,8 +709,10 @@ GROUP BY t2.x HAVING avg(t1.x) > 5 ORDER BY t2.x; Hash Cond: (t2.y = t1.x) -> Seq Scan on public.eager_agg_tab2_p1 t2 Output: t2.x, t2.y + Bloom Filter 1: keys=(t2.y) -> Hash Output: t1.x, (PARTIAL sum(t1.x)), (PARTIAL count(*)), (PARTIAL avg(t1.x)) + Bloom Filter 1 -> Partial HashAggregate Output: t1.x, PARTIAL sum(t1.x), PARTIAL count(*), PARTIAL avg(t1.x) Group Key: t1.x @@ -687,8 +723,10 @@ GROUP BY t2.x HAVING avg(t1.x) > 5 ORDER BY t2.x; Hash Cond: (t2_1.y = t1_1.x) -> Seq Scan on public.eager_agg_tab2_p2 t2_1 Output: t2_1.x, t2_1.y + Bloom Filter 2: keys=(t2_1.y) -> Hash Output: t1_1.x, (PARTIAL sum(t1_1.x)), (PARTIAL count(*)), (PARTIAL avg(t1_1.x)) + Bloom Filter 2 -> Partial HashAggregate Output: t1_1.x, PARTIAL sum(t1_1.x), PARTIAL count(*), PARTIAL avg(t1_1.x) Group Key: t1_1.x @@ -699,14 +737,16 @@ GROUP BY t2.x HAVING avg(t1.x) > 5 ORDER BY t2.x; Hash Cond: (t2_2.y = t1_2.x) -> Seq Scan on public.eager_agg_tab2_p3 t2_2 Output: t2_2.x, t2_2.y + Bloom Filter 3: keys=(t2_2.y) -> Hash Output: t1_2.x, (PARTIAL sum(t1_2.x)), (PARTIAL count(*)), (PARTIAL avg(t1_2.x)) + Bloom Filter 3 -> Partial HashAggregate Output: t1_2.x, PARTIAL sum(t1_2.x), PARTIAL count(*), PARTIAL avg(t1_2.x) Group Key: t1_2.x -> Seq Scan on public.eager_agg_tab1_p3 t1_2 Output: t1_2.x -(44 rows) +(50 rows) SELECT t2.x, sum(t1.x), count(*) FROM eager_agg_tab1 t1 @@ -748,8 +788,10 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t1.x = t2.x) -> Seq Scan on public.eager_agg_tab1_p1 t1 Output: t1.x + Bloom Filter 2: keys=(t1.x) -> Hash Output: t2.x, t3.x, (PARTIAL sum((t2.y + t3.y))) + Bloom Filter 2 -> Partial HashAggregate Output: t2.x, t3.x, PARTIAL sum((t2.y + t3.y)) Group Key: t2.x @@ -758,8 +800,10 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t2.x = t3.x) -> Seq Scan on public.eager_agg_tab1_p1 t2 Output: t2.y, t2.x + Bloom Filter 1: keys=(t2.x) -> Hash Output: t3.y, t3.x + Bloom Filter 1 -> Seq Scan on public.eager_agg_tab1_p1 t3 Output: t3.y, t3.x -> Finalize HashAggregate @@ -770,8 +814,10 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t1_1.x = t2_1.x) -> Seq Scan on public.eager_agg_tab1_p2 t1_1 Output: t1_1.x + Bloom Filter 4: keys=(t1_1.x) -> Hash Output: t2_1.x, t3_1.x, (PARTIAL sum((t2_1.y + t3_1.y))) + Bloom Filter 4 -> Partial HashAggregate Output: t2_1.x, t3_1.x, PARTIAL sum((t2_1.y + t3_1.y)) Group Key: t2_1.x @@ -780,8 +826,10 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t2_1.x = t3_1.x) -> Seq Scan on public.eager_agg_tab1_p2 t2_1 Output: t2_1.y, t2_1.x + Bloom Filter 3: keys=(t2_1.x) -> Hash Output: t3_1.y, t3_1.x + Bloom Filter 3 -> Seq Scan on public.eager_agg_tab1_p2 t3_1 Output: t3_1.y, t3_1.x -> Finalize HashAggregate @@ -792,8 +840,10 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t1_2.x = t2_2.x) -> Seq Scan on public.eager_agg_tab1_p3 t1_2 Output: t1_2.x + Bloom Filter 6: keys=(t1_2.x) -> Hash Output: t2_2.x, t3_2.x, (PARTIAL sum((t2_2.y + t3_2.y))) + Bloom Filter 6 -> Partial HashAggregate Output: t2_2.x, t3_2.x, PARTIAL sum((t2_2.y + t3_2.y)) Group Key: t2_2.x @@ -802,11 +852,13 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t2_2.x = t3_2.x) -> Seq Scan on public.eager_agg_tab1_p3 t2_2 Output: t2_2.y, t2_2.x + Bloom Filter 5: keys=(t2_2.x) -> Hash Output: t3_2.y, t3_2.x + Bloom Filter 5 -> Seq Scan on public.eager_agg_tab1_p3 t3_2 Output: t3_2.y, t3_2.x -(70 rows) +(82 rows) SELECT t1.x, sum(t2.y + t3.y) FROM eager_agg_tab1 t1 @@ -976,8 +1028,10 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t2.y = t1.x) -> Seq Scan on public.eager_agg_tab2_p1 t2 Output: t2.y + Bloom Filter 1: keys=(t2.y) -> Hash Output: t1.x, (PARTIAL sum(t1.y)), (PARTIAL count(*)) + Bloom Filter 1 -> Partial HashAggregate Output: t1.x, PARTIAL sum(t1.y), PARTIAL count(*) Group Key: t1.x @@ -991,8 +1045,10 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t2_1.y = t1_1.x) -> Seq Scan on public.eager_agg_tab2_p2 t2_1 Output: t2_1.y + Bloom Filter 2: keys=(t2_1.y) -> Hash Output: t1_1.x, (PARTIAL sum(t1_1.y)), (PARTIAL count(*)) + Bloom Filter 2 -> Partial HashAggregate Output: t1_1.x, PARTIAL sum(t1_1.y), PARTIAL count(*) Group Key: t1_1.x @@ -1006,14 +1062,16 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t2_2.y = t1_2.x) -> Seq Scan on public.eager_agg_tab2_p3 t2_2 Output: t2_2.y + Bloom Filter 3: keys=(t2_2.y) -> Hash Output: t1_2.x, (PARTIAL sum(t1_2.y)), (PARTIAL count(*)) + Bloom Filter 3 -> Partial HashAggregate Output: t1_2.x, PARTIAL sum(t1_2.y), PARTIAL count(*) Group Key: t1_2.x -> Seq Scan on public.eager_agg_tab1_p3 t1_2 Output: t1_2.x, t1_2.y -(49 rows) +(55 rows) SELECT t1.x, sum(t1.y), count(*) FROM eager_agg_tab1 t1 @@ -1076,8 +1134,10 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t1.x = t2.x) -> Seq Scan on public.eager_agg_tab_ml_p1 t1 Output: t1.x + Bloom Filter 1: keys=(t1.x) -> Hash Output: t2.x, (PARTIAL sum(t2.y)), (PARTIAL count(*)) + Bloom Filter 1 -> Partial HashAggregate Output: t2.x, PARTIAL sum(t2.y), PARTIAL count(*) Group Key: t2.x @@ -1091,8 +1151,10 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t1_1.x = t2_1.x) -> Seq Scan on public.eager_agg_tab_ml_p2_s1 t1_1 Output: t1_1.x + Bloom Filter 2: keys=(t1_1.x) -> Hash Output: t2_1.x, (PARTIAL sum(t2_1.y)), (PARTIAL count(*)) + Bloom Filter 2 -> Partial HashAggregate Output: t2_1.x, PARTIAL sum(t2_1.y), PARTIAL count(*) Group Key: t2_1.x @@ -1106,8 +1168,10 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t1_2.x = t2_2.x) -> Seq Scan on public.eager_agg_tab_ml_p2_s2 t1_2 Output: t1_2.x + Bloom Filter 3: keys=(t1_2.x) -> Hash Output: t2_2.x, (PARTIAL sum(t2_2.y)), (PARTIAL count(*)) + Bloom Filter 3 -> Partial HashAggregate Output: t2_2.x, PARTIAL sum(t2_2.y), PARTIAL count(*) Group Key: t2_2.x @@ -1121,8 +1185,10 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t1_3.x = t2_3.x) -> Seq Scan on public.eager_agg_tab_ml_p3_s1 t1_3 Output: t1_3.x + Bloom Filter 4: keys=(t1_3.x) -> Hash Output: t2_3.x, (PARTIAL sum(t2_3.y)), (PARTIAL count(*)) + Bloom Filter 4 -> Partial HashAggregate Output: t2_3.x, PARTIAL sum(t2_3.y), PARTIAL count(*) Group Key: t2_3.x @@ -1136,14 +1202,16 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t1_4.x = t2_4.x) -> Seq Scan on public.eager_agg_tab_ml_p3_s2 t1_4 Output: t1_4.x + Bloom Filter 5: keys=(t1_4.x) -> Hash Output: t2_4.x, (PARTIAL sum(t2_4.y)), (PARTIAL count(*)) + Bloom Filter 5 -> Partial HashAggregate Output: t2_4.x, PARTIAL sum(t2_4.y), PARTIAL count(*) Group Key: t2_4.x -> Seq Scan on public.eager_agg_tab_ml_p3_s2 t2_4 Output: t2_4.y, t2_4.x -(79 rows) +(89 rows) SELECT t1.x, sum(t2.y), count(*) FROM eager_agg_tab_ml t1 @@ -1204,8 +1272,10 @@ GROUP BY t1.y ORDER BY t1.y; Hash Cond: (t1.x = t2.x) -> Seq Scan on public.eager_agg_tab_ml_p1 t1 Output: t1.y, t1.x + Bloom Filter 1: keys=(t1.x) -> Hash Output: t2.x, (PARTIAL sum(t2.y)), (PARTIAL count(*)) + Bloom Filter 1 -> Partial HashAggregate Output: t2.x, PARTIAL sum(t2.y), PARTIAL count(*) Group Key: t2.x @@ -1216,8 +1286,10 @@ GROUP BY t1.y ORDER BY t1.y; Hash Cond: (t1_1.x = t2_1.x) -> Seq Scan on public.eager_agg_tab_ml_p2_s1 t1_1 Output: t1_1.y, t1_1.x + Bloom Filter 2: keys=(t1_1.x) -> Hash Output: t2_1.x, (PARTIAL sum(t2_1.y)), (PARTIAL count(*)) + Bloom Filter 2 -> Partial HashAggregate Output: t2_1.x, PARTIAL sum(t2_1.y), PARTIAL count(*) Group Key: t2_1.x @@ -1228,8 +1300,10 @@ GROUP BY t1.y ORDER BY t1.y; Hash Cond: (t1_2.x = t2_2.x) -> Seq Scan on public.eager_agg_tab_ml_p2_s2 t1_2 Output: t1_2.y, t1_2.x + Bloom Filter 3: keys=(t1_2.x) -> Hash Output: t2_2.x, (PARTIAL sum(t2_2.y)), (PARTIAL count(*)) + Bloom Filter 3 -> Partial HashAggregate Output: t2_2.x, PARTIAL sum(t2_2.y), PARTIAL count(*) Group Key: t2_2.x @@ -1240,8 +1314,10 @@ GROUP BY t1.y ORDER BY t1.y; Hash Cond: (t1_3.x = t2_3.x) -> Seq Scan on public.eager_agg_tab_ml_p3_s1 t1_3 Output: t1_3.y, t1_3.x + Bloom Filter 4: keys=(t1_3.x) -> Hash Output: t2_3.x, (PARTIAL sum(t2_3.y)), (PARTIAL count(*)) + Bloom Filter 4 -> Partial HashAggregate Output: t2_3.x, PARTIAL sum(t2_3.y), PARTIAL count(*) Group Key: t2_3.x @@ -1252,14 +1328,16 @@ GROUP BY t1.y ORDER BY t1.y; Hash Cond: (t1_4.x = t2_4.x) -> Seq Scan on public.eager_agg_tab_ml_p3_s2 t1_4 Output: t1_4.y, t1_4.x + Bloom Filter 5: keys=(t1_4.x) -> Hash Output: t2_4.x, (PARTIAL sum(t2_4.y)), (PARTIAL count(*)) + Bloom Filter 5 -> Partial HashAggregate Output: t2_4.x, PARTIAL sum(t2_4.y), PARTIAL count(*) Group Key: t2_4.x -> Seq Scan on public.eager_agg_tab_ml_p3_s2 t2_4 Output: t2_4.y, t2_4.x -(67 rows) +(77 rows) SELECT t1.y, sum(t2.y), count(*) FROM eager_agg_tab_ml t1 @@ -1321,8 +1399,10 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t1.x = t2.x) -> Seq Scan on public.eager_agg_tab_ml_p1 t1 Output: t1.x + Bloom Filter 2: keys=(t1.x) -> Hash Output: t2.x, t3.x, (PARTIAL sum((t2.y + t3.y))), (PARTIAL count(*)) + Bloom Filter 2 -> Partial HashAggregate Output: t2.x, t3.x, PARTIAL sum((t2.y + t3.y)), PARTIAL count(*) Group Key: t2.x @@ -1331,8 +1411,10 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t2.x = t3.x) -> Seq Scan on public.eager_agg_tab_ml_p1 t2 Output: t2.y, t2.x + Bloom Filter 1: keys=(t2.x) -> Hash Output: t3.y, t3.x + Bloom Filter 1 -> Seq Scan on public.eager_agg_tab_ml_p1 t3 Output: t3.y, t3.x -> Finalize HashAggregate @@ -1343,8 +1425,10 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t1_1.x = t2_1.x) -> Seq Scan on public.eager_agg_tab_ml_p2_s1 t1_1 Output: t1_1.x + Bloom Filter 4: keys=(t1_1.x) -> Hash Output: t2_1.x, t3_1.x, (PARTIAL sum((t2_1.y + t3_1.y))), (PARTIAL count(*)) + Bloom Filter 4 -> Partial HashAggregate Output: t2_1.x, t3_1.x, PARTIAL sum((t2_1.y + t3_1.y)), PARTIAL count(*) Group Key: t2_1.x @@ -1353,8 +1437,10 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t2_1.x = t3_1.x) -> Seq Scan on public.eager_agg_tab_ml_p2_s1 t2_1 Output: t2_1.y, t2_1.x + Bloom Filter 3: keys=(t2_1.x) -> Hash Output: t3_1.y, t3_1.x + Bloom Filter 3 -> Seq Scan on public.eager_agg_tab_ml_p2_s1 t3_1 Output: t3_1.y, t3_1.x -> Finalize HashAggregate @@ -1365,8 +1451,10 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t1_2.x = t2_2.x) -> Seq Scan on public.eager_agg_tab_ml_p2_s2 t1_2 Output: t1_2.x + Bloom Filter 6: keys=(t1_2.x) -> Hash Output: t2_2.x, t3_2.x, (PARTIAL sum((t2_2.y + t3_2.y))), (PARTIAL count(*)) + Bloom Filter 6 -> Partial HashAggregate Output: t2_2.x, t3_2.x, PARTIAL sum((t2_2.y + t3_2.y)), PARTIAL count(*) Group Key: t2_2.x @@ -1375,8 +1463,10 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t2_2.x = t3_2.x) -> Seq Scan on public.eager_agg_tab_ml_p2_s2 t2_2 Output: t2_2.y, t2_2.x + Bloom Filter 5: keys=(t2_2.x) -> Hash Output: t3_2.y, t3_2.x + Bloom Filter 5 -> Seq Scan on public.eager_agg_tab_ml_p2_s2 t3_2 Output: t3_2.y, t3_2.x -> Finalize HashAggregate @@ -1387,8 +1477,10 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t1_3.x = t2_3.x) -> Seq Scan on public.eager_agg_tab_ml_p3_s1 t1_3 Output: t1_3.x + Bloom Filter 8: keys=(t1_3.x) -> Hash Output: t2_3.x, t3_3.x, (PARTIAL sum((t2_3.y + t3_3.y))), (PARTIAL count(*)) + Bloom Filter 8 -> Partial HashAggregate Output: t2_3.x, t3_3.x, PARTIAL sum((t2_3.y + t3_3.y)), PARTIAL count(*) Group Key: t2_3.x @@ -1397,8 +1489,10 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t2_3.x = t3_3.x) -> Seq Scan on public.eager_agg_tab_ml_p3_s1 t2_3 Output: t2_3.y, t2_3.x + Bloom Filter 7: keys=(t2_3.x) -> Hash Output: t3_3.y, t3_3.x + Bloom Filter 7 -> Seq Scan on public.eager_agg_tab_ml_p3_s1 t3_3 Output: t3_3.y, t3_3.x -> Finalize HashAggregate @@ -1409,8 +1503,10 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t1_4.x = t2_4.x) -> Seq Scan on public.eager_agg_tab_ml_p3_s2 t1_4 Output: t1_4.x + Bloom Filter 10: keys=(t1_4.x) -> Hash Output: t2_4.x, t3_4.x, (PARTIAL sum((t2_4.y + t3_4.y))), (PARTIAL count(*)) + Bloom Filter 10 -> Partial HashAggregate Output: t2_4.x, t3_4.x, PARTIAL sum((t2_4.y + t3_4.y)), PARTIAL count(*) Group Key: t2_4.x @@ -1419,11 +1515,13 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t2_4.x = t3_4.x) -> Seq Scan on public.eager_agg_tab_ml_p3_s2 t2_4 Output: t2_4.y, t2_4.x + Bloom Filter 9: keys=(t2_4.x) -> Hash Output: t3_4.y, t3_4.x + Bloom Filter 9 -> Seq Scan on public.eager_agg_tab_ml_p3_s2 t3_4 Output: t3_4.y, t3_4.x -(114 rows) +(134 rows) SELECT t1.x, sum(t2.y + t3.y), count(*) FROM eager_agg_tab_ml t1 @@ -1485,8 +1583,10 @@ GROUP BY t3.y ORDER BY t3.y; Hash Cond: (t1.x = t2.x) -> Seq Scan on public.eager_agg_tab_ml_p1 t1 Output: t1.x + Bloom Filter 2: keys=(t1.x) -> Hash Output: t2.x, t3.y, t3.x, (PARTIAL sum((t2.y + t3.y))), (PARTIAL count(*)) + Bloom Filter 2 -> Partial HashAggregate Output: t2.x, t3.y, t3.x, PARTIAL sum((t2.y + t3.y)), PARTIAL count(*) Group Key: t2.x, t3.y, t3.x @@ -1495,8 +1595,10 @@ GROUP BY t3.y ORDER BY t3.y; Hash Cond: (t2.x = t3.x) -> Seq Scan on public.eager_agg_tab_ml_p1 t2 Output: t2.y, t2.x + Bloom Filter 1: keys=(t2.x) -> Hash Output: t3.y, t3.x + Bloom Filter 1 -> Seq Scan on public.eager_agg_tab_ml_p1 t3 Output: t3.y, t3.x -> Hash Join @@ -1504,8 +1606,10 @@ GROUP BY t3.y ORDER BY t3.y; Hash Cond: (t1_1.x = t2_1.x) -> Seq Scan on public.eager_agg_tab_ml_p2_s1 t1_1 Output: t1_1.x + Bloom Filter 4: keys=(t1_1.x) -> Hash Output: t2_1.x, t3_1.y, t3_1.x, (PARTIAL sum((t2_1.y + t3_1.y))), (PARTIAL count(*)) + Bloom Filter 4 -> Partial HashAggregate Output: t2_1.x, t3_1.y, t3_1.x, PARTIAL sum((t2_1.y + t3_1.y)), PARTIAL count(*) Group Key: t2_1.x, t3_1.y, t3_1.x @@ -1514,8 +1618,10 @@ GROUP BY t3.y ORDER BY t3.y; Hash Cond: (t2_1.x = t3_1.x) -> Seq Scan on public.eager_agg_tab_ml_p2_s1 t2_1 Output: t2_1.y, t2_1.x + Bloom Filter 3: keys=(t2_1.x) -> Hash Output: t3_1.y, t3_1.x + Bloom Filter 3 -> Seq Scan on public.eager_agg_tab_ml_p2_s1 t3_1 Output: t3_1.y, t3_1.x -> Hash Join @@ -1523,8 +1629,10 @@ GROUP BY t3.y ORDER BY t3.y; Hash Cond: (t1_2.x = t2_2.x) -> Seq Scan on public.eager_agg_tab_ml_p2_s2 t1_2 Output: t1_2.x + Bloom Filter 6: keys=(t1_2.x) -> Hash Output: t2_2.x, t3_2.y, t3_2.x, (PARTIAL sum((t2_2.y + t3_2.y))), (PARTIAL count(*)) + Bloom Filter 6 -> Partial HashAggregate Output: t2_2.x, t3_2.y, t3_2.x, PARTIAL sum((t2_2.y + t3_2.y)), PARTIAL count(*) Group Key: t2_2.x, t3_2.y, t3_2.x @@ -1533,8 +1641,10 @@ GROUP BY t3.y ORDER BY t3.y; Hash Cond: (t2_2.x = t3_2.x) -> Seq Scan on public.eager_agg_tab_ml_p2_s2 t2_2 Output: t2_2.y, t2_2.x + Bloom Filter 5: keys=(t2_2.x) -> Hash Output: t3_2.y, t3_2.x + Bloom Filter 5 -> Seq Scan on public.eager_agg_tab_ml_p2_s2 t3_2 Output: t3_2.y, t3_2.x -> Hash Join @@ -1542,8 +1652,10 @@ GROUP BY t3.y ORDER BY t3.y; Hash Cond: (t1_3.x = t2_3.x) -> Seq Scan on public.eager_agg_tab_ml_p3_s1 t1_3 Output: t1_3.x + Bloom Filter 8: keys=(t1_3.x) -> Hash Output: t2_3.x, t3_3.y, t3_3.x, (PARTIAL sum((t2_3.y + t3_3.y))), (PARTIAL count(*)) + Bloom Filter 8 -> Partial HashAggregate Output: t2_3.x, t3_3.y, t3_3.x, PARTIAL sum((t2_3.y + t3_3.y)), PARTIAL count(*) Group Key: t2_3.x, t3_3.y, t3_3.x @@ -1552,8 +1664,10 @@ GROUP BY t3.y ORDER BY t3.y; Hash Cond: (t2_3.x = t3_3.x) -> Seq Scan on public.eager_agg_tab_ml_p3_s1 t2_3 Output: t2_3.y, t2_3.x + Bloom Filter 7: keys=(t2_3.x) -> Hash Output: t3_3.y, t3_3.x + Bloom Filter 7 -> Seq Scan on public.eager_agg_tab_ml_p3_s1 t3_3 Output: t3_3.y, t3_3.x -> Hash Join @@ -1561,8 +1675,10 @@ GROUP BY t3.y ORDER BY t3.y; Hash Cond: (t1_4.x = t2_4.x) -> Seq Scan on public.eager_agg_tab_ml_p3_s2 t1_4 Output: t1_4.x + Bloom Filter 10: keys=(t1_4.x) -> Hash Output: t2_4.x, t3_4.y, t3_4.x, (PARTIAL sum((t2_4.y + t3_4.y))), (PARTIAL count(*)) + Bloom Filter 10 -> Partial HashAggregate Output: t2_4.x, t3_4.y, t3_4.x, PARTIAL sum((t2_4.y + t3_4.y)), PARTIAL count(*) Group Key: t2_4.x, t3_4.y, t3_4.x @@ -1571,11 +1687,13 @@ GROUP BY t3.y ORDER BY t3.y; Hash Cond: (t2_4.x = t3_4.x) -> Seq Scan on public.eager_agg_tab_ml_p3_s2 t2_4 Output: t2_4.y, t2_4.x + Bloom Filter 9: keys=(t2_4.x) -> Hash Output: t3_4.y, t3_4.x + Bloom Filter 9 -> Seq Scan on public.eager_agg_tab_ml_p3_s2 t3_4 Output: t3_4.y, t3_4.x -(102 rows) +(122 rows) SELECT t3.y, sum(t2.y + t3.y), count(*) FROM eager_agg_tab_ml t1 @@ -1638,8 +1756,10 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t1.x = t2.x) -> Seq Scan on public.eager_agg_tab_ml_p1 t1 Output: t1.x + Bloom Filter 1: keys=(t1.x) -> Hash Output: t2.x, (PARTIAL sum(t2.y)), (PARTIAL count(*)) + Bloom Filter 1 -> Partial HashAggregate Output: t2.x, PARTIAL sum(t2.y), PARTIAL count(*) Group Key: t2.x @@ -1653,8 +1773,10 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t1_1.x = t2_1.x) -> Seq Scan on public.eager_agg_tab_ml_p2_s1 t1_1 Output: t1_1.x + Bloom Filter 2: keys=(t1_1.x) -> Hash Output: t2_1.x, (PARTIAL sum(t2_1.y)), (PARTIAL count(*)) + Bloom Filter 2 -> Partial HashAggregate Output: t2_1.x, PARTIAL sum(t2_1.y), PARTIAL count(*) Group Key: t2_1.x @@ -1668,8 +1790,10 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t1_2.x = t2_2.x) -> Seq Scan on public.eager_agg_tab_ml_p2_s2 t1_2 Output: t1_2.x + Bloom Filter 3: keys=(t1_2.x) -> Hash Output: t2_2.x, (PARTIAL sum(t2_2.y)), (PARTIAL count(*)) + Bloom Filter 3 -> Partial HashAggregate Output: t2_2.x, PARTIAL sum(t2_2.y), PARTIAL count(*) Group Key: t2_2.x @@ -1683,8 +1807,10 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t1_3.x = t2_3.x) -> Seq Scan on public.eager_agg_tab_ml_p3_s1 t1_3 Output: t1_3.x + Bloom Filter 4: keys=(t1_3.x) -> Hash Output: t2_3.x, (PARTIAL sum(t2_3.y)), (PARTIAL count(*)) + Bloom Filter 4 -> Partial HashAggregate Output: t2_3.x, PARTIAL sum(t2_3.y), PARTIAL count(*) Group Key: t2_3.x @@ -1698,14 +1824,16 @@ GROUP BY t1.x ORDER BY t1.x; Hash Cond: (t1_4.x = t2_4.x) -> Seq Scan on public.eager_agg_tab_ml_p3_s2 t1_4 Output: t1_4.x + Bloom Filter 5: keys=(t1_4.x) -> Hash Output: t2_4.x, (PARTIAL sum(t2_4.y)), (PARTIAL count(*)) + Bloom Filter 5 -> Partial HashAggregate Output: t2_4.x, PARTIAL sum(t2_4.y), PARTIAL count(*) Group Key: t2_4.x -> Seq Scan on public.eager_agg_tab_ml_p3_s2 t2_4 Output: t2_4.y, t2_4.x -(79 rows) +(89 rows) SELECT t1.x, sum(t2.y), count(*) FROM eager_agg_tab_ml t1 diff --git a/src/test/regress/expected/join.out b/src/test/regress/expected/join.out index 78bf022f7b4..5cbf2b554ab 100644 --- a/src/test/regress/expected/join.out +++ b/src/test/regress/expected/join.out @@ -1909,20 +1909,24 @@ select * from tenk1 a, tenk1 b where exists(select * from tenk1 c where b.twothousand = c.twothousand and b.fivethous <> c.fivethous) and a.tenthous = b.tenthous and a.tenthous < 5000; - QUERY PLAN ------------------------------------------------ + QUERY PLAN +-------------------------------------------------- Hash Semi Join Hash Cond: (b.twothousand = c.twothousand) Join Filter: (b.fivethous <> c.fivethous) -> Hash Join Hash Cond: (b.tenthous = a.tenthous) -> Seq Scan on tenk1 b + Bloom Filter 1: keys=(tenthous) + Bloom Filter 2: keys=(twothousand) -> Hash + Bloom Filter 1 -> Seq Scan on tenk1 a Filter: (tenthous < 5000) -> Hash + Bloom Filter 2 -> Seq Scan on tenk1 c -(11 rows) +(15 rows) -- -- More complicated constructs @@ -2378,9 +2382,11 @@ order by t1.unique1; Hash Cond: ((t1.two = t2.two) AND (t1.unique1 = (SubPlan expr_1))) -> Bitmap Heap Scan on tenk1 t1 Recheck Cond: (unique1 < 10) + Bloom Filter 1: keys=(two, unique1) -> Bitmap Index Scan on tenk1_unique1 Index Cond: (unique1 < 10) -> Hash + Bloom Filter 1 -> Bitmap Heap Scan on tenk1 t2 Recheck Cond: (unique1 < 10) -> Bitmap Index Scan on tenk1_unique1 @@ -2392,7 +2398,7 @@ order by t1.unique1; -> Limit -> Index Only Scan using tenk1_unique1 on tenk1 Index Cond: ((unique1 IS NOT NULL) AND (unique1 = t2.unique1)) -(20 rows) +(22 rows) -- Ensure we get the expected result select t1.unique1,t2.unique1 from tenk1 t1 @@ -2598,12 +2604,14 @@ select * from int4_tbl t1 Join Filter: (t2.f1 > 0) Filter: (t3.f1 IS NULL) -> Seq Scan on int4_tbl t2 + Bloom Filter 1: keys=(f1) -> Materialize -> Seq Scan on int4_tbl t3 -> Seq Scan on tenk1 t4 -> Hash + Bloom Filter 1 -> Seq Scan on int4_tbl t1 -(13 rows) +(15 rows) explain (costs off) select * from int4_tbl t1 @@ -2622,13 +2630,15 @@ select * from int4_tbl t1 Join Filter: (t2.f1 > 0) Filter: (t2.f1 <> COALESCE(t3.f1, '-1'::integer)) -> Seq Scan on int4_tbl t2 + Bloom Filter 1: keys=(f1) -> Materialize -> Seq Scan on int4_tbl t3 -> Hash + Bloom Filter 1 -> Seq Scan on int4_tbl t1 -> Materialize -> Seq Scan on tenk1 t4 -(14 rows) +(16 rows) explain (costs off) select * from onek t1 @@ -3122,10 +3132,12 @@ select count(*) from tenk1 a, tenk1 b -> Hash Join Hash Cond: (a.hundred = b.thousand) -> Index Only Scan using tenk1_hundred on tenk1 a + Bloom Filter 1: keys=(hundred) -> Hash + Bloom Filter 1 -> Seq Scan on tenk1 b Filter: ((fivethous % 10) < 10) -(7 rows) +(9 rows) select count(*) from tenk1 a, tenk1 b where a.hundred = b.thousand and (b.fivethous % 10) < 10; @@ -3168,11 +3180,13 @@ ORDER BY 1; Hash Cond: (b.f1 = c.f1) Filter: (COALESCE(c.f1, 0) = 0) -> Seq Scan on tt3 b + Bloom Filter 1: keys=(f1) -> Hash -> Seq Scan on tt3 c -> Hash + Bloom Filter 1 -> Seq Scan on tt4 a -(13 rows) +(15 rows) SELECT a.f1 FROM tt4 a @@ -3210,10 +3224,12 @@ where t1.filt = 5; Hash Join Hash Cond: (t2.val = t1.val) -> Seq Scan on skewedtable t2 + Bloom Filter 1: keys=(val) -> Hash + Bloom Filter 1 -> Seq Scan on skewedtable t1 Filter: (filt = 5) -(6 rows) +(8 rows) drop table skewedtable; -- @@ -3227,9 +3243,11 @@ where unique1 in (select unique2 from tenk1 b); Hash Semi Join Hash Cond: (a.unique1 = b.unique2) -> Seq Scan on tenk1 a + Bloom Filter 1: keys=(unique1) -> Hash + Bloom Filter 1 -> Index Only Scan using tenk1_unique2 on tenk1 b -(5 rows) +(7 rows) -- sadly, this is not an antijoin explain (costs off) @@ -3251,9 +3269,11 @@ where exists (select 1 from tenk1 b where a.unique1 = b.unique2); Hash Semi Join Hash Cond: (a.unique1 = b.unique2) -> Seq Scan on tenk1 a + Bloom Filter 1: keys=(unique1) -> Hash + Bloom Filter 1 -> Index Only Scan using tenk1_unique2 on tenk1 b -(5 rows) +(7 rows) explain (costs off) select a.* from tenk1 a @@ -3290,11 +3310,13 @@ where (hundred, thousand) in (select twothousand, twothousand from onek); Hash Cond: (tenk1.hundred = onek.twothousand) -> Seq Scan on tenk1 Filter: (hundred = thousand) + Bloom Filter 1: keys=(hundred) -> Hash + Bloom Filter 1 -> HashAggregate Group Key: onek.twothousand -> Seq Scan on onek -(8 rows) +(10 rows) reset enable_memoize; -- @@ -3311,17 +3333,19 @@ where t2.a is null; Hash Right Anti Join Hash Cond: (t2.b = t1.unique1) -> Seq Scan on tbl_anti t2 + Bloom Filter 1: keys=(b) -> Hash + Bloom Filter 1 -> Seq Scan on tenk1 t1 -(5 rows) +(7 rows) -- this is an antijoin, as t2.a is non-null for any matching row explain (costs off) select * from tenk1 t1 left join (tbl_anti t2 left join tbl_anti t3 on t2.c = t3.c) on t1.unique1 = t2.b where t2.a is null; - QUERY PLAN -------------------------------------------- + QUERY PLAN +---------------------------------------------- Hash Right Anti Join Hash Cond: (t2.b = t1.unique1) -> Merge Left Join @@ -3329,20 +3353,22 @@ where t2.a is null; -> Sort Sort Key: t2.c -> Seq Scan on tbl_anti t2 + Bloom Filter 1: keys=(b) -> Sort Sort Key: t3.c -> Seq Scan on tbl_anti t3 -> Hash + Bloom Filter 1 -> Seq Scan on tenk1 t1 -(12 rows) +(14 rows) -- this is not an antijoin, as t3.a can be nulled by t2/t3 join explain (costs off) select * from tenk1 t1 left join (tbl_anti t2 left join tbl_anti t3 on t2.c = t3.c) on t1.unique1 = t2.b where t3.a is null; - QUERY PLAN -------------------------------------------- + QUERY PLAN +---------------------------------------------- Hash Right Join Hash Cond: (t2.b = t1.unique1) Filter: (t3.a IS NULL) @@ -3351,12 +3377,14 @@ where t3.a is null; -> Sort Sort Key: t2.c -> Seq Scan on tbl_anti t2 + Bloom Filter 1: keys=(b) -> Sort Sort Key: t3.c -> Seq Scan on tbl_anti t3 -> Hash + Bloom Filter 1 -> Seq Scan on tenk1 t1 -(13 rows) +(15 rows) rollback; -- @@ -3370,9 +3398,11 @@ where exists (select 1 from tenk1 b where a.unique1 = b.unique2 group by b.uniqu Hash Semi Join Hash Cond: (a.unique1 = b.unique2) -> Seq Scan on tenk1 a + Bloom Filter 1: keys=(unique1) -> Hash + Bloom Filter 1 -> Index Only Scan using tenk1_unique2 on tenk1 b -(5 rows) +(7 rows) -- -- regression test for proper handling of outer joins within antijoins @@ -3989,11 +4019,13 @@ where q1 = thousand or q2 = thousand; -> Seq Scan on q2 -> Bitmap Heap Scan on tenk1 Recheck Cond: ((q1.q1 = thousand) OR (q2.q2 = thousand)) + Bloom Filter 1: keys=(twothousand) -> Bitmap Index Scan on tenk1_thous_tenthous Index Cond: (thousand = ANY (ARRAY[q1.q1, q2.q2])) -> Hash + Bloom Filter 1 -> Seq Scan on int4_tbl -(12 rows) +(14 rows) explain (costs off) select * from @@ -4010,11 +4042,13 @@ where thousand = (q1 + q2); -> Seq Scan on q2 -> Bitmap Heap Scan on tenk1 Recheck Cond: (thousand = (q1.q1 + q2.q2)) + Bloom Filter 1: keys=(twothousand) -> Bitmap Index Scan on tenk1_thous_tenthous Index Cond: (thousand = (q1.q1 + q2.q2)) -> Hash + Bloom Filter 1 -> Seq Scan on int4_tbl -(12 rows) +(14 rows) -- -- test ability to generate a suitable plan for a star-schema query @@ -4120,8 +4154,10 @@ where t1.unique1 < i4.f1; Hash Cond: (t2.ten = t1.tenthous) -> Seq Scan on public.tenk1 t2 Output: t2.unique1, t2.unique2, t2.two, t2.four, t2.ten, t2.twenty, t2.hundred, t2.thousand, t2.twothousand, t2.fivethous, t2.tenthous, t2.odd, t2.even, t2.stringu1, t2.stringu2, t2.string4 + Bloom Filter 1: keys=(t2.ten) -> Hash Output: t1.tenthous, t1.unique1 + Bloom Filter 1 -> Nested Loop Output: t1.tenthous, t1.unique1 -> Subquery Scan on ss0 @@ -4137,7 +4173,7 @@ where t1.unique1 < i4.f1; -> Seq Scan on public.int8_tbl i8 Output: i8.q1, i8.q2 Filter: (i8.q1 = ((64)::information_schema.cardinal_number)::integer) -(33 rows) +(35 rows) select ss1.d1 from tenk1 as t1 @@ -5234,6 +5270,7 @@ order by i0.f1, x; Output: i1.f1, i2.q1, i2.q2, '123'::bigint -> Seq Scan on public.int4_tbl i1 Output: i1.f1 + Bloom Filter 1: keys=(i1.f1) -> Materialize Output: i2.q1, i2.q2 -> Seq Scan on public.int8_tbl i2 @@ -5241,9 +5278,10 @@ order by i0.f1, x; Filter: (123 = i2.q2) -> Hash Output: i0.f1 + Bloom Filter 1 -> Seq Scan on public.int4_tbl i0 Output: i0.f1 -(19 rows) +(21 rows) select * from int4_tbl i0 left join @@ -5297,8 +5335,10 @@ select t1.* from Hash Cond: (i8.q1 = i8b2.q1) -> Seq Scan on public.int8_tbl i8 Output: i8.q1, i8.q2 + Bloom Filter 1: keys=(i8.q1) -> Hash Output: i8b2.q1, (NULL::integer) + Bloom Filter 1 -> Seq Scan on public.int8_tbl i8b2 Output: i8b2.q1, NULL::integer -> Hash @@ -5309,7 +5349,7 @@ select t1.* from Output: i4.f1 -> Seq Scan on public.int4_tbl i4 Output: i4.f1 -(30 rows) +(32 rows) select t1.* from text_tbl t1 @@ -5360,10 +5400,12 @@ select t1.* from Output: i8b2.q1, NULL::integer -> Seq Scan on public.int8_tbl i8b2 Output: i8b2.q1, i8b2.q2 + Bloom Filter 1: keys=(i8b2.q1) -> Materialize -> Seq Scan on public.int4_tbl i4b2 -> Hash Output: i8.q1, i8.q2 + Bloom Filter 1 -> Seq Scan on public.int8_tbl i8 Output: i8.q1, i8.q2 -> Hash @@ -5374,7 +5416,7 @@ select t1.* from Output: i4.f1 -> Seq Scan on public.int4_tbl i4 Output: i4.f1 -(34 rows) +(36 rows) select t1.* from text_tbl t1 @@ -5427,12 +5469,16 @@ select t1.* from Hash Cond: (i8b2.q1 = i4b2.f1) -> Seq Scan on public.int8_tbl i8b2 Output: i8b2.q1, i8b2.q2 + Bloom Filter 1: keys=(i8b2.q1) + Bloom Filter 2: keys=(i8b2.q1) -> Hash Output: i4b2.f1 + Bloom Filter 1 -> Seq Scan on public.int4_tbl i4b2 Output: i4b2.f1 -> Hash Output: i8.q1, i8.q2 + Bloom Filter 2 -> Seq Scan on public.int8_tbl i8 Output: i8.q1, i8.q2 -> Hash @@ -5443,7 +5489,7 @@ select t1.* from Output: i4.f1 -> Seq Scan on public.int4_tbl i4 Output: i4.f1 -(37 rows) +(41 rows) select t1.* from text_tbl t1 @@ -5794,15 +5840,17 @@ where ss1.c2 = 0; Filter: (i43.f1 = 0) -> Seq Scan on public.int4_tbl i41 Output: i41.f1 + Bloom Filter 1: keys=(i41.f1) -> Hash Output: i42.f1 + Bloom Filter 1 -> Seq Scan on public.int4_tbl i42 Output: i42.f1 -> Limit Output: (i41.f1), (i8.q1), (i8.q2), (i42.f1), (i43.f1), ((42)) -> Seq Scan on public.text_tbl Output: i41.f1, i8.q1, i8.q2, i42.f1, i43.f1, (42) -(25 rows) +(27 rows) select ss2.* from int4_tbl i41 @@ -5934,12 +5982,14 @@ select a.unique1, b.unique2 Hash Cond: (b.unique2 = a.unique1) -> Seq Scan on onek b Filter: (ANY ((unique2 = (SubPlan any_1).col1) AND ((random() > '0'::double precision) = (SubPlan any_1).col2))) + Bloom Filter 1: keys=(unique2) SubPlan any_1 -> Seq Scan on int8_tbl c Filter: (q1 < b.unique1) -> Hash + Bloom Filter 1 -> Index Only Scan using onek_unique1 on onek a -(9 rows) +(11 rows) select a.unique1, b.unique2 from onek a left join onek b on a.unique1 = b.unique2 @@ -6092,14 +6142,16 @@ explain (costs off) select id from a where id in ( select b.id from b left join c on b.id = c.id ); - QUERY PLAN ----------------------------- + QUERY PLAN +----------------------------------- Hash Join Hash Cond: (a.id = b.id) -> Seq Scan on a + Bloom Filter 1: keys=(id) -> Hash + Bloom Filter 1 -> Seq Scan on b -(5 rows) +(7 rows) -- check optimization with oddly-nested outer joins explain (costs off) @@ -6522,16 +6574,18 @@ explain (costs off) select c.id, ss.a from c left join (select d.a from onerow, d left join b on d.a = b.id) ss on c.id = ss.a; - QUERY PLAN --------------------------------- + QUERY PLAN +---------------------------------------- Hash Right Join Hash Cond: (d.a = c.id) -> Nested Loop -> Seq Scan on onerow -> Seq Scan on d + Bloom Filter 1: keys=(a) -> Hash + Bloom Filter 1 -> Seq Scan on c -(7 rows) +(9 rows) -- check the case when the placeholder relates to an outer join and its -- inner in the press field but actually uses only the outer side of the join @@ -6861,23 +6915,27 @@ where exists (select 1 from t t4 Hash Cond: (t6.b = t4.b) -> Seq Scan on pg_temp.t t6 Output: t6.a, t6.b + Bloom Filter 2: keys=(t6.b) -> Hash Output: t4.b, t5.b, t5.a + Bloom Filter 2 -> Hash Join Output: t4.b, t5.b, t5.a Inner Unique: true Hash Cond: (t5.b = t4.b) -> Seq Scan on pg_temp.t t5 Output: t5.a, t5.b + Bloom Filter 1: keys=(t5.b) -> Hash Output: t4.b, t4.a + Bloom Filter 1 -> Index Scan using t_a_key on pg_temp.t t4 Output: t4.b, t4.a Index Cond: (t4.a = 1) -> Index Only Scan using t_a_key on pg_temp.t t3 Output: t3.a Index Cond: (t3.a = t5.a) -(32 rows) +(36 rows) select t1.a from t t1 left join t t2 on t1.a = t2.a @@ -9063,13 +9121,15 @@ select * from Output: b.q1, COALESCE(b.q2, '42'::bigint) -> Seq Scan on public.int8_tbl d Output: d.q1, COALESCE((COALESCE(b.q2, '42'::bigint)), d.q2) + Bloom Filter 1: keys=(d.q1) -> Hash Output: c.q1, c.q2 + Bloom Filter 1 -> Seq Scan on public.int8_tbl c Output: c.q1, c.q2 -> Result Output: (COALESCE((COALESCE(b.q2, '42'::bigint)), d.q2)) -(24 rows) +(26 rows) -- another case requiring nested PlaceHolderVars explain (verbose, costs off) @@ -9128,25 +9188,29 @@ select c.*,a.*,ss1.q1,ss2.q1,ss3.* from Join Filter: (b.q1 < b2.f1) -> Seq Scan on public.int8_tbl b Output: b.q1, b.q2 + Bloom Filter 1: keys=(b.q1) -> Materialize Output: b2.f1 -> Seq Scan on public.int4_tbl b2 Output: b2.f1 -> Hash Output: a.q1, a.q2 + Bloom Filter 1 -> Seq Scan on public.int8_tbl a Output: a.q1, a.q2 -> Seq Scan on public.int8_tbl d Output: d.q1, COALESCE((COALESCE(b.q2, (b2.f1)::bigint)), d.q2) + Bloom Filter 2: keys=(d.q1) -> Hash Output: c.q1, c.q2 + Bloom Filter 2 -> Seq Scan on public.int8_tbl c Output: c.q1, c.q2 -> Materialize Output: i.f1 -> Seq Scan on public.int4_tbl i Output: i.f1 -(34 rows) +(38 rows) -- check processing of postponed quals (bug #9041) explain (verbose, costs off) @@ -9453,8 +9517,10 @@ select t1.b, ss.phv from join_ut1 t1 left join lateral Hash Cond: (t3.b = t2.a) -> Seq Scan on public.join_ut1 t3 Output: t3.a, t3.b, t3.c + Bloom Filter 1: keys=(t3.b) -> Hash Output: t2.a + Bloom Filter 1 -> Append -> Seq Scan on public.join_pt1p1p1 t2_1 Output: t2_1.a @@ -9462,7 +9528,7 @@ select t1.b, ss.phv from join_ut1 t1 left join lateral -> Seq Scan on public.join_pt1p2 t2_2 Output: t2_2.a Filter: (t1.a = t2_2.a) -(21 rows) +(23 rows) select t1.b, ss.phv from join_ut1 t1 left join lateral (select t2.a as t2a, t3.a t3a, least(t1.a, t2.a, t3.a) phv @@ -9496,12 +9562,14 @@ select * from fkest f1 Hash Cond: ((f2.x = f1.x) AND (f2.x10b = f1.x10)) -> Seq Scan on fkest f2 Filter: (x100 = 2) + Bloom Filter 1: keys=(x, x10b) -> Hash + Bloom Filter 1 -> Seq Scan on fkest f1 Filter: (x100 = 2) -> Index Scan using fkest_x_x10_x100_idx on fkest f3 Index Cond: (x = f1.x) -(10 rows) +(12 rows) alter table fkest add constraint fk foreign key (x, x10b, x100) references fkest (x, x10, x100); @@ -9517,13 +9585,15 @@ select * from fkest f1 -> Hash Join Hash Cond: (f3.x = f2.x) -> Seq Scan on fkest f3 + Bloom Filter 1: keys=(x) -> Hash + Bloom Filter 1 -> Seq Scan on fkest f2 Filter: (x100 = 2) -> Hash -> Seq Scan on fkest f1 Filter: (x100 = 2) -(11 rows) +(13 rows) rollback; -- @@ -9576,19 +9646,21 @@ analyze j3; -- ensure join is properly marked as unique explain (verbose, costs off) select * from j1 inner join j2 on j1.id = j2.id; - QUERY PLAN ------------------------------------ + QUERY PLAN +-------------------------------------- Hash Join Output: j1.id, j2.id Inner Unique: true Hash Cond: (j1.id = j2.id) -> Seq Scan on public.j1 Output: j1.id + Bloom Filter 1: keys=(j1.id) -> Hash Output: j2.id + Bloom Filter 1 -> Seq Scan on public.j2 Output: j2.id -(10 rows) +(12 rows) -- ensure join is not unique when not an equi-join explain (verbose, costs off) @@ -9609,19 +9681,21 @@ select * from j1 inner join j2 on j1.id > j2.id; -- ensure non-unique rel is not chosen as inner explain (verbose, costs off) select * from j1 inner join j3 on j1.id = j3.id; - QUERY PLAN ------------------------------------ + QUERY PLAN +-------------------------------------- Hash Join Output: j1.id, j3.id Inner Unique: true Hash Cond: (j3.id = j1.id) -> Seq Scan on public.j3 Output: j3.id + Bloom Filter 1: keys=(j3.id) -> Hash Output: j1.id + Bloom Filter 1 -> Seq Scan on public.j1 Output: j1.id -(10 rows) +(12 rows) -- ensure left join is marked as unique explain (verbose, costs off) @@ -9692,19 +9766,21 @@ select * from j1 cross join j2; -- ensure a natural join is marked as unique explain (verbose, costs off) select * from j1 natural join j2; - QUERY PLAN ------------------------------------ + QUERY PLAN +-------------------------------------- Hash Join Output: j1.id Inner Unique: true Hash Cond: (j1.id = j2.id) -> Seq Scan on public.j1 Output: j1.id + Bloom Filter 1: keys=(j1.id) -> Hash Output: j2.id + Bloom Filter 1 -> Seq Scan on public.j2 Output: j2.id -(10 rows) +(12 rows) -- ensure a distinct clause allows the inner to become unique explain (verbose, costs off) diff --git a/src/test/regress/expected/join_hash.out b/src/test/regress/expected/join_hash.out index 75009e29720..0a8ade8b961 100644 --- a/src/test/regress/expected/join_hash.out +++ b/src/test/regress/expected/join_hash.out @@ -90,15 +90,17 @@ set local work_mem = '4MB'; set local hash_mem_multiplier = 1.0; explain (costs off) select count(*) from simple r join simple s using (id); - QUERY PLAN ----------------------------------------- + QUERY PLAN +----------------------------------------- Aggregate -> Hash Join Hash Cond: (r.id = s.id) -> Seq Scan on simple r + Bloom Filter 1: keys=(id) -> Hash + Bloom Filter 1 -> Seq Scan on simple s -(6 rows) +(8 rows) select count(*) from simple r join simple s using (id); count @@ -203,15 +205,17 @@ set local work_mem = '128kB'; set local hash_mem_multiplier = 1.0; explain (costs off) select count(*) from simple r join simple s using (id); - QUERY PLAN ----------------------------------------- + QUERY PLAN +----------------------------------------- Aggregate -> Hash Join Hash Cond: (r.id = s.id) -> Seq Scan on simple r + Bloom Filter 1: keys=(id) -> Hash + Bloom Filter 1 -> Seq Scan on simple s -(6 rows) +(8 rows) select count(*) from simple r join simple s using (id); count @@ -330,9 +334,11 @@ explain (costs off) -> Hash Join Hash Cond: (r.id = s.id) -> Seq Scan on simple r + Bloom Filter 1: keys=(id) -> Hash + Bloom Filter 1 -> Seq Scan on bigger_than_it_looks s -(6 rows) +(8 rows) select count(*) FROM simple r JOIN bigger_than_it_looks s USING (id); count @@ -445,9 +451,11 @@ explain (costs off) -> Hash Join Hash Cond: (r.id = s.id) -> Seq Scan on simple r + Bloom Filter 1: keys=(id) -> Hash + Bloom Filter 1 -> Seq Scan on extremely_skewed s -(6 rows) +(8 rows) select count(*) from simple r join extremely_skewed s using (id); count @@ -1149,9 +1157,11 @@ lateral (select t1.fivethous, i4.f1 from tenk1 t1 join int4_tbl i4 -> Hash Join Hash Cond: (t1.fivethous = (i4.f1 + i8.q2)) -> Seq Scan on tenk1 t1 + Bloom Filter 1: keys=(fivethous) -> Hash + Bloom Filter 1 -> Seq Scan on int4_tbl i4 -(9 rows) +(11 rows) select i8.q2, ss.* from int8_tbl i8, diff --git a/src/test/regress/expected/merge.out b/src/test/regress/expected/merge.out index 9cb1d87066a..c5aa11cd249 100644 --- a/src/test/regress/expected/merge.out +++ b/src/test/regress/expected/merge.out @@ -323,15 +323,17 @@ USING source AS s ON t.tid = s.sid WHEN MATCHED THEN UPDATE SET balance = 0; - QUERY PLAN ----------------------------------------- + QUERY PLAN +------------------------------------------ Merge on target t -> Hash Join Hash Cond: (s.sid = t.tid) -> Seq Scan on source s + Bloom Filter 1: keys=(sid) -> Hash + Bloom Filter 1 -> Seq Scan on target t -(6 rows) +(8 rows) EXPLAIN (COSTS OFF) MERGE INTO target t @@ -339,15 +341,17 @@ USING source AS s ON t.tid = s.sid WHEN MATCHED THEN DELETE; - QUERY PLAN ----------------------------------------- + QUERY PLAN +------------------------------------------ Merge on target t -> Hash Join Hash Cond: (s.sid = t.tid) -> Seq Scan on source s + Bloom Filter 1: keys=(sid) -> Hash + Bloom Filter 1 -> Seq Scan on target t -(6 rows) +(8 rows) EXPLAIN (COSTS OFF) MERGE INTO target t @@ -1831,8 +1835,10 @@ WHEN MATCHED AND t.c > s.cnt THEN Join Filter: (t.b < (SubPlan expr_1)) -> Seq Scan on public.tgt t Output: t.ctid, t.a, t.b + Bloom Filter 1: keys=(t.a) -> Hash Output: s.a, s.b, s.c, s.d, s.ctid + Bloom Filter 1 -> Seq Scan on public.src s Output: s.a, s.b, s.c, s.d, s.ctid SubPlan expr_1 @@ -1856,7 +1862,7 @@ WHEN MATCHED AND t.c > s.cnt THEN -> Seq Scan on public.ref r_1 Output: r_1.ab, r_1.cd Filter: ((r_1.ab = (s.a + s.b)) AND (r_1.cd = (s.c - s.d))) -(32 rows) +(34 rows) DROP TABLE src, tgt, ref; -- Subqueries diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index c3261bff209..b52528870ef 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -582,10 +582,12 @@ WHERE my_int_eq(a.unique2, 42); Hash Join Hash Cond: (b.unique1 = a.unique1) -> Seq Scan on tenk1 b + Bloom Filter 1: keys=(unique1) -> Hash + Bloom Filter 1 -> Seq Scan on tenk1 a Filter: my_int_eq(unique2, 42) -(6 rows) +(8 rows) -- With support function that knows it's int4eq, we get a different plan CREATE FUNCTION test_support_func(internal) diff --git a/src/test/regress/expected/partition_aggregate.out b/src/test/regress/expected/partition_aggregate.out index c30304b99c7..be56036461b 100644 --- a/src/test/regress/expected/partition_aggregate.out +++ b/src/test/regress/expected/partition_aggregate.out @@ -460,23 +460,29 @@ SELECT t1.x, sum(t1.y), count(*) FROM pagg_tab1 t1, pagg_tab2 t2 WHERE t1.x = t2 -> Hash Join Hash Cond: (t1.x = t2.y) -> Seq Scan on pagg_tab1_p1 t1 + Bloom Filter 1: keys=(x) -> Hash + Bloom Filter 1 -> Seq Scan on pagg_tab2_p1 t2 -> HashAggregate Group Key: t1_1.x -> Hash Join Hash Cond: (t1_1.x = t2_1.y) -> Seq Scan on pagg_tab1_p2 t1_1 + Bloom Filter 2: keys=(x) -> Hash + Bloom Filter 2 -> Seq Scan on pagg_tab2_p2 t2_1 -> HashAggregate Group Key: t1_2.x -> Hash Join Hash Cond: (t2_2.y = t1_2.x) -> Seq Scan on pagg_tab2_p3 t2_2 + Bloom Filter 3: keys=(y) -> Hash + Bloom Filter 3 -> Seq Scan on pagg_tab1_p3 t1_2 -(24 rows) +(30 rows) SELECT t1.x, sum(t1.y), count(*) FROM pagg_tab1 t1, pagg_tab2 t2 WHERE t1.x = t2.y GROUP BY t1.x ORDER BY 1, 2, 3; x | sum | count @@ -533,23 +539,29 @@ SELECT t2.y, sum(t1.y), count(*) FROM pagg_tab1 t1, pagg_tab2 t2 WHERE t1.x = t2 -> Hash Join Hash Cond: (t1.x = t2.y) -> Seq Scan on pagg_tab1_p1 t1 + Bloom Filter 1: keys=(x) -> Hash + Bloom Filter 1 -> Seq Scan on pagg_tab2_p1 t2 -> HashAggregate Group Key: t2_1.y -> Hash Join Hash Cond: (t1_1.x = t2_1.y) -> Seq Scan on pagg_tab1_p2 t1_1 + Bloom Filter 2: keys=(x) -> Hash + Bloom Filter 2 -> Seq Scan on pagg_tab2_p2 t2_1 -> HashAggregate Group Key: t2_2.y -> Hash Join Hash Cond: (t2_2.y = t1_2.x) -> Seq Scan on pagg_tab2_p3 t2_2 + Bloom Filter 3: keys=(y) -> Hash + Bloom Filter 3 -> Seq Scan on pagg_tab1_p3 t1_2 -(24 rows) +(30 rows) -- When GROUP BY clause does not match; partial aggregation is performed for each partition. -- Also test GroupAggregate paths by disabling hash aggregates. @@ -572,7 +584,9 @@ SELECT t1.y, sum(t1.x), count(*) FROM pagg_tab1 t1, pagg_tab2 t2 WHERE t1.x = t2 -> Hash Join Hash Cond: (t1.x = t2.y) -> Seq Scan on pagg_tab1_p1 t1 + Bloom Filter 1: keys=(x) -> Hash + Bloom Filter 1 -> Seq Scan on pagg_tab2_p1 t2 -> Partial GroupAggregate Group Key: t1_1.y @@ -581,7 +595,9 @@ SELECT t1.y, sum(t1.x), count(*) FROM pagg_tab1 t1, pagg_tab2 t2 WHERE t1.x = t2 -> Hash Join Hash Cond: (t1_1.x = t2_1.y) -> Seq Scan on pagg_tab1_p2 t1_1 + Bloom Filter 2: keys=(x) -> Hash + Bloom Filter 2 -> Seq Scan on pagg_tab2_p2 t2_1 -> Partial GroupAggregate Group Key: t1_2.y @@ -590,9 +606,11 @@ SELECT t1.y, sum(t1.x), count(*) FROM pagg_tab1 t1, pagg_tab2 t2 WHERE t1.x = t2 -> Hash Join Hash Cond: (t2_2.y = t1_2.x) -> Seq Scan on pagg_tab2_p3 t2_2 + Bloom Filter 3: keys=(y) -> Hash + Bloom Filter 3 -> Seq Scan on pagg_tab1_p3 t1_2 -(34 rows) +(40 rows) SELECT t1.y, sum(t1.x), count(*) FROM pagg_tab1 t1, pagg_tab2 t2 WHERE t1.x = t2.y GROUP BY t1.y HAVING avg(t1.x) > 10 ORDER BY 1, 2, 3; y | sum | count @@ -638,9 +656,11 @@ SELECT b.y, sum(a.y) FROM pagg_tab1 a LEFT JOIN pagg_tab2 b ON a.x = b.y GROUP B -> Hash Right Join Hash Cond: (b_2.y = a_2.x) -> Seq Scan on pagg_tab2_p3 b_2 + Bloom Filter 1: keys=(y) -> Hash + Bloom Filter 1 -> Seq Scan on pagg_tab1_p3 a_2 -(26 rows) +(28 rows) SELECT b.y, sum(a.y) FROM pagg_tab1 a LEFT JOIN pagg_tab2 b ON a.x = b.y GROUP BY b.y ORDER BY 1 NULLS LAST; y | sum @@ -667,14 +687,18 @@ SELECT b.y, sum(a.y) FROM pagg_tab1 a RIGHT JOIN pagg_tab2 b ON a.x = b.y GROUP -> Hash Right Join Hash Cond: (a.x = b.y) -> Seq Scan on pagg_tab1_p1 a + Bloom Filter 1: keys=(x) -> Hash + Bloom Filter 1 -> Seq Scan on pagg_tab2_p1 b -> HashAggregate Group Key: b_1.y -> Hash Right Join Hash Cond: (a_1.x = b_1.y) -> Seq Scan on pagg_tab1_p2 a_1 + Bloom Filter 2: keys=(x) -> Hash + Bloom Filter 2 -> Seq Scan on pagg_tab2_p2 b_1 -> HashAggregate Group Key: b_2.y @@ -683,7 +707,7 @@ SELECT b.y, sum(a.y) FROM pagg_tab1 a RIGHT JOIN pagg_tab2 b ON a.x = b.y GROUP -> Seq Scan on pagg_tab2_p3 b_2 -> Hash -> Seq Scan on pagg_tab1_p3 a_2 -(24 rows) +(28 rows) SELECT b.y, sum(a.y) FROM pagg_tab1 a RIGHT JOIN pagg_tab2 b ON a.x = b.y GROUP BY b.y ORDER BY 1 NULLS LAST; y | sum diff --git a/src/test/regress/expected/partition_join.out b/src/test/regress/expected/partition_join.out index 38643d41fd7..1906b3641a3 100644 --- a/src/test/regress/expected/partition_join.out +++ b/src/test/regress/expected/partition_join.out @@ -36,22 +36,28 @@ SELECT t1.a, t1.c, t2.b, t2.c FROM prt1 t1, prt2 t2 WHERE t1.a = t2.b AND t1.b = -> Hash Join Hash Cond: (t2_1.b = t1_1.a) -> Seq Scan on prt2_p1 t2_1 + Bloom Filter 1: keys=(b) -> Hash + Bloom Filter 1 -> Seq Scan on prt1_p1 t1_1 Filter: (b = 0) -> Hash Join Hash Cond: (t2_2.b = t1_2.a) -> Seq Scan on prt2_p2 t2_2 + Bloom Filter 2: keys=(b) -> Hash + Bloom Filter 2 -> Seq Scan on prt1_p2 t1_2 Filter: (b = 0) -> Hash Join Hash Cond: (t2_3.b = t1_3.a) -> Seq Scan on prt2_p3 t2_3 + Bloom Filter 3: keys=(b) -> Hash + Bloom Filter 3 -> Seq Scan on prt1_p3 t1_3 Filter: (b = 0) -(21 rows) +(27 rows) SELECT t1.a, t1.c, t2.b, t2.c FROM prt1 t1, prt2 t2 WHERE t1.a = t2.b AND t1.b = 0 ORDER BY t1.a, t2.b; a | c | b | c @@ -77,22 +83,28 @@ SELECT t1.a, t1.c, t2.b, t2.c FROM prt1 t1, prt2 t2 WHERE t1.a = t2.a AND t1.a = -> Hash Join Hash Cond: (t1_1.a = t2_1.a) -> Seq Scan on prt1_p1 t1_1 + Bloom Filter 1: keys=(a) -> Hash + Bloom Filter 1 -> Seq Scan on prt2_p1 t2_1 Filter: (a = b) -> Hash Join Hash Cond: (t1_2.a = t2_2.a) -> Seq Scan on prt1_p2 t1_2 + Bloom Filter 2: keys=(a) -> Hash + Bloom Filter 2 -> Seq Scan on prt2_p2 t2_2 Filter: (a = b) -> Hash Join Hash Cond: (t1_3.a = t2_3.a) -> Seq Scan on prt1_p3 t1_3 + Bloom Filter 3: keys=(a) -> Hash + Bloom Filter 3 -> Seq Scan on prt2_p3 t2_3 Filter: (a = b) -(21 rows) +(27 rows) SELECT t1.a, t1.c, t2.b, t2.c FROM prt1 t1, prt2 t2 WHERE t1.a = t2.a AND t1.a = t2.b ORDER BY t1.a, t2.b; a | c | b | c @@ -202,13 +214,17 @@ SELECT t1.a, t1.c, t2.b, t2.c FROM prt1 t1 RIGHT JOIN prt2 t2 ON t1.a = t2.b WHE -> Hash Right Join Hash Cond: (t1_1.a = t2_1.b) -> Seq Scan on prt1_p1 t1_1 + Bloom Filter 1: keys=(a) -> Hash + Bloom Filter 1 -> Seq Scan on prt2_p1 t2_1 Filter: (a = 0) -> Hash Right Join Hash Cond: (t1_2.a = t2_2.b) -> Seq Scan on prt1_p2 t1_2 + Bloom Filter 2: keys=(a) -> Hash + Bloom Filter 2 -> Seq Scan on prt2_p2 t2_2 Filter: (a = 0) -> Nested Loop Left Join @@ -216,7 +232,7 @@ SELECT t1.a, t1.c, t2.b, t2.c FROM prt1 t1 RIGHT JOIN prt2 t2 ON t1.a = t2.b WHE Filter: (a = 0) -> Index Scan using iprt1_p3_a on prt1_p3 t1_3 Index Cond: (a = t2_3.b) -(20 rows) +(24 rows) SELECT t1.a, t1.c, t2.b, t2.c FROM prt1 t1 RIGHT JOIN prt2 t2 ON t1.a = t2.b WHERE t2.a = 0 ORDER BY t1.a, t2.b; a | c | b | c @@ -283,10 +299,12 @@ SELECT t1.a, t1.c, t2.b, t2.c FROM prt1 t1, prt2 t2 WHERE t1.a = t2.b AND t1.a < Hash Cond: (t2.b = t1.a) -> Seq Scan on prt2_p2 t2 Filter: (b > 250) + Bloom Filter 1: keys=(b) -> Hash + Bloom Filter 1 -> Seq Scan on prt1_p2 t1 Filter: ((a < 450) AND (b = 0)) -(9 rows) +(11 rows) SELECT t1.a, t1.c, t2.b, t2.c FROM prt1 t1, prt2 t2 WHERE t1.a = t2.b AND t1.a < 450 AND t2.b > 250 AND t1.b = 0 ORDER BY t1.a, t2.b; a | c | b | c @@ -382,14 +400,18 @@ SELECT t1.* FROM prt1 t1 WHERE t1.a IN (SELECT t2.b FROM prt2 t2 WHERE t2.a = 0) Hash Cond: (t1_1.a = t2_1.b) -> Seq Scan on prt1_p1 t1_1 Filter: (b = 0) + Bloom Filter 1: keys=(a) -> Hash + Bloom Filter 1 -> Seq Scan on prt2_p1 t2_1 Filter: (a = 0) -> Hash Semi Join Hash Cond: (t1_2.a = t2_2.b) -> Seq Scan on prt1_p2 t1_2 Filter: (b = 0) + Bloom Filter 2: keys=(a) -> Hash + Bloom Filter 2 -> Seq Scan on prt2_p2 t2_2 Filter: (a = 0) -> Nested Loop Semi Join @@ -399,7 +421,7 @@ SELECT t1.* FROM prt1 t1 WHERE t1.a IN (SELECT t2.b FROM prt2 t2 WHERE t2.a = 0) -> Materialize -> Seq Scan on prt2_p3 t2_3 Filter: (a = 0) -(24 rows) +(28 rows) SELECT t1.* FROM prt1 t1 WHERE t1.a IN (SELECT t2.b FROM prt2 t2 WHERE t2.a = 0) AND t1.b = 0 ORDER BY t1.a; a | b | c @@ -515,19 +537,25 @@ SELECT t1.a, ss.t2a, ss.t2c FROM prt1 t1 LEFT JOIN LATERAL -> Hash Join Hash Cond: (t2_1.a = t3_1.b) -> Seq Scan on prt1_p1 t2_1 + Bloom Filter 1: keys=(a) -> Hash + Bloom Filter 1 -> Seq Scan on prt2_p1 t3_1 -> Hash Join Hash Cond: (t2_2.a = t3_2.b) -> Seq Scan on prt1_p2 t2_2 + Bloom Filter 2: keys=(a) -> Hash + Bloom Filter 2 -> Seq Scan on prt2_p2 t3_2 -> Hash Join Hash Cond: (t2_3.a = t3_3.b) -> Seq Scan on prt1_p3 t2_3 + Bloom Filter 3: keys=(a) -> Hash + Bloom Filter 3 -> Seq Scan on prt2_p3 t3_3 -(26 rows) +(32 rows) SELECT t1.a, ss.t2a, ss.t2c FROM prt1 t1 LEFT JOIN LATERAL (SELECT t2.a AS t2a, t3.a AS t3a, t2.b t2b, t2.c t2c, least(t1.a,t2.a,t3.a) FROM prt1 t2 JOIN prt2 t3 ON (t2.a = t3.b)) ss @@ -728,29 +756,41 @@ SELECT * FROM prt1 t1 JOIN prt1 t2 ON t1.a = t2.a WHERE t1.a IN (SELECT a FROM p -> Hash Join Hash Cond: (t1_1.a = t2_1.a) -> Seq Scan on prt1_p1 t1_1 + Bloom Filter 1: keys=(a) + Bloom Filter 2: keys=(a) -> Hash + Bloom Filter 1 -> Seq Scan on prt1_p1 t2_1 -> Hash + Bloom Filter 2 -> Seq Scan on prt1_p1 t3_1 -> Hash Semi Join Hash Cond: (t1_2.a = t3_2.a) -> Hash Join Hash Cond: (t1_2.a = t2_2.a) -> Seq Scan on prt1_p2 t1_2 + Bloom Filter 3: keys=(a) + Bloom Filter 4: keys=(a) -> Hash + Bloom Filter 3 -> Seq Scan on prt1_p2 t2_2 -> Hash + Bloom Filter 4 -> Seq Scan on prt1_p2 t3_2 -> Hash Semi Join Hash Cond: (t1_3.a = t3_3.a) -> Hash Join Hash Cond: (t1_3.a = t2_3.a) -> Seq Scan on prt1_p3 t1_3 + Bloom Filter 5: keys=(a) + Bloom Filter 6: keys=(a) -> Hash + Bloom Filter 5 -> Seq Scan on prt1_p3 t2_3 -> Hash + Bloom Filter 6 -> Seq Scan on prt1_p3 t3_3 -(28 rows) +(40 rows) -- -- partitioned by expression @@ -821,7 +861,9 @@ SELECT t1.a, t1.c, t2.b, t2.c, t3.a + t3.b, t3.c FROM prt1 t1, prt2 t2, prt1_e t -> Hash Join Hash Cond: (t2_1.b = t1_1.a) -> Seq Scan on prt2_p1 t2_1 + Bloom Filter 1: keys=(b) -> Hash + Bloom Filter 1 -> Seq Scan on prt1_p1 t1_1 Filter: (b = 0) -> Index Scan using iprt1_e_p1_ab2 on prt1_e_p1 t3_1 @@ -831,7 +873,9 @@ SELECT t1.a, t1.c, t2.b, t2.c, t3.a + t3.b, t3.c FROM prt1 t1, prt2 t2, prt1_e t -> Hash Join Hash Cond: (t2_2.b = t1_2.a) -> Seq Scan on prt2_p2 t2_2 + Bloom Filter 2: keys=(b) -> Hash + Bloom Filter 2 -> Seq Scan on prt1_p2 t1_2 Filter: (b = 0) -> Index Scan using iprt1_e_p2_ab2 on prt1_e_p2 t3_2 @@ -841,12 +885,14 @@ SELECT t1.a, t1.c, t2.b, t2.c, t3.a + t3.b, t3.c FROM prt1 t1, prt2 t2, prt1_e t -> Hash Join Hash Cond: (t2_3.b = t1_3.a) -> Seq Scan on prt2_p3 t2_3 + Bloom Filter 3: keys=(b) -> Hash + Bloom Filter 3 -> Seq Scan on prt1_p3 t1_3 Filter: (b = 0) -> Index Scan using iprt1_e_p3_ab2 on prt1_e_p3 t3_3 Index Cond: (((a + b) / 2) = t2_3.b) -(33 rows) +(39 rows) SELECT t1.a, t1.c, t2.b, t2.c, t3.a + t3.b, t3.c FROM prt1 t1, prt2 t2, prt1_e t3 WHERE t1.a = t2.b AND t1.a = (t3.a + t3.b)/2 AND t1.b = 0 ORDER BY t1.a, t2.b; a | c | b | c | ?column? | c @@ -871,7 +917,9 @@ SELECT t1.a, t1.c, t2.b, t2.c, t3.a + t3.b, t3.c FROM (prt1 t1 LEFT JOIN prt2 t2 -> Hash Right Join Hash Cond: (t2_1.b = t1_1.a) -> Seq Scan on prt2_p1 t2_1 + Bloom Filter 1: keys=(b) -> Hash + Bloom Filter 1 -> Seq Scan on prt1_p1 t1_1 Filter: (b = 0) -> Hash Right Join @@ -881,7 +929,9 @@ SELECT t1.a, t1.c, t2.b, t2.c, t3.a + t3.b, t3.c FROM (prt1 t1 LEFT JOIN prt2 t2 -> Hash Right Join Hash Cond: (t2_2.b = t1_2.a) -> Seq Scan on prt2_p2 t2_2 + Bloom Filter 2: keys=(b) -> Hash + Bloom Filter 2 -> Seq Scan on prt1_p2 t1_2 Filter: (b = 0) -> Hash Right Join @@ -891,10 +941,12 @@ SELECT t1.a, t1.c, t2.b, t2.c, t3.a + t3.b, t3.c FROM (prt1 t1 LEFT JOIN prt2 t2 -> Hash Right Join Hash Cond: (t2_3.b = t1_3.a) -> Seq Scan on prt2_p3 t2_3 + Bloom Filter 3: keys=(b) -> Hash + Bloom Filter 3 -> Seq Scan on prt1_p3 t1_3 Filter: (b = 0) -(33 rows) +(39 rows) SELECT t1.a, t1.c, t2.b, t2.c, t3.a + t3.b, t3.c FROM (prt1 t1 LEFT JOIN prt2 t2 ON t1.a = t2.b) LEFT JOIN prt1_e t3 ON (t1.a = (t3.a + t3.b)/2) WHERE t1.b = 0 ORDER BY t1.a, t2.b, t3.a + t3.b; a | c | b | c | ?column? | c @@ -924,7 +976,9 @@ SELECT t1.a, t1.c, t2.b, t2.c, t3.a + t3.b, t3.c FROM (prt1 t1 LEFT JOIN prt2 t2 -> Hash Right Join Hash Cond: (t1_1.a = ((t3_1.a + t3_1.b) / 2)) -> Seq Scan on prt1_p1 t1_1 + Bloom Filter 1: keys=(a) -> Hash + Bloom Filter 1 -> Seq Scan on prt1_e_p1 t3_1 Filter: (c = 0) -> Index Scan using iprt2_p1_b on prt2_p1 t2_1 @@ -933,7 +987,9 @@ SELECT t1.a, t1.c, t2.b, t2.c, t3.a + t3.b, t3.c FROM (prt1 t1 LEFT JOIN prt2 t2 -> Hash Right Join Hash Cond: (t1_2.a = ((t3_2.a + t3_2.b) / 2)) -> Seq Scan on prt1_p2 t1_2 + Bloom Filter 2: keys=(a) -> Hash + Bloom Filter 2 -> Seq Scan on prt1_e_p2 t3_2 Filter: (c = 0) -> Index Scan using iprt2_p2_b on prt2_p2 t2_2 @@ -942,12 +998,14 @@ SELECT t1.a, t1.c, t2.b, t2.c, t3.a + t3.b, t3.c FROM (prt1 t1 LEFT JOIN prt2 t2 -> Hash Right Join Hash Cond: (t1_3.a = ((t3_3.a + t3_3.b) / 2)) -> Seq Scan on prt1_p3 t1_3 + Bloom Filter 3: keys=(a) -> Hash + Bloom Filter 3 -> Seq Scan on prt1_e_p3 t3_3 Filter: (c = 0) -> Index Scan using iprt2_p3_b on prt2_p3 t2_3 Index Cond: (b = t1_3.a) -(30 rows) +(36 rows) SELECT t1.a, t1.c, t2.b, t2.c, t3.a + t3.b, t3.c FROM (prt1 t1 LEFT JOIN prt2 t2 ON t1.a = t2.b) RIGHT JOIN prt1_e t3 ON (t1.a = (t3.a + t3.b)/2) WHERE t3.c = 0 ORDER BY t1.a, t2.b, t3.a + t3.b; a | c | b | c | ?column? | c @@ -1205,7 +1263,9 @@ SELECT t1.* FROM prt1 t1 WHERE t1.a IN (SELECT t1.b FROM prt2 t1 WHERE t1.b IN ( -> Hash Semi Join Hash Cond: (t1_6.b = ((t1_9.a + t1_9.b) / 2)) -> Seq Scan on prt2_p1 t1_6 + Bloom Filter 1: keys=(b) -> Hash + Bloom Filter 1 -> Seq Scan on prt1_e_p1 t1_9 Filter: (c = 0) -> Index Scan using iprt1_p1_a on prt1_p1 t1_3 @@ -1218,7 +1278,9 @@ SELECT t1.* FROM prt1 t1 WHERE t1.a IN (SELECT t1.b FROM prt2 t1 WHERE t1.b IN ( -> Hash Semi Join Hash Cond: (t1_7.b = ((t1_10.a + t1_10.b) / 2)) -> Seq Scan on prt2_p2 t1_7 + Bloom Filter 2: keys=(b) -> Hash + Bloom Filter 2 -> Seq Scan on prt1_e_p2 t1_10 Filter: (c = 0) -> Index Scan using iprt1_p2_a on prt1_p2 t1_4 @@ -1231,13 +1293,15 @@ SELECT t1.* FROM prt1 t1 WHERE t1.a IN (SELECT t1.b FROM prt2 t1 WHERE t1.b IN ( -> Hash Semi Join Hash Cond: (t1_8.b = ((t1_11.a + t1_11.b) / 2)) -> Seq Scan on prt2_p3 t1_8 + Bloom Filter 3: keys=(b) -> Hash + Bloom Filter 3 -> Seq Scan on prt1_e_p3 t1_11 Filter: (c = 0) -> Index Scan using iprt1_p3_a on prt1_p3 t1_5 Index Cond: (a = t1_8.b) Filter: (b = 0) -(41 rows) +(47 rows) SELECT t1.* FROM prt1 t1 WHERE t1.a IN (SELECT t1.b FROM prt2 t1 WHERE t1.b IN (SELECT (t1.a + t1.b)/2 FROM prt1_e t1 WHERE t1.c = 0)) AND t1.b = 0 ORDER BY t1.a; a | b | c @@ -1567,29 +1631,41 @@ SELECT avg(t1.a), avg(t2.b), avg(t3.a + t3.b), t1.c, t2.c, t3.c FROM plt1 t1, pl -> Hash Join Hash Cond: ((t1_1.b = t2_1.b) AND (t1_1.c = t2_1.c)) -> Seq Scan on plt1_p1 t1_1 + Bloom Filter 1: keys=(b, c) + Bloom Filter 2: keys=(c) -> Hash + Bloom Filter 1 -> Seq Scan on plt2_p1 t2_1 -> Hash + Bloom Filter 2 -> Seq Scan on plt1_e_p1 t3_1 -> Hash Join Hash Cond: (t1_2.c = ltrim(t3_2.c, 'A'::text)) -> Hash Join Hash Cond: ((t1_2.b = t2_2.b) AND (t1_2.c = t2_2.c)) -> Seq Scan on plt1_p2 t1_2 + Bloom Filter 3: keys=(b, c) + Bloom Filter 4: keys=(c) -> Hash + Bloom Filter 3 -> Seq Scan on plt2_p2 t2_2 -> Hash + Bloom Filter 4 -> Seq Scan on plt1_e_p2 t3_2 -> Hash Join Hash Cond: (t1_3.c = ltrim(t3_3.c, 'A'::text)) -> Hash Join Hash Cond: ((t1_3.b = t2_3.b) AND (t1_3.c = t2_3.c)) -> Seq Scan on plt1_p3 t1_3 + Bloom Filter 5: keys=(b, c) + Bloom Filter 6: keys=(c) -> Hash + Bloom Filter 5 -> Seq Scan on plt2_p3 t2_3 -> Hash + Bloom Filter 6 -> Seq Scan on plt1_e_p3 t3_3 -(32 rows) +(44 rows) SELECT avg(t1.a), avg(t2.b), avg(t3.a + t3.b), t1.c, t2.c, t3.c FROM plt1 t1, plt2 t2, plt1_e t3 WHERE t1.b = t2.b AND t1.c = t2.c AND ltrim(t3.c, 'A') = t1.c GROUP BY t1.c, t2.c, t3.c ORDER BY t1.c, t2.c, t3.c; avg | avg | avg | c | c | c @@ -1637,23 +1713,29 @@ SELECT t1.a, t1.c, t2.b, t2.c FROM (SELECT * FROM prt1 WHERE a = 1 AND a = 2) t1 -> Hash Join Hash Cond: (t3_1.a = t2_1.b) -> Seq Scan on prt1_p1 t3_1 + Bloom Filter 1: keys=(a) -> Hash + Bloom Filter 1 -> Seq Scan on prt2_p1 t2_1 -> Hash Join Hash Cond: (t3_2.a = t2_2.b) -> Seq Scan on prt1_p2 t3_2 + Bloom Filter 2: keys=(a) -> Hash + Bloom Filter 2 -> Seq Scan on prt2_p2 t2_2 -> Hash Join Hash Cond: (t3_3.a = t2_3.b) -> Seq Scan on prt1_p3 t3_3 + Bloom Filter 3: keys=(a) -> Hash + Bloom Filter 3 -> Seq Scan on prt2_p3 t2_3 -> Hash -> Result Replaces: Scan on prt1 One-Time Filter: false -(22 rows) +(28 rows) EXPLAIN (COSTS OFF) SELECT t1.a, t1.c, t2.b, t2.c FROM (SELECT * FROM prt1 WHERE a = 1 AND a = 2) t1 FULL JOIN prt2 t2 ON t1.a = t2.b WHERE t2.a = 0 ORDER BY t1.a, t2.b; @@ -1715,29 +1797,41 @@ SELECT avg(t1.a), avg(t2.b), avg(t3.a + t3.b), t1.c, t2.c, t3.c FROM pht1 t1, ph -> Hash Join Hash Cond: ((t1_1.b = t2_1.b) AND (t1_1.c = t2_1.c)) -> Seq Scan on pht1_p1 t1_1 + Bloom Filter 1: keys=(b, c) + Bloom Filter 2: keys=(c) -> Hash + Bloom Filter 1 -> Seq Scan on pht2_p1 t2_1 -> Hash + Bloom Filter 2 -> Seq Scan on pht1_e_p1 t3_1 -> Hash Join Hash Cond: (t1_2.c = ltrim(t3_2.c, 'A'::text)) -> Hash Join Hash Cond: ((t1_2.b = t2_2.b) AND (t1_2.c = t2_2.c)) -> Seq Scan on pht1_p2 t1_2 + Bloom Filter 3: keys=(b, c) + Bloom Filter 4: keys=(c) -> Hash + Bloom Filter 3 -> Seq Scan on pht2_p2 t2_2 -> Hash + Bloom Filter 4 -> Seq Scan on pht1_e_p2 t3_2 -> Hash Join Hash Cond: (t1_3.c = ltrim(t3_3.c, 'A'::text)) -> Hash Join Hash Cond: ((t1_3.b = t2_3.b) AND (t1_3.c = t2_3.c)) -> Seq Scan on pht1_p3 t1_3 + Bloom Filter 5: keys=(b, c) + Bloom Filter 6: keys=(c) -> Hash + Bloom Filter 5 -> Seq Scan on pht2_p3 t2_3 -> Hash + Bloom Filter 6 -> Seq Scan on pht1_e_p3 t3_3 -(32 rows) +(44 rows) SELECT avg(t1.a), avg(t2.b), avg(t3.a + t3.b), t1.c, t2.c, t3.c FROM pht1 t1, pht2 t2, pht1_e t3 WHERE t1.b = t2.b AND t1.c = t2.c AND ltrim(t3.c, 'A') = t1.c GROUP BY t1.c, t2.c, t3.c ORDER BY t1.c, t2.c, t3.c; avg | avg | avg | c | c | c @@ -1767,22 +1861,28 @@ SELECT t1.a, t1.c, t2.b, t2.c FROM prt1 t1, prt2 t2 WHERE t1.a = t2.b AND t1.b = -> Hash Join Hash Cond: (t2_1.b = t1_1.a) -> Seq Scan on prt2_p1 t2_1 + Bloom Filter 1: keys=(b) -> Hash + Bloom Filter 1 -> Seq Scan on prt1_p1 t1_1 Filter: (b = 0) -> Hash Join Hash Cond: (t2_2.b = t1_2.a) -> Seq Scan on prt2_p2 t2_2 + Bloom Filter 2: keys=(b) -> Hash + Bloom Filter 2 -> Seq Scan on prt1_p2 t1_2 Filter: (b = 0) -> Hash Join Hash Cond: (t2_3.b = t1_3.a) -> Seq Scan on prt2_p3 t2_3 + Bloom Filter 3: keys=(b) -> Hash + Bloom Filter 3 -> Seq Scan on prt1_p3 t1_3 Filter: (b = 0) -(21 rows) +(27 rows) -- test default partition behavior for list ALTER TABLE plt1 DETACH PARTITION plt1_p3; @@ -1803,22 +1903,28 @@ SELECT avg(t1.a), avg(t2.b), t1.c, t2.c FROM plt1 t1 RIGHT JOIN plt2 t2 ON t1.c -> Hash Join Hash Cond: (t2_1.c = t1_1.c) -> Seq Scan on plt2_p1 t2_1 + Bloom Filter 1: keys=(c) -> Hash + Bloom Filter 1 -> Seq Scan on plt1_p1 t1_1 Filter: ((a % 25) = 0) -> Hash Join Hash Cond: (t2_2.c = t1_2.c) -> Seq Scan on plt2_p2 t2_2 + Bloom Filter 2: keys=(c) -> Hash + Bloom Filter 2 -> Seq Scan on plt1_p2 t1_2 Filter: ((a % 25) = 0) -> Hash Join Hash Cond: (t2_3.c = t1_3.c) -> Seq Scan on plt2_p3 t2_3 + Bloom Filter 3: keys=(c) -> Hash + Bloom Filter 3 -> Seq Scan on plt1_p3 t1_3 Filter: ((a % 25) = 0) -(23 rows) +(29 rows) -- -- multiple levels of partitioning @@ -1854,7 +1960,9 @@ SELECT t1.a, t1.c, t2.b, t2.c FROM prt1_l t1, prt2_l t2 WHERE t1.a = t2.b AND t1 -> Hash Join Hash Cond: (t2_1.b = t1_1.a) -> Seq Scan on prt2_l_p1 t2_1 + Bloom Filter 1: keys=(b) -> Hash + Bloom Filter 1 -> Seq Scan on prt1_l_p1 t1_1 Filter: (b = 0) -> Hash Join @@ -1876,7 +1984,7 @@ SELECT t1.a, t1.c, t2.b, t2.c FROM prt1_l t1, prt2_l t2 WHERE t1.a = t2.b AND t1 -> Hash -> Seq Scan on prt1_l_p3_p1 t1_5 Filter: (b = 0) -(28 rows) +(30 rows) SELECT t1.a, t1.c, t2.b, t2.c FROM prt1_l t1, prt2_l t2 WHERE t1.a = t2.b AND t1.b = 0 ORDER BY t1.a, t2.b; a | c | b | c @@ -2355,19 +2463,25 @@ SELECT t1.a, t1.c, t2.b, t2.c FROM prt1 t1, prt4_n t2, prt2 t3 WHERE t1.a = t2.a -> Hash Join Hash Cond: (t1_1.a = t3_1.b) -> Seq Scan on prt1_p1 t1_1 + Bloom Filter 1: keys=(a) -> Hash + Bloom Filter 1 -> Seq Scan on prt2_p1 t3_1 -> Hash Join Hash Cond: (t1_2.a = t3_2.b) -> Seq Scan on prt1_p2 t1_2 + Bloom Filter 2: keys=(a) -> Hash + Bloom Filter 2 -> Seq Scan on prt2_p2 t3_2 -> Hash Join Hash Cond: (t1_3.a = t3_3.b) -> Seq Scan on prt1_p3 t1_3 + Bloom Filter 3: keys=(a) -> Hash + Bloom Filter 3 -> Seq Scan on prt2_p3 t3_3 -(23 rows) +(29 rows) -- partitionwise join can not be applied if there are no equi-join conditions -- between partition keys @@ -2639,22 +2753,28 @@ SELECT t1.a, t1.c, t2.b, t2.c FROM prt1_adv t1 INNER JOIN prt2_adv t2 ON (t1.a = -> Hash Join Hash Cond: (t2_1.b = t1_1.a) -> Seq Scan on prt2_adv_p1 t2_1 + Bloom Filter 1: keys=(b) -> Hash + Bloom Filter 1 -> Seq Scan on prt1_adv_p1 t1_1 Filter: (b = 0) -> Hash Join Hash Cond: (t2_2.b = t1_2.a) -> Seq Scan on prt2_adv_p2 t2_2 + Bloom Filter 2: keys=(b) -> Hash + Bloom Filter 2 -> Seq Scan on prt1_adv_p2 t1_2 Filter: (b = 0) -> Hash Join Hash Cond: (t2_3.b = t1_3.a) -> Seq Scan on prt2_adv_p3 t2_3 + Bloom Filter 3: keys=(b) -> Hash + Bloom Filter 3 -> Seq Scan on prt1_adv_p3 t1_3 Filter: (b = 0) -(21 rows) +(27 rows) SELECT t1.a, t1.c, t2.b, t2.c FROM prt1_adv t1 INNER JOIN prt2_adv t2 ON (t1.a = t2.b) WHERE t1.b = 0 ORDER BY t1.a, t2.b; a | c | b | c @@ -2680,22 +2800,28 @@ SELECT t1.* FROM prt1_adv t1 WHERE EXISTS (SELECT 1 FROM prt2_adv t2 WHERE t1.a -> Hash Right Semi Join Hash Cond: (t2_1.b = t1_1.a) -> Seq Scan on prt2_adv_p1 t2_1 + Bloom Filter 1: keys=(b) -> Hash + Bloom Filter 1 -> Seq Scan on prt1_adv_p1 t1_1 Filter: (b = 0) -> Hash Right Semi Join Hash Cond: (t2_2.b = t1_2.a) -> Seq Scan on prt2_adv_p2 t2_2 + Bloom Filter 2: keys=(b) -> Hash + Bloom Filter 2 -> Seq Scan on prt1_adv_p2 t1_2 Filter: (b = 0) -> Hash Right Semi Join Hash Cond: (t2_3.b = t1_3.a) -> Seq Scan on prt2_adv_p3 t2_3 + Bloom Filter 3: keys=(b) -> Hash + Bloom Filter 3 -> Seq Scan on prt1_adv_p3 t1_3 Filter: (b = 0) -(21 rows) +(27 rows) SELECT t1.* FROM prt1_adv t1 WHERE EXISTS (SELECT 1 FROM prt2_adv t2 WHERE t1.a = t2.b) AND t1.b = 0 ORDER BY t1.a; a | b | c @@ -2721,22 +2847,28 @@ SELECT t1.a, t1.c, t2.b, t2.c FROM prt1_adv t1 LEFT JOIN prt2_adv t2 ON (t1.a = -> Hash Right Join Hash Cond: (t2_1.b = t1_1.a) -> Seq Scan on prt2_adv_p1 t2_1 + Bloom Filter 1: keys=(b) -> Hash + Bloom Filter 1 -> Seq Scan on prt1_adv_p1 t1_1 Filter: (b = 0) -> Hash Right Join Hash Cond: (t2_2.b = t1_2.a) -> Seq Scan on prt2_adv_p2 t2_2 + Bloom Filter 2: keys=(b) -> Hash + Bloom Filter 2 -> Seq Scan on prt1_adv_p2 t1_2 Filter: (b = 0) -> Hash Right Join Hash Cond: (t2_3.b = t1_3.a) -> Seq Scan on prt2_adv_p3 t2_3 + Bloom Filter 3: keys=(b) -> Hash + Bloom Filter 3 -> Seq Scan on prt1_adv_p3 t1_3 Filter: (b = 0) -(21 rows) +(27 rows) SELECT t1.a, t1.c, t2.b, t2.c FROM prt1_adv t1 LEFT JOIN prt2_adv t2 ON (t1.a = t2.b) WHERE t1.b = 0 ORDER BY t1.a, t2.b; a | c | b | c @@ -2766,22 +2898,28 @@ SELECT t1.* FROM prt1_adv t1 WHERE NOT EXISTS (SELECT 1 FROM prt2_adv t2 WHERE t -> Hash Right Anti Join Hash Cond: (t2_1.b = t1_1.a) -> Seq Scan on prt2_adv_p1 t2_1 + Bloom Filter 1: keys=(b) -> Hash + Bloom Filter 1 -> Seq Scan on prt1_adv_p1 t1_1 Filter: (b = 0) -> Hash Right Anti Join Hash Cond: (t2_2.b = t1_2.a) -> Seq Scan on prt2_adv_p2 t2_2 + Bloom Filter 2: keys=(b) -> Hash + Bloom Filter 2 -> Seq Scan on prt1_adv_p2 t1_2 Filter: (b = 0) -> Hash Right Anti Join Hash Cond: (t2_3.b = t1_3.a) -> Seq Scan on prt2_adv_p3 t2_3 + Bloom Filter 3: keys=(b) -> Hash + Bloom Filter 3 -> Seq Scan on prt1_adv_p3 t1_3 Filter: (b = 0) -(21 rows) +(27 rows) SELECT t1.* FROM prt1_adv t1 WHERE NOT EXISTS (SELECT 1 FROM prt2_adv t2 WHERE t1.a = t2.b) AND t1.b = 0 ORDER BY t1.a; a | b | c @@ -2848,22 +2986,28 @@ SELECT t1.a, t1.c, t2.b, t2.c FROM prt1_adv t1 INNER JOIN prt2_adv t2 ON (t1.a = -> Hash Join Hash Cond: (t2_1.b = t1_1.a) -> Seq Scan on prt2_adv_p1 t2_1 + Bloom Filter 1: keys=(b) -> Hash + Bloom Filter 1 -> Seq Scan on prt1_adv_p1 t1_1 Filter: (b = 0) -> Hash Join Hash Cond: (t2_2.b = t1_2.a) -> Seq Scan on prt2_adv_p2 t2_2 + Bloom Filter 2: keys=(b) -> Hash + Bloom Filter 2 -> Seq Scan on prt1_adv_p2 t1_2 Filter: (b = 0) -> Hash Join Hash Cond: (t2_3.b = t1_3.a) -> Seq Scan on prt2_adv_p3 t2_3 + Bloom Filter 3: keys=(b) -> Hash + Bloom Filter 3 -> Seq Scan on prt1_adv_p3 t1_3 Filter: (b = 0) -(21 rows) +(27 rows) SELECT t1.a, t1.c, t2.b, t2.c FROM prt1_adv t1 INNER JOIN prt2_adv t2 ON (t1.a = t2.b) WHERE t1.b = 0 ORDER BY t1.a, t2.b; a | c | b | c @@ -2889,22 +3033,28 @@ SELECT t1.* FROM prt1_adv t1 WHERE EXISTS (SELECT 1 FROM prt2_adv t2 WHERE t1.a -> Hash Right Semi Join Hash Cond: (t2_1.b = t1_1.a) -> Seq Scan on prt2_adv_p1 t2_1 + Bloom Filter 1: keys=(b) -> Hash + Bloom Filter 1 -> Seq Scan on prt1_adv_p1 t1_1 Filter: (b = 0) -> Hash Right Semi Join Hash Cond: (t2_2.b = t1_2.a) -> Seq Scan on prt2_adv_p2 t2_2 + Bloom Filter 2: keys=(b) -> Hash + Bloom Filter 2 -> Seq Scan on prt1_adv_p2 t1_2 Filter: (b = 0) -> Hash Right Semi Join Hash Cond: (t2_3.b = t1_3.a) -> Seq Scan on prt2_adv_p3 t2_3 + Bloom Filter 3: keys=(b) -> Hash + Bloom Filter 3 -> Seq Scan on prt1_adv_p3 t1_3 Filter: (b = 0) -(21 rows) +(27 rows) SELECT t1.* FROM prt1_adv t1 WHERE EXISTS (SELECT 1 FROM prt2_adv t2 WHERE t1.a = t2.b) AND t1.b = 0 ORDER BY t1.a; a | b | c @@ -2930,22 +3080,28 @@ SELECT t1.a, t1.c, t2.b, t2.c FROM prt1_adv t1 LEFT JOIN prt2_adv t2 ON (t1.a = -> Hash Right Join Hash Cond: (t2_1.b = t1_1.a) -> Seq Scan on prt2_adv_p1 t2_1 + Bloom Filter 1: keys=(b) -> Hash + Bloom Filter 1 -> Seq Scan on prt1_adv_p1 t1_1 Filter: (b = 0) -> Hash Right Join Hash Cond: (t2_2.b = t1_2.a) -> Seq Scan on prt2_adv_p2 t2_2 + Bloom Filter 2: keys=(b) -> Hash + Bloom Filter 2 -> Seq Scan on prt1_adv_p2 t1_2 Filter: (b = 0) -> Hash Right Join Hash Cond: (t2_3.b = t1_3.a) -> Seq Scan on prt2_adv_p3 t2_3 + Bloom Filter 3: keys=(b) -> Hash + Bloom Filter 3 -> Seq Scan on prt1_adv_p3 t1_3 Filter: (b = 0) -(21 rows) +(27 rows) SELECT t1.a, t1.c, t2.b, t2.c FROM prt1_adv t1 LEFT JOIN prt2_adv t2 ON (t1.a = t2.b) WHERE t1.b = 0 ORDER BY t1.a, t2.b; a | c | b | c @@ -3001,22 +3157,28 @@ SELECT t1.* FROM prt1_adv t1 WHERE NOT EXISTS (SELECT 1 FROM prt2_adv t2 WHERE t -> Hash Right Anti Join Hash Cond: (t2_1.b = t1_1.a) -> Seq Scan on prt2_adv_p1 t2_1 + Bloom Filter 1: keys=(b) -> Hash + Bloom Filter 1 -> Seq Scan on prt1_adv_p1 t1_1 Filter: (b = 0) -> Hash Right Anti Join Hash Cond: (t2_2.b = t1_2.a) -> Seq Scan on prt2_adv_p2 t2_2 + Bloom Filter 2: keys=(b) -> Hash + Bloom Filter 2 -> Seq Scan on prt1_adv_p2 t1_2 Filter: (b = 0) -> Hash Right Anti Join Hash Cond: (t2_3.b = t1_3.a) -> Seq Scan on prt2_adv_p3 t2_3 + Bloom Filter 3: keys=(b) -> Hash + Bloom Filter 3 -> Seq Scan on prt1_adv_p3 t1_3 Filter: (b = 0) -(21 rows) +(27 rows) SELECT t1.* FROM prt1_adv t1 WHERE NOT EXISTS (SELECT 1 FROM prt2_adv t2 WHERE t1.a = t2.b) AND t1.b = 0 ORDER BY t1.a; a | b | c @@ -3102,24 +3264,32 @@ SELECT t1.b, t1.c, t2.a, t2.c, t3.a, t3.c FROM prt2_adv t1 LEFT JOIN prt1_adv t2 -> Hash Right Join Hash Cond: (t2_2.a = t1_2.b) -> Seq Scan on prt1_adv_p2 t2_2 + Bloom Filter 2: keys=(a) -> Hash + Bloom Filter 2 -> Hash Join Hash Cond: (t3_2.a = t1_2.b) -> Seq Scan on prt1_adv_p2 t3_2 + Bloom Filter 1: keys=(a) -> Hash + Bloom Filter 1 -> Seq Scan on prt2_adv_p2 t1_2 Filter: (a = 0) -> Hash Right Join Hash Cond: (t2_3.a = t1_3.b) -> Seq Scan on prt1_adv_p3 t2_3 + Bloom Filter 4: keys=(a) -> Hash + Bloom Filter 4 -> Hash Join Hash Cond: (t3_3.a = t1_3.b) -> Seq Scan on prt1_adv_p3 t3_3 + Bloom Filter 3: keys=(a) -> Hash + Bloom Filter 3 -> Seq Scan on prt2_adv_p3 t1_3 Filter: (a = 0) -(31 rows) +(39 rows) SELECT t1.b, t1.c, t2.a, t2.c, t3.a, t3.c FROM prt2_adv t1 LEFT JOIN prt1_adv t2 ON (t1.b = t2.a) INNER JOIN prt1_adv t3 ON (t1.b = t3.a) WHERE t1.a = 0 ORDER BY t1.b, t2.a, t3.a; b | c | a | c | a | c @@ -3289,16 +3459,20 @@ SELECT t1.a, t1.c, t2.b, t2.c FROM prt1_adv t1 INNER JOIN prt2_adv t2 ON (t1.a = -> Hash Join Hash Cond: (t2_1.b = t1_2.a) -> Seq Scan on prt2_adv_p1 t2_1 + Bloom Filter 1: keys=(b) -> Hash + Bloom Filter 1 -> Seq Scan on prt1_adv_p1 t1_2 Filter: (b = 0) -> Hash Join Hash Cond: (t2_2.b = t1_1.a) -> Seq Scan on prt2_adv_p2 t2_2 + Bloom Filter 2: keys=(b) -> Hash + Bloom Filter 2 -> Seq Scan on prt1_adv_p2 t1_1 Filter: (b = 0) -(15 rows) +(19 rows) SELECT t1.a, t1.c, t2.b, t2.c FROM prt1_adv t1 INNER JOIN prt2_adv t2 ON (t1.a = t2.b) WHERE t1.b = 0 ORDER BY t1.a, t2.b; a | c | b | c @@ -3390,24 +3564,32 @@ SELECT t1.a, t1.c, t2.b, t2.c, t3.a, t3.c FROM prt1_adv t1 LEFT JOIN prt2_adv t2 -> Hash Right Join Hash Cond: (t3_1.a = t1_1.a) -> Seq Scan on prt3_adv_p1 t3_1 + Bloom Filter 2: keys=(a) -> Hash + Bloom Filter 2 -> Hash Right Join Hash Cond: (t2_2.b = t1_1.a) -> Seq Scan on prt2_adv_p2 t2_2 + Bloom Filter 1: keys=(b) -> Hash + Bloom Filter 1 -> Seq Scan on prt1_adv_p2 t1_1 Filter: (b = 0) -> Hash Right Join Hash Cond: (t3_2.a = t1_2.a) -> Seq Scan on prt3_adv_p2 t3_2 + Bloom Filter 4: keys=(a) -> Hash + Bloom Filter 4 -> Hash Right Join Hash Cond: (t2_1.b = t1_2.a) -> Seq Scan on prt2_adv_p1 t2_1 + Bloom Filter 3: keys=(b) -> Hash + Bloom Filter 3 -> Seq Scan on prt1_adv_p1 t1_2 Filter: (b = 0) -(23 rows) +(31 rows) SELECT t1.a, t1.c, t2.b, t2.c, t3.a, t3.c FROM prt1_adv t1 LEFT JOIN prt2_adv t2 ON (t1.a = t2.b) LEFT JOIN prt3_adv t3 ON (t1.a = t3.a) WHERE t1.b = 0 ORDER BY t1.a, t2.b, t3.a; a | c | b | c | a | c @@ -3449,16 +3631,20 @@ SELECT t1.a, t1.c, t2.b, t2.c FROM prt1_adv t1 INNER JOIN prt2_adv t2 ON (t1.a = -> Hash Join Hash Cond: (t2_1.b = t1_1.a) -> Seq Scan on prt2_adv_p1 t2_1 + Bloom Filter 1: keys=(b) -> Hash + Bloom Filter 1 -> Seq Scan on prt1_adv_p1 t1_1 Filter: ((a < 300) AND (b = 0)) -> Hash Join Hash Cond: (t2_2.b = t1_2.a) -> Seq Scan on prt2_adv_p2 t2_2 + Bloom Filter 2: keys=(b) -> Hash + Bloom Filter 2 -> Seq Scan on prt1_adv_p2 t1_2 Filter: ((a < 300) AND (b = 0)) -(15 rows) +(19 rows) SELECT t1.a, t1.c, t2.b, t2.c FROM prt1_adv t1 INNER JOIN prt2_adv t2 ON (t1.a = t2.b) WHERE t1.a < 300 AND t1.b = 0 ORDER BY t1.a, t2.b; a | c | b | c @@ -3488,16 +3674,20 @@ SELECT t1.a, t1.c, t2.b, t2.c FROM prt1_adv t1 INNER JOIN prt2_adv t2 ON (t1.a = -> Hash Join Hash Cond: (t2_1.b = t1_1.a) -> Seq Scan on prt2_adv_p1 t2_1 + Bloom Filter 1: keys=(b) -> Hash + Bloom Filter 1 -> Seq Scan on prt1_adv_p1 t1_1 Filter: ((a >= 100) AND (a < 300) AND (b = 0)) -> Hash Join Hash Cond: (t2_2.b = t1_2.a) -> Seq Scan on prt2_adv_p2 t2_2 + Bloom Filter 2: keys=(b) -> Hash + Bloom Filter 2 -> Seq Scan on prt1_adv_p2 t1_2 Filter: ((a >= 100) AND (a < 300) AND (b = 0)) -(15 rows) +(19 rows) SELECT t1.a, t1.c, t2.b, t2.c FROM prt1_adv t1 INNER JOIN prt2_adv t2 ON (t1.a = t2.b) WHERE t1.a >= 100 AND t1.a < 300 AND t1.b = 0 ORDER BY t1.a, t2.b; a | c | b | c @@ -3538,22 +3728,28 @@ SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 INNER JOIN plt2_adv t2 ON (t1.a = -> Hash Join Hash Cond: ((t2_1.a = t1_1.a) AND (t2_1.c = t1_1.c)) -> Seq Scan on plt2_adv_p1 t2_1 + Bloom Filter 1: keys=(a, c) -> Hash + Bloom Filter 1 -> Seq Scan on plt1_adv_p1 t1_1 Filter: (b < 10) -> Hash Join Hash Cond: ((t2_2.a = t1_2.a) AND (t2_2.c = t1_2.c)) -> Seq Scan on plt2_adv_p2 t2_2 + Bloom Filter 2: keys=(a, c) -> Hash + Bloom Filter 2 -> Seq Scan on plt1_adv_p2 t1_2 Filter: (b < 10) -> Hash Join Hash Cond: ((t2_3.a = t1_3.a) AND (t2_3.c = t1_3.c)) -> Seq Scan on plt2_adv_p3 t2_3 + Bloom Filter 3: keys=(a, c) -> Hash + Bloom Filter 3 -> Seq Scan on plt1_adv_p3 t1_3 Filter: (b < 10) -(21 rows) +(27 rows) SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 INNER JOIN plt2_adv t2 ON (t1.a = t2.a AND t1.c = t2.c) WHERE t1.b < 10 ORDER BY t1.a; a | c | a | c @@ -3575,22 +3771,28 @@ SELECT t1.* FROM plt1_adv t1 WHERE EXISTS (SELECT 1 FROM plt2_adv t2 WHERE t1.a -> Hash Right Semi Join Hash Cond: ((t2_1.a = t1_1.a) AND (t2_1.c = t1_1.c)) -> Seq Scan on plt2_adv_p1 t2_1 + Bloom Filter 1: keys=(a, c) -> Hash + Bloom Filter 1 -> Seq Scan on plt1_adv_p1 t1_1 Filter: (b < 10) -> Hash Right Semi Join Hash Cond: ((t2_2.a = t1_2.a) AND (t2_2.c = t1_2.c)) -> Seq Scan on plt2_adv_p2 t2_2 + Bloom Filter 2: keys=(a, c) -> Hash + Bloom Filter 2 -> Seq Scan on plt1_adv_p2 t1_2 Filter: (b < 10) -> Hash Right Semi Join Hash Cond: ((t2_3.a = t1_3.a) AND (t2_3.c = t1_3.c)) -> Seq Scan on plt2_adv_p3 t2_3 + Bloom Filter 3: keys=(a, c) -> Hash + Bloom Filter 3 -> Seq Scan on plt1_adv_p3 t1_3 Filter: (b < 10) -(21 rows) +(27 rows) SELECT t1.* FROM plt1_adv t1 WHERE EXISTS (SELECT 1 FROM plt2_adv t2 WHERE t1.a = t2.a AND t1.c = t2.c) AND t1.b < 10 ORDER BY t1.a; a | b | c @@ -3612,22 +3814,28 @@ SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 LEFT JOIN plt2_adv t2 ON (t1.a = -> Hash Right Join Hash Cond: ((t2_1.a = t1_1.a) AND (t2_1.c = t1_1.c)) -> Seq Scan on plt2_adv_p1 t2_1 + Bloom Filter 1: keys=(a, c) -> Hash + Bloom Filter 1 -> Seq Scan on plt1_adv_p1 t1_1 Filter: (b < 10) -> Hash Right Join Hash Cond: ((t2_2.a = t1_2.a) AND (t2_2.c = t1_2.c)) -> Seq Scan on plt2_adv_p2 t2_2 + Bloom Filter 2: keys=(a, c) -> Hash + Bloom Filter 2 -> Seq Scan on plt1_adv_p2 t1_2 Filter: (b < 10) -> Hash Right Join Hash Cond: ((t2_3.a = t1_3.a) AND (t2_3.c = t1_3.c)) -> Seq Scan on plt2_adv_p3 t2_3 + Bloom Filter 3: keys=(a, c) -> Hash + Bloom Filter 3 -> Seq Scan on plt1_adv_p3 t1_3 Filter: (b < 10) -(21 rows) +(27 rows) SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 LEFT JOIN plt2_adv t2 ON (t1.a = t2.a AND t1.c = t2.c) WHERE t1.b < 10 ORDER BY t1.a; a | c | a | c @@ -3651,22 +3859,28 @@ SELECT t1.* FROM plt1_adv t1 WHERE NOT EXISTS (SELECT 1 FROM plt2_adv t2 WHERE t -> Hash Right Anti Join Hash Cond: ((t2_1.a = t1_1.a) AND (t2_1.c = t1_1.c)) -> Seq Scan on plt2_adv_p1 t2_1 + Bloom Filter 1: keys=(a, c) -> Hash + Bloom Filter 1 -> Seq Scan on plt1_adv_p1 t1_1 Filter: (b < 10) -> Hash Right Anti Join Hash Cond: ((t2_2.a = t1_2.a) AND (t2_2.c = t1_2.c)) -> Seq Scan on plt2_adv_p2 t2_2 + Bloom Filter 2: keys=(a, c) -> Hash + Bloom Filter 2 -> Seq Scan on plt1_adv_p2 t1_2 Filter: (b < 10) -> Hash Right Anti Join Hash Cond: ((t2_3.a = t1_3.a) AND (t2_3.c = t1_3.c)) -> Seq Scan on plt2_adv_p3 t2_3 + Bloom Filter 3: keys=(a, c) -> Hash + Bloom Filter 3 -> Seq Scan on plt1_adv_p3 t1_3 Filter: (b < 10) -(21 rows) +(27 rows) SELECT t1.* FROM plt1_adv t1 WHERE NOT EXISTS (SELECT 1 FROM plt2_adv t2 WHERE t1.a = t2.a AND t1.c = t2.c) AND t1.b < 10 ORDER BY t1.a; a | b | c @@ -3731,22 +3945,28 @@ SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 INNER JOIN plt2_adv t2 ON (t1.a = -> Hash Join Hash Cond: ((t2_1.a = t1_1.a) AND (t2_1.c = t1_1.c)) -> Seq Scan on plt2_adv_p1 t2_1 + Bloom Filter 1: keys=(a, c) -> Hash + Bloom Filter 1 -> Seq Scan on plt1_adv_p1 t1_1 Filter: (b < 10) -> Hash Join Hash Cond: ((t2_2.a = t1_2.a) AND (t2_2.c = t1_2.c)) -> Seq Scan on plt2_adv_p2 t2_2 + Bloom Filter 2: keys=(a, c) -> Hash + Bloom Filter 2 -> Seq Scan on plt1_adv_p2 t1_2 Filter: (b < 10) -> Hash Join Hash Cond: ((t2_3.a = t1_3.a) AND (t2_3.c = t1_3.c)) -> Seq Scan on plt2_adv_p3 t2_3 + Bloom Filter 3: keys=(a, c) -> Hash + Bloom Filter 3 -> Seq Scan on plt1_adv_p3 t1_3 Filter: (b < 10) -(21 rows) +(27 rows) SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 INNER JOIN plt2_adv t2 ON (t1.a = t2.a AND t1.c = t2.c) WHERE t1.b < 10 ORDER BY t1.a; a | c | a | c @@ -3768,22 +3988,28 @@ SELECT t1.* FROM plt1_adv t1 WHERE EXISTS (SELECT 1 FROM plt2_adv t2 WHERE t1.a -> Hash Right Semi Join Hash Cond: ((t2_1.a = t1_1.a) AND (t2_1.c = t1_1.c)) -> Seq Scan on plt2_adv_p1 t2_1 + Bloom Filter 1: keys=(a, c) -> Hash + Bloom Filter 1 -> Seq Scan on plt1_adv_p1 t1_1 Filter: (b < 10) -> Hash Right Semi Join Hash Cond: ((t2_2.a = t1_2.a) AND (t2_2.c = t1_2.c)) -> Seq Scan on plt2_adv_p2 t2_2 + Bloom Filter 2: keys=(a, c) -> Hash + Bloom Filter 2 -> Seq Scan on plt1_adv_p2 t1_2 Filter: (b < 10) -> Hash Right Semi Join Hash Cond: ((t2_3.a = t1_3.a) AND (t2_3.c = t1_3.c)) -> Seq Scan on plt2_adv_p3 t2_3 + Bloom Filter 3: keys=(a, c) -> Hash + Bloom Filter 3 -> Seq Scan on plt1_adv_p3 t1_3 Filter: (b < 10) -(21 rows) +(27 rows) SELECT t1.* FROM plt1_adv t1 WHERE EXISTS (SELECT 1 FROM plt2_adv t2 WHERE t1.a = t2.a AND t1.c = t2.c) AND t1.b < 10 ORDER BY t1.a; a | b | c @@ -3805,22 +4031,28 @@ SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 LEFT JOIN plt2_adv t2 ON (t1.a = -> Hash Right Join Hash Cond: ((t2_1.a = t1_1.a) AND (t2_1.c = t1_1.c)) -> Seq Scan on plt2_adv_p1 t2_1 + Bloom Filter 1: keys=(a, c) -> Hash + Bloom Filter 1 -> Seq Scan on plt1_adv_p1 t1_1 Filter: (b < 10) -> Hash Right Join Hash Cond: ((t2_2.a = t1_2.a) AND (t2_2.c = t1_2.c)) -> Seq Scan on plt2_adv_p2 t2_2 + Bloom Filter 2: keys=(a, c) -> Hash + Bloom Filter 2 -> Seq Scan on plt1_adv_p2 t1_2 Filter: (b < 10) -> Hash Right Join Hash Cond: ((t2_3.a = t1_3.a) AND (t2_3.c = t1_3.c)) -> Seq Scan on plt2_adv_p3 t2_3 + Bloom Filter 3: keys=(a, c) -> Hash + Bloom Filter 3 -> Seq Scan on plt1_adv_p3 t1_3 Filter: (b < 10) -(21 rows) +(27 rows) SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 LEFT JOIN plt2_adv t2 ON (t1.a = t2.a AND t1.c = t2.c) WHERE t1.b < 10 ORDER BY t1.a; a | c | a | c @@ -3870,22 +4102,28 @@ SELECT t1.* FROM plt1_adv t1 WHERE NOT EXISTS (SELECT 1 FROM plt2_adv t2 WHERE t -> Hash Right Anti Join Hash Cond: ((t2_1.a = t1_1.a) AND (t2_1.c = t1_1.c)) -> Seq Scan on plt2_adv_p1 t2_1 + Bloom Filter 1: keys=(a, c) -> Hash + Bloom Filter 1 -> Seq Scan on plt1_adv_p1 t1_1 Filter: (b < 10) -> Hash Right Anti Join Hash Cond: ((t2_2.a = t1_2.a) AND (t2_2.c = t1_2.c)) -> Seq Scan on plt2_adv_p2 t2_2 + Bloom Filter 2: keys=(a, c) -> Hash + Bloom Filter 2 -> Seq Scan on plt1_adv_p2 t1_2 Filter: (b < 10) -> Hash Right Anti Join Hash Cond: ((t2_3.a = t1_3.a) AND (t2_3.c = t1_3.c)) -> Seq Scan on plt2_adv_p3 t2_3 + Bloom Filter 3: keys=(a, c) -> Hash + Bloom Filter 3 -> Seq Scan on plt1_adv_p3 t1_3 Filter: (b < 10) -(21 rows) +(27 rows) SELECT t1.* FROM plt1_adv t1 WHERE NOT EXISTS (SELECT 1 FROM plt2_adv t2 WHERE t1.a = t2.a AND t1.c = t2.c) AND t1.b < 10 ORDER BY t1.a; a | b | c @@ -4098,22 +4336,28 @@ SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 INNER JOIN plt2_adv t2 ON (t1.a = -> Hash Join Hash Cond: ((t2_1.a = t1_1.a) AND (t2_1.c = t1_1.c)) -> Seq Scan on plt2_adv_p1 t2_1 + Bloom Filter 1: keys=(a, c) -> Hash + Bloom Filter 1 -> Seq Scan on plt1_adv_p1_null t1_1 Filter: (b < 10) -> Hash Join Hash Cond: ((t2_2.a = t1_2.a) AND (t2_2.c = t1_2.c)) -> Seq Scan on plt2_adv_p2 t2_2 + Bloom Filter 2: keys=(a, c) -> Hash + Bloom Filter 2 -> Seq Scan on plt1_adv_p2 t1_2 Filter: (b < 10) -> Hash Join Hash Cond: ((t2_3.a = t1_3.a) AND (t2_3.c = t1_3.c)) -> Seq Scan on plt2_adv_p3_null t2_3 + Bloom Filter 3: keys=(a, c) -> Hash + Bloom Filter 3 -> Seq Scan on plt1_adv_p3 t1_3 Filter: (b < 10) -(21 rows) +(27 rows) SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 INNER JOIN plt2_adv t2 ON (t1.a = t2.a AND t1.c = t2.c) WHERE t1.b < 10 ORDER BY t1.a; a | c | a | c @@ -4135,22 +4379,28 @@ SELECT t1.* FROM plt1_adv t1 WHERE EXISTS (SELECT 1 FROM plt2_adv t2 WHERE t1.a -> Hash Right Semi Join Hash Cond: ((t2_1.a = t1_1.a) AND (t2_1.c = t1_1.c)) -> Seq Scan on plt2_adv_p1 t2_1 + Bloom Filter 1: keys=(a, c) -> Hash + Bloom Filter 1 -> Seq Scan on plt1_adv_p1_null t1_1 Filter: (b < 10) -> Hash Right Semi Join Hash Cond: ((t2_2.a = t1_2.a) AND (t2_2.c = t1_2.c)) -> Seq Scan on plt2_adv_p2 t2_2 + Bloom Filter 2: keys=(a, c) -> Hash + Bloom Filter 2 -> Seq Scan on plt1_adv_p2 t1_2 Filter: (b < 10) -> Hash Right Semi Join Hash Cond: ((t2_3.a = t1_3.a) AND (t2_3.c = t1_3.c)) -> Seq Scan on plt2_adv_p3_null t2_3 + Bloom Filter 3: keys=(a, c) -> Hash + Bloom Filter 3 -> Seq Scan on plt1_adv_p3 t1_3 Filter: (b < 10) -(21 rows) +(27 rows) SELECT t1.* FROM plt1_adv t1 WHERE EXISTS (SELECT 1 FROM plt2_adv t2 WHERE t1.a = t2.a AND t1.c = t2.c) AND t1.b < 10 ORDER BY t1.a; a | b | c @@ -4172,22 +4422,28 @@ SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 LEFT JOIN plt2_adv t2 ON (t1.a = -> Hash Right Join Hash Cond: ((t2_1.a = t1_1.a) AND (t2_1.c = t1_1.c)) -> Seq Scan on plt2_adv_p1 t2_1 + Bloom Filter 1: keys=(a, c) -> Hash + Bloom Filter 1 -> Seq Scan on plt1_adv_p1_null t1_1 Filter: (b < 10) -> Hash Right Join Hash Cond: ((t2_2.a = t1_2.a) AND (t2_2.c = t1_2.c)) -> Seq Scan on plt2_adv_p2 t2_2 + Bloom Filter 2: keys=(a, c) -> Hash + Bloom Filter 2 -> Seq Scan on plt1_adv_p2 t1_2 Filter: (b < 10) -> Hash Right Join Hash Cond: ((t2_3.a = t1_3.a) AND (t2_3.c = t1_3.c)) -> Seq Scan on plt2_adv_p3_null t2_3 + Bloom Filter 3: keys=(a, c) -> Hash + Bloom Filter 3 -> Seq Scan on plt1_adv_p3 t1_3 Filter: (b < 10) -(21 rows) +(27 rows) SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 LEFT JOIN plt2_adv t2 ON (t1.a = t2.a AND t1.c = t2.c) WHERE t1.b < 10 ORDER BY t1.a; a | c | a | c @@ -4212,22 +4468,28 @@ SELECT t1.* FROM plt1_adv t1 WHERE NOT EXISTS (SELECT 1 FROM plt2_adv t2 WHERE t -> Hash Right Anti Join Hash Cond: ((t2_1.a = t1_1.a) AND (t2_1.c = t1_1.c)) -> Seq Scan on plt2_adv_p1 t2_1 + Bloom Filter 1: keys=(a, c) -> Hash + Bloom Filter 1 -> Seq Scan on plt1_adv_p1_null t1_1 Filter: (b < 10) -> Hash Right Anti Join Hash Cond: ((t2_2.a = t1_2.a) AND (t2_2.c = t1_2.c)) -> Seq Scan on plt2_adv_p2 t2_2 + Bloom Filter 2: keys=(a, c) -> Hash + Bloom Filter 2 -> Seq Scan on plt1_adv_p2 t1_2 Filter: (b < 10) -> Hash Right Anti Join Hash Cond: ((t2_3.a = t1_3.a) AND (t2_3.c = t1_3.c)) -> Seq Scan on plt2_adv_p3_null t2_3 + Bloom Filter 3: keys=(a, c) -> Hash + Bloom Filter 3 -> Seq Scan on plt1_adv_p3 t1_3 Filter: (b < 10) -(21 rows) +(27 rows) SELECT t1.* FROM plt1_adv t1 WHERE NOT EXISTS (SELECT 1 FROM plt2_adv t2 WHERE t1.a = t2.a AND t1.c = t2.c) AND t1.b < 10 ORDER BY t1.a; a | b | c @@ -4303,22 +4565,28 @@ SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 INNER JOIN plt2_adv t2 ON (t1.a = -> Hash Join Hash Cond: ((t2_1.a = t1_1.a) AND (t2_1.c = t1_1.c)) -> Seq Scan on plt2_adv_p1 t2_1 + Bloom Filter 1: keys=(a, c) -> Hash + Bloom Filter 1 -> Seq Scan on plt1_adv_p1 t1_1 Filter: (b < 10) -> Hash Join Hash Cond: ((t2_2.a = t1_2.a) AND (t2_2.c = t1_2.c)) -> Seq Scan on plt2_adv_p2 t2_2 + Bloom Filter 2: keys=(a, c) -> Hash + Bloom Filter 2 -> Seq Scan on plt1_adv_p2 t1_2 Filter: (b < 10) -> Hash Join Hash Cond: ((t2_3.a = t1_3.a) AND (t2_3.c = t1_3.c)) -> Seq Scan on plt2_adv_p3 t2_3 + Bloom Filter 3: keys=(a, c) -> Hash + Bloom Filter 3 -> Seq Scan on plt1_adv_p3 t1_3 Filter: (b < 10) -(21 rows) +(27 rows) SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 INNER JOIN plt2_adv t2 ON (t1.a = t2.a AND t1.c = t2.c) WHERE t1.b < 10 ORDER BY t1.a; a | c | a | c @@ -4394,22 +4662,28 @@ SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 INNER JOIN plt2_adv t2 ON (t1.a = -> Hash Join Hash Cond: ((t2_1.a = t1_1.a) AND (t2_1.c = t1_1.c)) -> Seq Scan on plt2_adv_p1 t2_1 + Bloom Filter 1: keys=(a, c) -> Hash + Bloom Filter 1 -> Seq Scan on plt1_adv_p1 t1_1 Filter: (b < 10) -> Hash Join Hash Cond: ((t2_2.a = t1_2.a) AND (t2_2.c = t1_2.c)) -> Seq Scan on plt2_adv_p2 t2_2 + Bloom Filter 2: keys=(a, c) -> Hash + Bloom Filter 2 -> Seq Scan on plt1_adv_p2 t1_2 Filter: (b < 10) -> Hash Join Hash Cond: ((t2_3.a = t1_3.a) AND (t2_3.c = t1_3.c)) -> Seq Scan on plt2_adv_p3 t2_3 + Bloom Filter 3: keys=(a, c) -> Hash + Bloom Filter 3 -> Seq Scan on plt1_adv_p3 t1_3 Filter: (b < 10) -(21 rows) +(27 rows) SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 INNER JOIN plt2_adv t2 ON (t1.a = t2.a AND t1.c = t2.c) WHERE t1.b < 10 ORDER BY t1.a; a | c | a | c @@ -4431,19 +4705,25 @@ SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 LEFT JOIN plt2_adv t2 ON (t1.a = -> Hash Right Join Hash Cond: ((t2_1.a = t1_1.a) AND (t2_1.c = t1_1.c)) -> Seq Scan on plt2_adv_p1 t2_1 + Bloom Filter 1: keys=(a, c) -> Hash + Bloom Filter 1 -> Seq Scan on plt1_adv_p1 t1_1 Filter: (b < 10) -> Hash Right Join Hash Cond: ((t2_2.a = t1_2.a) AND (t2_2.c = t1_2.c)) -> Seq Scan on plt2_adv_p2 t2_2 + Bloom Filter 2: keys=(a, c) -> Hash + Bloom Filter 2 -> Seq Scan on plt1_adv_p2 t1_2 Filter: (b < 10) -> Hash Right Join Hash Cond: ((t2_3.a = t1_3.a) AND (t2_3.c = t1_3.c)) -> Seq Scan on plt2_adv_p3 t2_3 + Bloom Filter 3: keys=(a, c) -> Hash + Bloom Filter 3 -> Seq Scan on plt1_adv_p3 t1_3 Filter: (b < 10) -> Nested Loop Left Join @@ -4451,7 +4731,7 @@ SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 LEFT JOIN plt2_adv t2 ON (t1.a = -> Seq Scan on plt1_adv_extra t1_4 Filter: (b < 10) -> Seq Scan on plt2_adv_extra t2_4 -(26 rows) +(32 rows) SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 LEFT JOIN plt2_adv t2 ON (t1.a = t2.a AND t1.c = t2.c) WHERE t1.b < 10 ORDER BY t1.a; a | c | a | c @@ -4525,31 +4805,43 @@ SELECT t1.a, t1.c, t2.a, t2.c, t3.a, t3.c FROM plt1_adv t1 LEFT JOIN plt2_adv t2 -> Hash Right Join Hash Cond: ((t3_1.a = t1_1.a) AND (t3_1.c = t1_1.c)) -> Seq Scan on plt1_adv_p1 t3_1 + Bloom Filter 2: keys=(a, c) -> Hash + Bloom Filter 2 -> Hash Right Join Hash Cond: ((t2_1.a = t1_1.a) AND (t2_1.c = t1_1.c)) -> Seq Scan on plt2_adv_p1 t2_1 + Bloom Filter 1: keys=(a, c) -> Hash + Bloom Filter 1 -> Seq Scan on plt1_adv_p1 t1_1 Filter: (b < 10) -> Hash Right Join Hash Cond: ((t3_2.a = t1_2.a) AND (t3_2.c = t1_2.c)) -> Seq Scan on plt1_adv_p2 t3_2 + Bloom Filter 4: keys=(a, c) -> Hash + Bloom Filter 4 -> Hash Right Join Hash Cond: ((t2_2.a = t1_2.a) AND (t2_2.c = t1_2.c)) -> Seq Scan on plt2_adv_p2 t2_2 + Bloom Filter 3: keys=(a, c) -> Hash + Bloom Filter 3 -> Seq Scan on plt1_adv_p2 t1_2 Filter: (b < 10) -> Hash Right Join Hash Cond: ((t3_3.a = t1_3.a) AND (t3_3.c = t1_3.c)) -> Seq Scan on plt1_adv_p3 t3_3 + Bloom Filter 6: keys=(a, c) -> Hash + Bloom Filter 6 -> Hash Right Join Hash Cond: ((t2_3.a = t1_3.a) AND (t2_3.c = t1_3.c)) -> Seq Scan on plt2_adv_p3 t2_3 + Bloom Filter 5: keys=(a, c) -> Hash + Bloom Filter 5 -> Seq Scan on plt1_adv_p3 t1_3 Filter: (b < 10) -> Nested Loop Left Join @@ -4560,7 +4852,7 @@ SELECT t1.a, t1.c, t2.a, t2.c, t3.a, t3.c FROM plt1_adv t1 LEFT JOIN plt2_adv t2 Filter: (b < 10) -> Seq Scan on plt2_adv_extra t2_4 -> Seq Scan on plt1_adv_extra t3_4 -(41 rows) +(53 rows) SELECT t1.a, t1.c, t2.a, t2.c, t3.a, t3.c FROM plt1_adv t1 LEFT JOIN plt2_adv t2 ON (t1.a = t2.a AND t1.c = t2.c) LEFT JOIN plt1_adv t3 ON (t1.a = t3.a AND t1.c = t3.c) WHERE t1.b < 10 ORDER BY t1.a; a | c | a | c | a | c @@ -4596,16 +4888,20 @@ SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 INNER JOIN plt2_adv t2 ON (t1.a = -> Hash Join Hash Cond: ((t2_1.a = t1_2.a) AND (t2_1.c = t1_2.c)) -> Seq Scan on plt2_adv_p1 t2_1 + Bloom Filter 1: keys=(a, c) -> Hash + Bloom Filter 1 -> Seq Scan on plt1_adv_p1 t1_2 Filter: (b < 10) -> Hash Join Hash Cond: ((t2_2.a = t1_1.a) AND (t2_2.c = t1_1.c)) -> Seq Scan on plt2_adv_p2 t2_2 + Bloom Filter 2: keys=(a, c) -> Hash + Bloom Filter 2 -> Seq Scan on plt1_adv_p2 t1_1 Filter: (b < 10) -(15 rows) +(19 rows) SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 INNER JOIN plt2_adv t2 ON (t1.a = t2.a AND t1.c = t2.c) WHERE t1.b < 10 ORDER BY t1.a; a | c | a | c @@ -4687,24 +4983,32 @@ SELECT t1.a, t1.c, t2.a, t2.c, t3.a, t3.c FROM plt1_adv t1 LEFT JOIN plt2_adv t2 -> Hash Right Join Hash Cond: ((t3_1.a = t1_1.a) AND (t3_1.c = t1_1.c)) -> Seq Scan on plt3_adv_p1 t3_1 + Bloom Filter 2: keys=(a, c) -> Hash + Bloom Filter 2 -> Hash Right Join Hash Cond: ((t2_2.a = t1_1.a) AND (t2_2.c = t1_1.c)) -> Seq Scan on plt2_adv_p2 t2_2 + Bloom Filter 1: keys=(a, c) -> Hash + Bloom Filter 1 -> Seq Scan on plt1_adv_p2 t1_1 Filter: (b < 10) -> Hash Right Join Hash Cond: ((t3_2.a = t1_2.a) AND (t3_2.c = t1_2.c)) -> Seq Scan on plt3_adv_p2 t3_2 + Bloom Filter 4: keys=(a, c) -> Hash + Bloom Filter 4 -> Hash Right Join Hash Cond: ((t2_1.a = t1_2.a) AND (t2_1.c = t1_2.c)) -> Seq Scan on plt2_adv_p1 t2_1 + Bloom Filter 3: keys=(a, c) -> Hash + Bloom Filter 3 -> Seq Scan on plt1_adv_p1 t1_2 Filter: (b < 10) -(23 rows) +(31 rows) SELECT t1.a, t1.c, t2.a, t2.c, t3.a, t3.c FROM plt1_adv t1 LEFT JOIN plt2_adv t2 ON (t1.a = t2.a AND t1.c = t2.c) LEFT JOIN plt3_adv t3 ON (t1.a = t3.a AND t1.c = t3.c) WHERE t1.b < 10 ORDER BY t1.a; a | c | a | c | a | c @@ -4733,16 +5037,20 @@ SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 INNER JOIN plt2_adv t2 ON (t1.a = -> Hash Join Hash Cond: ((t2_1.a = t1_2.a) AND (t2_1.c = t1_2.c)) -> Seq Scan on plt2_adv_p1_null t2_1 + Bloom Filter 1: keys=(a, c) -> Hash + Bloom Filter 1 -> Seq Scan on plt1_adv_p1 t1_2 Filter: (b < 10) -> Hash Join Hash Cond: ((t2_2.a = t1_1.a) AND (t2_2.c = t1_1.c)) -> Seq Scan on plt2_adv_p2 t2_2 + Bloom Filter 2: keys=(a, c) -> Hash + Bloom Filter 2 -> Seq Scan on plt1_adv_p2 t1_1 Filter: (b < 10) -(15 rows) +(19 rows) SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 INNER JOIN plt2_adv t2 ON (t1.a = t2.a AND t1.c = t2.c) WHERE t1.b < 10 ORDER BY t1.a; a | c | a | c @@ -4767,10 +5075,12 @@ SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 INNER JOIN plt2_adv t2 ON (t1.a = -> Hash Join Hash Cond: ((t2.a = t1.a) AND (t2.c = t1.c)) -> Seq Scan on plt2_adv_p2 t2 + Bloom Filter 1: keys=(a, c) -> Hash + Bloom Filter 1 -> Seq Scan on plt1_adv_p2 t1 Filter: (b < 10) -(8 rows) +(10 rows) SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 INNER JOIN plt2_adv t2 ON (t1.a = t2.a AND t1.c = t2.c) WHERE t1.b < 10 ORDER BY t1.a; a | c | a | c @@ -4809,16 +5119,20 @@ SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 INNER JOIN plt2_adv t2 ON (t1.a = -> Hash Join Hash Cond: ((t2_1.a = t1_1.a) AND (t2_1.c = t1_1.c)) -> Seq Scan on plt2_adv_p3 t2_1 + Bloom Filter 1: keys=(a, c) -> Hash + Bloom Filter 1 -> Seq Scan on plt1_adv_p3 t1_1 Filter: ((b < 10) AND (c = ANY ('{0003,0004,0005}'::text[]))) -> Hash Join Hash Cond: ((t2_2.a = t1_2.a) AND (t2_2.c = t1_2.c)) -> Seq Scan on plt2_adv_p4 t2_2 + Bloom Filter 2: keys=(a, c) -> Hash + Bloom Filter 2 -> Seq Scan on plt1_adv_p4 t1_2 Filter: ((b < 10) AND (c = ANY ('{0003,0004,0005}'::text[]))) -(15 rows) +(19 rows) SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 INNER JOIN plt2_adv t2 ON (t1.a = t2.a AND t1.c = t2.c) WHERE t1.c IN ('0003', '0004', '0005') AND t1.b < 10 ORDER BY t1.a; a | c | a | c @@ -4837,10 +5151,12 @@ SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 LEFT JOIN plt2_adv t2 ON (t1.a = -> Hash Right Join Hash Cond: ((t2.a = t1.a) AND (t2.c = t1.c)) -> Seq Scan on plt2_adv_p4 t2 + Bloom Filter 1: keys=(a, c) -> Hash + Bloom Filter 1 -> Seq Scan on plt1_adv_p4 t1 Filter: ((c IS NULL) AND (b < 10)) -(8 rows) +(10 rows) SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 LEFT JOIN plt2_adv t2 ON (t1.a = t2.a AND t1.c = t2.c) WHERE t1.c IS NULL AND t1.b < 10 ORDER BY t1.a; a | c | a | c @@ -4862,16 +5178,20 @@ SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 INNER JOIN plt2_adv t2 ON (t1.a = -> Hash Join Hash Cond: ((t2_1.a = t1_1.a) AND (t2_1.c = t1_1.c)) -> Seq Scan on plt2_adv_p3 t2_1 + Bloom Filter 1: keys=(a, c) -> Hash + Bloom Filter 1 -> Seq Scan on plt1_adv_p3 t1_1 Filter: ((b < 10) AND (c = ANY ('{0003,0004,0005}'::text[]))) -> Hash Join Hash Cond: ((t2_2.a = t1_2.a) AND (t2_2.c = t1_2.c)) -> Seq Scan on plt2_adv_p4 t2_2 + Bloom Filter 2: keys=(a, c) -> Hash + Bloom Filter 2 -> Seq Scan on plt1_adv_p4 t1_2 Filter: ((b < 10) AND (c = ANY ('{0003,0004,0005}'::text[]))) -(15 rows) +(19 rows) SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 INNER JOIN plt2_adv t2 ON (t1.a = t2.a AND t1.c = t2.c) WHERE t1.c IN ('0003', '0004', '0005') AND t1.b < 10 ORDER BY t1.a; a | c | a | c @@ -4890,10 +5210,12 @@ SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 LEFT JOIN plt2_adv t2 ON (t1.a = -> Hash Right Join Hash Cond: ((t2.a = t1.a) AND (t2.c = t1.c)) -> Seq Scan on plt2_adv_p4 t2 + Bloom Filter 1: keys=(a, c) -> Hash + Bloom Filter 1 -> Seq Scan on plt1_adv_p4 t1 Filter: ((c IS NULL) AND (b < 10)) -(8 rows) +(10 rows) SELECT t1.a, t1.c, t2.a, t2.c FROM plt1_adv t1 LEFT JOIN plt2_adv t2 ON (t1.a = t2.a AND t1.c = t2.c) WHERE t1.c IS NULL AND t1.b < 10 ORDER BY t1.a; a | c | a | c @@ -5015,12 +5337,16 @@ SELECT t1.*, t2.* FROM alpha t1 INNER JOIN beta t2 ON (t1.a = t2.a AND t1.b = t2 Hash Cond: ((t1_1.a = t2_1.a) AND (t1_1.b = t2_1.b)) -> Seq Scan on alpha_neg_p1 t1_1 Filter: ((b >= 125) AND (b < 225)) + Bloom Filter 1: keys=(a, b) -> Hash + Bloom Filter 1 -> Seq Scan on beta_neg_p1 t2_1 -> Hash Join Hash Cond: ((t2_2.a = t1_2.a) AND (t2_2.b = t1_2.b)) -> Seq Scan on beta_neg_p2 t2_2 + Bloom Filter 2: keys=(a, b) -> Hash + Bloom Filter 2 -> Seq Scan on alpha_neg_p2 t1_2 Filter: ((b >= 125) AND (b < 225)) -> Hash Join @@ -5037,7 +5363,7 @@ SELECT t1.*, t2.* FROM alpha t1 INNER JOIN beta t2 ON (t1.a = t2.a AND t1.b = t2 Filter: ((b >= 125) AND (b < 225)) -> Seq Scan on alpha_pos_p3 t1_6 Filter: ((b >= 125) AND (b < 225)) -(29 rows) +(33 rows) SELECT t1.*, t2.* FROM alpha t1 INNER JOIN beta t2 ON (t1.a = t2.a AND t1.b = t2.b) WHERE t1.b >= 125 AND t1.b < 225 ORDER BY t1.a, t1.b; a | b | c | a | b | c @@ -5150,14 +5476,18 @@ SELECT t1.*, t2.* FROM alpha t1 INNER JOIN beta t2 ON (t1.a = t2.a AND t1.b = t2 Hash Cond: ((t1_1.a = t2_1.a) AND (t1_1.b = t2_1.b) AND (t1_1.c = t2_1.c)) -> Seq Scan on alpha_neg_p1 t1_1 Filter: ((c = ANY ('{0004,0009}'::text[])) AND (((b >= 100) AND (b < 110)) OR ((b >= 200) AND (b < 210)))) + Bloom Filter 1: keys=(a, b, c) -> Hash + Bloom Filter 1 -> Seq Scan on beta_neg_p1 t2_1 Filter: (((b >= 100) AND (b < 110)) OR ((b >= 200) AND (b < 210))) -> Hash Join Hash Cond: ((t1_2.a = t2_2.a) AND (t1_2.b = t2_2.b) AND (t1_2.c = t2_2.c)) -> Seq Scan on alpha_neg_p2 t1_2 Filter: ((c = ANY ('{0004,0009}'::text[])) AND (((b >= 100) AND (b < 110)) OR ((b >= 200) AND (b < 210)))) + Bloom Filter 2: keys=(a, b, c) -> Hash + Bloom Filter 2 -> Seq Scan on beta_neg_p2 t2_2 Filter: (((b >= 100) AND (b < 110)) OR ((b >= 200) AND (b < 210))) -> Nested Loop @@ -5172,7 +5502,7 @@ SELECT t1.*, t2.* FROM alpha t1 INNER JOIN beta t2 ON (t1.a = t2.a AND t1.b = t2 Filter: ((c = ANY ('{0004,0009}'::text[])) AND (((b >= 100) AND (b < 110)) OR ((b >= 200) AND (b < 210)))) -> Seq Scan on beta_pos_p3 t2_4 Filter: (((b >= 100) AND (b < 110)) OR ((b >= 200) AND (b < 210))) -(29 rows) +(33 rows) SELECT t1.*, t2.* FROM alpha t1 INNER JOIN beta t2 ON (t1.a = t2.a AND t1.b = t2.b AND t1.c = t2.c) WHERE ((t1.b >= 100 AND t1.b < 110) OR (t1.b >= 200 AND t1.b < 210)) AND ((t2.b >= 100 AND t2.b < 110) OR (t2.b >= 200 AND t2.b < 210)) AND t1.c IN ('0004', '0009') ORDER BY t1.a, t1.b; a | b | c | a | b | c @@ -5316,19 +5646,25 @@ EXPLAIN (COSTS OFF) SELECT * FROM pht1 p1 JOIN pht1 p2 USING (c) LIMIT 1000; -> Hash Join Hash Cond: (p1_1.c = p2_1.c) -> Seq Scan on pht1_p1 p1_1 + Bloom Filter 1: keys=(c) -> Hash + Bloom Filter 1 -> Seq Scan on pht1_p1 p2_1 -> Hash Join Hash Cond: (p1_2.c = p2_2.c) -> Seq Scan on pht1_p2 p1_2 + Bloom Filter 2: keys=(c) -> Hash + Bloom Filter 2 -> Seq Scan on pht1_p2 p2_2 -> Hash Join Hash Cond: (p1_3.c = p2_3.c) -> Seq Scan on pht1_p3 p1_3 + Bloom Filter 3: keys=(c) -> Hash + Bloom Filter 3 -> Seq Scan on pht1_p3 p2_3 -(17 rows) +(23 rows) RESET enable_mergejoin; SET max_parallel_workers_per_gather = 1; diff --git a/src/test/regress/expected/predicate.out b/src/test/regress/expected/predicate.out index feae77cb840..079f6422fdc 100644 --- a/src/test/regress/expected/predicate.out +++ b/src/test/regress/expected/predicate.out @@ -748,14 +748,16 @@ SELECT id FROM dist_tab WHERE row_nn IS DISTINCT FROM ROW(1, 5)::dist_row_t; SET enable_nestloop TO off; EXPLAIN (COSTS OFF) SELECT * FROM dist_tab t1 JOIN dist_tab t2 ON t1.val_nn IS NOT DISTINCT FROM t2.val_nn; - QUERY PLAN --------------------------------------- + QUERY PLAN +--------------------------------------- Hash Join Hash Cond: (t1.val_nn = t2.val_nn) -> Seq Scan on dist_tab t1 + Bloom Filter 1: keys=(val_nn) -> Hash + Bloom Filter 1 -> Seq Scan on dist_tab t2 -(5 rows) +(7 rows) SELECT * FROM dist_tab t1 JOIN dist_tab t2 ON t1.val_nn IS NOT DISTINCT FROM t2.val_nn; id | val_nn | val_null | row_nn | id | val_nn | val_null | row_nn diff --git a/src/test/regress/expected/privileges.out b/src/test/regress/expected/privileges.out index 0de13612818..ca8ce2868d4 100644 --- a/src/test/regress/expected/privileges.out +++ b/src/test/regress/expected/privileges.out @@ -618,10 +618,12 @@ EXPLAIN (COSTS OFF) SELECT * FROM atest12 x, atest12 y Hash Join Hash Cond: (x.a = y.b) -> Seq Scan on atest12 x + Bloom Filter 1: keys=(a) -> Hash + Bloom Filter 1 -> Seq Scan on atest12 y Filter: (abs(a) <<< 5) -(6 rows) +(8 rows) -- clean up (regress_priv_user1's objects are all dropped later) DROP FUNCTION leak2(integer, integer) CASCADE; diff --git a/src/test/regress/expected/returning.out b/src/test/regress/expected/returning.out index 196829e94fa..4625a9447c7 100644 --- a/src/test/regress/expected/returning.out +++ b/src/test/regress/expected/returning.out @@ -694,8 +694,10 @@ UPDATE joinview SET f3 = f3 + 1 WHERE f3 = 57 Hash Cond: (joinme_1.f2j = foo_1.f2) -> Seq Scan on pg_temp.joinme joinme_1 Output: joinme_1.ctid, joinme_1.f2j + Bloom Filter 1: keys=(joinme_1.f2j) -> Hash Output: foo_1.f2, foo_1.tableoid, foo_1.ctid + Bloom Filter 1 -> Seq Scan on pg_temp.foo foo_1 Output: foo_1.f2, foo_1.tableoid, foo_1.ctid -> Hash @@ -705,12 +707,14 @@ UPDATE joinview SET f3 = f3 + 1 WHERE f3 = 57 Hash Cond: (joinme.f2j = foo_2.f2) -> Seq Scan on pg_temp.joinme Output: joinme.ctid, joinme.other, joinme.f2j + Bloom Filter 2: keys=(joinme.f2j) -> Hash Output: foo_2.f1, foo_2.f3, foo_2.ctid, foo_2.f2, foo_2.tableoid + Bloom Filter 2 -> Seq Scan on pg_temp.foo foo_2 Output: foo_2.f1, foo_2.f3, foo_2.ctid, foo_2.f2, foo_2.tableoid Filter: (foo_2.f3 = 57) -(27 rows) +(31 rows) UPDATE joinview SET f3 = f3 + 1 WHERE f3 = 57 RETURNING old.*, new.*, *, new.f3 - old.f3 AS delta_f3; @@ -768,12 +772,14 @@ UPDATE joinview SET f3 = f3 + 1, f4 = 7 WHERE f3 = 58 Hash Cond: (joinme.f2j = foo.f2) -> Seq Scan on pg_temp.joinme Output: joinme.other, joinme.ctid, joinme.f2j + Bloom Filter 1: keys=(joinme.f2j) -> Hash Output: foo.f3, foo.f1, foo.f2, foo.f4, foo.ctid, foo.tableoid + Bloom Filter 1 -> Seq Scan on pg_temp.foo Output: foo.f3, foo.f1, foo.f2, foo.f4, foo.ctid, foo.tableoid Filter: (foo.f3 = 58) -(12 rows) +(14 rows) UPDATE joinview SET f3 = f3 + 1, f4 = 7 WHERE f3 = 58 RETURNING old.*, new.*, *, new.f3 - old.f3 AS delta_f3; -- should succeed diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out index 3a5e82c35bd..4d69d1cd84b 100644 --- a/src/test/regress/expected/rowsecurity.out +++ b/src/test/regress/expected/rowsecurity.out @@ -1,6 +1,8 @@ -- -- Test of Row-level security feature -- +-- disable bloom filter pushdown, to not interfere with calls to functions +SET enable_hashjoin_bloom = off; -- Clean up in case a prior regression run failed -- Suppress NOTICE messages when users/groups don't exist SET client_min_messages TO 'warning'; diff --git a/src/test/regress/expected/select_views.out b/src/test/regress/expected/select_views.out index 1aeed8452bd..fe7be9891d5 100644 --- a/src/test/regress/expected/select_views.out +++ b/src/test/regress/expected/select_views.out @@ -2,6 +2,8 @@ -- SELECT_VIEWS -- test the views defined in CREATE_VIEWS -- +-- disable bloom filter pushdown, to not interfere with calls to functions +SET enable_hashjoin_bloom = off; SELECT * FROM street; name | thepath | cname ------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------- diff --git a/src/test/regress/expected/stats_ext.out b/src/test/regress/expected/stats_ext.out index 37070c1a896..11cf56fdba4 100644 --- a/src/test/regress/expected/stats_ext.out +++ b/src/test/regress/expected/stats_ext.out @@ -3633,9 +3633,11 @@ SELECT * FROM sb_1 a, sb_2 b WHERE a.x = b.x AND a.y = b.y AND a.z = b.z; Hash Join Hash Cond: ((a.x = b.x) AND (a.y = b.y) AND (a.z = b.z)) -> Seq Scan on sb_1 a + Bloom Filter 1: keys=(x, y, z) -> Hash + Bloom Filter 1 -> Seq Scan on sb_2 b -(5 rows) +(7 rows) -- Check that the Hash Join bucket size estimator detects equal clauses correctly. SET enable_nestloop = 'off'; diff --git a/src/test/regress/expected/subselect.out b/src/test/regress/expected/subselect.out index a3778c23c34..5c7d49050db 100644 --- a/src/test/regress/expected/subselect.out +++ b/src/test/regress/expected/subselect.out @@ -406,9 +406,11 @@ select * from int4_tbl o where exists Hash Semi Join Hash Cond: (o.f1 = i.f1) -> Seq Scan on int4_tbl o + Bloom Filter 1: keys=(f1) -> Hash + Bloom Filter 1 -> Seq Scan on int4_tbl i -(5 rows) +(7 rows) explain (costs off) select * from int4_tbl o where not exists @@ -783,14 +785,16 @@ order by t1.a, t2.a; Hash Cond: (t2.a = (t3.b + 1)) -> Seq Scan on public.semijoin_unique_tbl t2 Output: t2.a, t2.b + Bloom Filter 1: keys=(t2.a) -> Hash Output: t3.a, t3.b + Bloom Filter 1 -> HashAggregate Output: t3.a, t3.b Group Key: (t3.a + 1), (t3.b + 1) -> Seq Scan on public.semijoin_unique_tbl t3 Output: t3.a, t3.b, (t3.a + 1), (t3.b + 1) -(24 rows) +(26 rows) -- encourage use of parallel plans set parallel_setup_cost=0; @@ -2176,11 +2180,13 @@ order by t1.ten; Hash Cond: (t2.fivethous = t1.unique1) -> Seq Scan on public.tenk1 t2 Output: t2.unique1, t2.unique2, t2.two, t2.four, t2.ten, t2.twenty, t2.hundred, t2.thousand, t2.twothousand, t2.fivethous, t2.tenthous, t2.odd, t2.even, t2.stringu1, t2.stringu2, t2.string4 + Bloom Filter 1: keys=(t2.fivethous) -> Hash Output: t1.ten, t1.unique1 + Bloom Filter 1 -> Seq Scan on public.tenk1 t1 Output: t1.ten, t1.unique1 -(15 rows) +(17 rows) select t1.ten, sum(x) from tenk1 t1 left join lateral ( @@ -2275,15 +2281,19 @@ order by 1, 2; Hash Cond: (t2.q2 = t3.q2) -> Seq Scan on public.int8_tbl t2 Output: t2.q1, t2.q2 + Bloom Filter 1: keys=(t2.q2) + Bloom Filter 2: keys=(t2.q2) -> Hash Output: t3.q2 + Bloom Filter 1 -> Seq Scan on public.int8_tbl t3 Output: t3.q2 -> Hash Output: t1.q1, t1.q2 + Bloom Filter 2 -> Seq Scan on public.int8_tbl t1 Output: t1.q1, t1.q2 -(19 rows) +(23 rows) select t1.q1, x from int8_tbl t1 left join @@ -2326,14 +2336,16 @@ order by 1, 2; Output: t2.q2, ((t2.q1 + 1)) -> Seq Scan on public.int8_tbl t2 Output: t2.q1, t2.q2 + Bloom Filter 1: keys=(t2.q2) -> Seq Scan on public.int8_tbl t3 Output: t3.q2, (t2.q1 + 1) Filter: (t2.q2 = t3.q2) -> Hash Output: t1.q1, t1.q2 + Bloom Filter 1 -> Seq Scan on public.int8_tbl t1 Output: t1.q1, t1.q2 -(17 rows) +(19 rows) select t1.q1, x from int8_tbl t1 left join @@ -2378,15 +2390,19 @@ order by 1, 2; Hash Cond: (t2.q2 = t3.q1) -> Seq Scan on public.int8_tbl t2 Output: t2.q1, t2.q2 + Bloom Filter 1: keys=(t2.q2) + Bloom Filter 2: keys=(t2.q1) -> Hash Output: t3.q1 + Bloom Filter 1 -> Seq Scan on public.int8_tbl t3 Output: t3.q1 -> Hash Output: t1.q1 + Bloom Filter 2 -> Seq Scan on public.int8_tbl t1 Output: t1.q1 -(19 rows) +(23 rows) select t1.q1, x from int8_tbl t1 left join @@ -2439,14 +2455,16 @@ order by 1, 2; Output: t2.q1, (t2.q2) -> Seq Scan on public.int8_tbl t2 Output: t2.q1, t2.q2 + Bloom Filter 1: keys=(t2.q1) -> Seq Scan on public.int8_tbl t3 Output: t3.q1, t2.q2 Filter: (t2.q2 = t3.q1) -> Hash Output: t1.q1 + Bloom Filter 1 -> Seq Scan on public.int8_tbl t1 Output: t1.q1 -(17 rows) +(19 rows) select t1.q1, x from int8_tbl t1 left join @@ -2510,19 +2528,21 @@ order by 1, 2, 3; Hash Cond: (t2.q1 = t3.q2) -> Seq Scan on public.int8_tbl t2 Output: t2.q1, t2.q2 + Bloom Filter 1: keys=(t2.q2) -> Hash Output: t3.q2, (COALESCE(t3.q1, t3.q1)) -> Seq Scan on public.int8_tbl t3 Output: t3.q2, COALESCE(t3.q1, t3.q1) -> Hash Output: t4.q1, t4.q2 + Bloom Filter 1 -> Seq Scan on public.int8_tbl t4 Output: t4.q1, t4.q2 -> Hash Output: t1.q2 -> Seq Scan on public.int8_tbl t1 Output: t1.q2 -(26 rows) +(28 rows) select ss2.* from int8_tbl t1 left join @@ -2590,11 +2610,13 @@ order by 1, 2, 3; Output: t3.q2, COALESCE(t3.q1, t3.q1) -> Seq Scan on public.int8_tbl t4 Output: t4.q1, t4.q2, (COALESCE(t3.q1, t3.q1)) + Bloom Filter 1: keys=(t4.q1) -> Hash Output: t1.q2 + Bloom Filter 1 -> Seq Scan on public.int8_tbl t1 Output: t1.q2 -(24 rows) +(26 rows) select ss2.* from int8_tbl t1 left join @@ -2895,11 +2917,13 @@ select * from tenk1 A where hundred in (select hundred from tenk2 B where B.odd Hash Join Hash Cond: ((a.odd = b.odd) AND (a.hundred = b.hundred)) -> Seq Scan on tenk1 a + Bloom Filter 1: keys=(odd, hundred) -> Hash + Bloom Filter 1 -> HashAggregate Group Key: b.odd, b.hundred -> Seq Scan on tenk2 b -(7 rows) +(9 rows) explain (costs off) select * from tenk1 A where exists @@ -2964,11 +2988,13 @@ ON B.hundred in (SELECT c.hundred FROM tenk2 C WHERE c.odd = b.odd); -> Hash Join Hash Cond: ((b.odd = c.odd) AND (b.hundred = c.hundred)) -> Seq Scan on tenk2 b + Bloom Filter 1: keys=(odd, hundred) -> Hash + Bloom Filter 1 -> HashAggregate Group Key: c.odd, c.hundred -> Seq Scan on tenk2 c -(10 rows) +(12 rows) -- we can pull up the sublink into the inner JoinExpr. explain (costs off) @@ -2983,13 +3009,15 @@ WHERE a.thousand < 750; Hash Cond: (a.hundred = c.hundred) -> Seq Scan on tenk1 a Filter: (thousand < 750) + Bloom Filter 1: keys=(hundred) -> Hash + Bloom Filter 1 -> HashAggregate Group Key: c.odd, c.hundred -> Seq Scan on tenk2 c -> Hash -> Seq Scan on tenk2 b -(12 rows) +(14 rows) -- we can pull up the aggregate sublink into RHS of a left join. explain (costs off) @@ -3126,9 +3154,11 @@ WHERE a.ten IN (VALUES (1), (2)); Hash Cond: (a.ten = c.ten) -> Seq Scan on onek a Filter: (ten = ANY ('{1,2}'::integer[])) + Bloom Filter 1: keys=(ten) -> Hash + Bloom Filter 1 -> Seq Scan on tenk1 c -(6 rows) +(8 rows) EXPLAIN (COSTS OFF) SELECT c.unique1,c.ten FROM tenk1 c JOIN onek a USING (ten) @@ -3139,9 +3169,11 @@ WHERE c.ten IN (VALUES (1), (2)); Hash Cond: (c.ten = a.ten) -> Seq Scan on tenk1 c Filter: (ten = ANY ('{1,2}'::integer[])) + Bloom Filter 1: keys=(ten) -> Hash + Bloom Filter 1 -> Seq Scan on onek a -(6 rows) +(8 rows) -- Constant expressions are simplified EXPLAIN (COSTS OFF) @@ -3311,9 +3343,11 @@ SELECT * FROM onek t1, lateral (SELECT * FROM onek t2 WHERE t2.ten IN (values (t -> Hash Semi Join Hash Cond: (t2.ten = "*VALUES*".column1) -> Seq Scan on onek t2 + Bloom Filter 1: keys=(ten) -> Hash + Bloom Filter 1 -> Values Scan on "*VALUES*" -(7 rows) +(9 rows) -- VtA causes the whole expression to be evaluated as a constant EXPLAIN (COSTS OFF) diff --git a/src/test/regress/expected/sysviews.out b/src/test/regress/expected/sysviews.out index 132b56a5864..a796e431415 100644 --- a/src/test/regress/expected/sysviews.out +++ b/src/test/regress/expected/sysviews.out @@ -163,6 +163,7 @@ select name, setting from pg_settings where name like 'enable%'; enable_group_by_reordering | on enable_hashagg | on enable_hashjoin | on + enable_hashjoin_bloom | on enable_incremental_sort | on enable_indexonlyscan | on enable_indexscan | on @@ -180,7 +181,7 @@ select name, setting from pg_settings where name like 'enable%'; enable_seqscan | on enable_sort | on enable_tidscan | on -(25 rows) +(26 rows) -- There are always wait event descriptions for various types. InjectionPoint -- may be present or absent, depending on history since last postmaster start. diff --git a/src/test/regress/expected/updatable_views.out b/src/test/regress/expected/updatable_views.out index 7b00c742776..7d4af80faf6 100644 --- a/src/test/regress/expected/updatable_views.out +++ b/src/test/regress/expected/updatable_views.out @@ -587,11 +587,13 @@ MERGE INTO rw_view1 t Hash Cond: (base_tbl.a = generate_series.generate_series) -> Bitmap Heap Scan on base_tbl Recheck Cond: (a > 0) + Bloom Filter 1: keys=(a) -> Bitmap Index Scan on base_tbl_pkey Index Cond: (a > 0) -> Hash + Bloom Filter 1 -> Function Scan on generate_series -(9 rows) +(11 rows) EXPLAIN (costs off) MERGE INTO rw_view1 t @@ -621,11 +623,13 @@ MERGE INTO rw_view1 t Hash Cond: (base_tbl.a = generate_series.generate_series) -> Bitmap Heap Scan on base_tbl Recheck Cond: (a > 0) + Bloom Filter 1: keys=(a) -> Bitmap Index Scan on base_tbl_pkey Index Cond: (a > 0) -> Hash + Bloom Filter 1 -> Function Scan on generate_series -(9 rows) +(11 rows) -- it's still updatable if we add a DO ALSO rule CREATE TABLE base_tbl_hist(ts timestamptz default now(), a int, b text); @@ -2777,12 +2781,14 @@ EXPLAIN (costs off) UPDATE rw_view1 SET a = a + 5; -> Hash Join Hash Cond: (b.a = r.a) -> Seq Scan on base_tbl b + Bloom Filter 1: keys=(a) -> Hash + Bloom Filter 1 -> Seq Scan on ref_tbl r SubPlan exists_1 -> Index Only Scan using ref_tbl_pkey on ref_tbl r_1 Index Cond: (a = b.a) -(9 rows) +(11 rows) DROP TABLE base_tbl, ref_tbl CASCADE; NOTICE: drop cascades to view rw_view1 diff --git a/src/test/regress/expected/window.out b/src/test/regress/expected/window.out index de0e14a686e..8a439b28df3 100644 --- a/src/test/regress/expected/window.out +++ b/src/test/regress/expected/window.out @@ -5474,10 +5474,12 @@ LIMIT 1; -> Hash Join Hash Cond: (t1.unique1 = t2.tenthous) -> Index Only Scan using tenk1_unique1 on tenk1 t1 + Bloom Filter 1: keys=(unique1) -> Hash + Bloom Filter 1 -> Seq Scan on tenk1 t2 Filter: (two = 1) -(9 rows) +(11 rows) -- Ensure we get a cheap total plan. This time use UNBOUNDED FOLLOWING, which -- needs to read all join rows to output the first WindowAgg row. diff --git a/src/test/regress/expected/with.out b/src/test/regress/expected/with.out index 77ded01b046..db8c77721e7 100644 --- a/src/test/regress/expected/with.out +++ b/src/test/regress/expected/with.out @@ -686,9 +686,11 @@ select count(*) from tenk1 a -> Hash Semi Join Hash Cond: (a.unique1 = x.unique1) -> Index Only Scan using tenk1_unique1 on tenk1 a + Bloom Filter 1: keys=(unique1) -> Hash + Bloom Filter 1 -> CTE Scan on x -(8 rows) +(10 rows) explain (costs off) with x as materialized (insert into tenk1 default values returning unique1) @@ -3246,8 +3248,10 @@ WHEN NOT MATCHED THEN INSERT VALUES(o.k, o.v); Hash Cond: (m.k = o.k) -> Seq Scan on public.m Output: m.ctid, m.k + Bloom Filter 1: keys=(m.k) -> Hash Output: o.k, o.v, o.* + Bloom Filter 1 -> Subquery Scan on o Output: o.k, o.v, o.* -> Result @@ -3258,7 +3262,7 @@ WHEN NOT MATCHED THEN INSERT VALUES(o.k, o.v); -> CTE Scan on cte_basic Output: (cte_basic.b || ' merge update'::text) Filter: (cte_basic.a = m.k) -(21 rows) +(23 rows) -- InitPlan WITH cte_init AS MATERIALIZED (SELECT 1 a, 'cte_init val' b) @@ -3295,13 +3299,15 @@ WHEN NOT MATCHED THEN INSERT VALUES(o.k, o.v); Hash Cond: (m.k = o.k) -> Seq Scan on public.m Output: m.ctid, m.k + Bloom Filter 1: keys=(m.k) -> Hash Output: o.k, o.v, o.* + Bloom Filter 1 -> Subquery Scan on o Output: o.k, o.v, o.* -> Result Output: 1, 'merge source InitPlan'::text -(21 rows) +(23 rows) -- MERGE source comes from CTE: WITH merge_source_cte AS MATERIALIZED (SELECT 15 a, 'merge_source_cte val' b) @@ -3339,11 +3345,13 @@ WHEN NOT MATCHED THEN INSERT VALUES(o.a, o.b || (SELECT merge_source_cte.*::text Hash Cond: (m.k = merge_source_cte.a) -> Seq Scan on public.m Output: m.ctid, m.k + Bloom Filter 1: keys=(m.k) -> Hash Output: merge_source_cte.a, merge_source_cte.b, merge_source_cte.* + Bloom Filter 1 -> CTE Scan on merge_source_cte Output: merge_source_cte.a, merge_source_cte.b, merge_source_cte.* -(20 rows) +(22 rows) DROP TABLE m; -- check that run to completion happens in proper ordering diff --git a/src/test/regress/sql/rowsecurity.sql b/src/test/regress/sql/rowsecurity.sql index 6b3566271df..390e79b04c3 100644 --- a/src/test/regress/sql/rowsecurity.sql +++ b/src/test/regress/sql/rowsecurity.sql @@ -2,6 +2,9 @@ -- Test of Row-level security feature -- +-- disable bloom filter pushdown, to not interfere with calls to functions +SET enable_hashjoin_bloom = off; + -- Clean up in case a prior regression run failed -- Suppress NOTICE messages when users/groups don't exist diff --git a/src/test/regress/sql/select_views.sql b/src/test/regress/sql/select_views.sql index e742f136990..09f96b0b1ae 100644 --- a/src/test/regress/sql/select_views.sql +++ b/src/test/regress/sql/select_views.sql @@ -3,6 +3,9 @@ -- test the views defined in CREATE_VIEWS -- +-- disable bloom filter pushdown, to not interfere with calls to functions +SET enable_hashjoin_bloom = off; + SELECT * FROM street; SELECT name, #thepath FROM iexit ORDER BY name COLLATE "C", 2; -- 2.54.0