BUG #6658: Database do not push down static condition to subquery.

The following bug has been logged on the website:

Bug reference: 6658
Logged by: Maxim Boguk
Email address: maxim(dot)boguk(at)gmail(dot)com
PostgreSQL version: 9.1.3
Operating system: Linux
Description:

Hi,

I have 2 queries which have the same plan/same performance on 8.3 but on
9.1.3 fist one doesn't perform well because condition doesn't pushed down to
subquery:

--slow on 9.1.3 but fast on 8.3.13
--helper just shell view
db=# explain (analyze,verbose) select id from
(SELECT
id,
regexp_replace(btrim(((((((COALESCE(NULLIF(start_number, 0) || ' '::text,
''::text) || address) || ' '::text) || suburb) || ' '::text) || state) || '
'::text) || postcode), ' +'::text, ' '::text, 'g'::text) AS search_string
FROM helper
WHERE
start_number IS NOT NULL AND
start_alpha IS NOT NULL AND
end_number IS NULL
) AS _t WHERE
search_string='something';

QUERY PLAN





------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Subquery Scan on _t (cost=7715.15..2141451.98 rows=55630 width=4) (actual
time=4132.959..4132.959 rows=0 loops=1)
Output: _t.id
Filter: (_t.search_string = 'something'::text)
-> Bitmap Heap Scan on data_table (cost=7715.15..1557337.03
rows=11125999 width=45) (actual time=92.290..4066.518 rows=403344 loops=1)
Output: id,
regexp_replace(btrim((((((((COALESCE(((NULLIF(start_number, 0))::text || '
'::text), ''::text) || upper(regexp_replace(btrim(((COALESCE((name || '
'::text), ''::text) || COALESCE((extension || ' '::text), ''::text)) ||
COALESCE(direction, ''::text))), ' +'::text, ' '::text, 'g'::text))) || '
'::text) || upper(regexp_replace(btrim(COALESCE(locality_name, ''::text)), '
+'::text, ' '::text, 'g'::text))) || ' '::text) ||
upper(regexp_replace(btrim(COALESCE(state_code, ''::text)), ' +'::text, '
'::text, 'g'::text))) || ' '::text) ||
upper(regexp_replace(btrim(COALESCE(post_code, ''::text)), ' +'::text, '
'::text, 'g'::text)))), ' +'::text, ' '::text, 'g'::text)
Recheck Cond: ((start_number IS NOT NULL) AND
(upper((start_alpha)::text) IS NOT NULL) AND (end_number IS NULL))
-> Bitmap Index Scan on ix_search_full5 (cost=0.00..4933.65
rows=11125999 width=0) (actual time=59.118..59.118 rows=403344 loops=1)
Total runtime: 4133.001 ms
(8 rows)

--fast on both 9.1.3 and 8.3.13
--flat version (search_string condition manually inlined)
db=# explain (analyze, verbose)
SELECT
id
FROM helper
WHERE
start_number IS NOT NULL AND
start_alpha IS NOT NULL AND
end_number IS NULL AND
regexp_replace(btrim(((((((COALESCE(NULLIF(start_number, 0) || ' '::text,
''::text) || address) || ' '::text) || suburb) || ' '::text) || state) || '
'::text) || postcode), ' +'::text, ' '::text, 'g'::text) = 'something';




QUERY PLAN





-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Index Scan using ix_search_full5 on data_table (cost=0.07..55842.16
rows=55630 width=4) (actual time=0.006..0.006 rows=0 loops=1)
Output: id
Index Cond: (regexp_replace(btrim((((((((COALESCE(((NULLIF(start_number,
0))::text || ' '::text), ''::text) ||
upper(regexp_replace(btrim(((COALESCE((name || ' '::text), ''::text) ||
COALESCE((extension || ' '::text), ''::text)) || COALESCE(direction,
''::text))), ' +'::text, ' '::text, 'g'::text))) || ' '::text) ||
upper(regexp_replace(btrim(COALESCE(locality_name, ''::text)), ' +'::text,
' '::text, 'g'::text))) || ' '::text) ||
upper(regexp_replace(btrim(COALESCE(state_code, ''::text)), ' +'::text, '
'::text, 'g'::text))) || ' '::text) ||
upper(regexp_replace(btrim(COALESCE(post_code, ''::text)), ' +'::text, '
'::text, 'g'::text)))), ' +'::text, ' '::text, 'g'::text) =
'something'::text)
Total runtime: 0.027 ms
(4 rows)

db=# \d+ ix_search_full5




Index "ix_search_full_address5"
Column | Type |



Definition




| Storage | Description
----------------+------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+-------------
regexp_replace | text |
regexp_replace(btrim(((((((COALESCE(NULLIF(start_number, 0) || ' '::text,
''::text) || upper(regexp_replace(btrim((COALESCE(name || ' '::text,
''::text) || COALESCE(extension || ' '::text, ''::text)) ||
COALESCE(direction, ''::text)), ' +'::text, ' '::text, 'g'::text))) || '
'::text) || upper(regexp_replace(btrim(COALESCE(locality_name, ''::text)), '
+'::text, ' '::text, 'g'::text))) || ' '::text) ||
upper(regexp_replace(btrim(COALESCE(state_code, ''::text)), ' +'::text, '
'::text, 'g'::text))) || ' '::text) ||
upper(regexp_replace(btrim(COALESCE(post_code, ''::text)), ' +'::text, '
'::text, 'g'::text))), ' +'::text, ' '::text, 'g'::text) | extended |
btree, for table "address", predicate (start_number IS NOT NULL AND
upper(start_alpha::text) IS NOT NULL AND end_number IS NULL)

All *_cost parameters default on both databases, no amout play with *_cost
setting on 9.1.3 helps. No changes with other settings on 9.1.3 help.

I can't yet create selfcontained test case but I working on in.

It's visible that planner see fast way is really faster/cheaper, however
something prevent planner from flatten subquery.

Kind Regards,
Maksym