Trying to understand Stats/Query planner issue

From: "Strange, John W" <john(dot)w(dot)strange(at)jpmorgan(dot)com>
To: "pgsql-performance(at)postgresql(dot)org" <pgsql-performance(at)postgresql(dot)org>
Subject: Trying to understand Stats/Query planner issue
Date: 2011-11-08 22:33:09
Message-ID: D86EB8F058615F40948C622359204B3F02F63E194C@EMASC201VS01.exchad.jpmchase.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

I have a question on how the analyzer works in this type of scenario.

We calculate some results and COPY INTO some partitioned tables, which we use some selects to aggregate the data back out into reports. Everyone once in a while the aggregation step picks a bad plan due to stats on the tables that were just populated. Updating the stats and rerunning the query seems to solve the problem, this only happens if we enable nested loop query plans.

Which leads me to a few questions:

Assumption: that starts aren't getting created fast enough and then the query planner then picks a bad plan since we query the tables shortly after being populated, so it decided to use a nested loop on a large set of results incorrectly.

- if there are no stats on the table how does the query planner identify the best query plan?
- we have tried really aggressive auto_analyze settings down to .001, so basically any insert will get the analyze running with no luck.
- will an analyze block on update to the statistics tables, which makes me wonder if we are updating too often?

The other option is just to analyze each table involved in the query after the insert, but that seems a bit counterproductive.

Thoughts?
_______________________________________________________________________________________________
| John W. Strange | Vice President | Global Commodities Technology
| J.P. Morgan | 700 Louisiana, 11th Floor | T: 713-236-4122 | C: 281-744-6476 | F: 713 236-3333
| john(dot)w(dot)strange(at)jpmchase(dot)com<mailto:john(dot)w(dot)strange(at)jpmchase(dot)com> | jpmorgan.com

This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.

Responses

Browse pgsql-performance by date

  From Date Subject
Next Message Josh Berkus 2011-11-08 23:29:03 STRICT SQL functions never inline
Previous Message Albe Laurenz 2011-11-08 13:18:18 Re: SSL encryption makes bytea transfer slow