Skip site navigation (1) Skip section navigation (2)

Re: On Scalability

From: Vincenzo Romano <vincenzo(dot)romano(at)notorand(dot)it>
To: jd(at)commandprompt(dot)com
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: On Scalability
Date: 2010-07-29 17:34:20
Message-ID: AANLkTikV96eAFxEuqk4k6jadtDomLhnJrwqtEh3mNKs6@mail.gmail.com (view raw or flat)
Thread:
Lists: pgsql-hackerspgsql-performance
2010/7/29 Joshua D. Drake <jd(at)commandprompt(dot)com>:
> On Thu, 2010-07-29 at 19:08 +0200, Vincenzo Romano wrote:
>> Hi all.
>> I'm wondering about PGSQL scalability.
>> In particular I have two main topics in my mind:
>>
>> 1. What'd be the behavior of the query planner in the case I have
>> a single huge table with hundreds or thousands of partial indexes
>> (just differing by the WHERE clause).
>> This is an idea of mine to make index-partitioning instead of
>> table-partitioning.
>
> Well the planner is not going to care about the partial indexes that
> don't match the where clause but what you are suggesting is going to
> make writes and maintenance extremely expensive. It will also increase
> planning time as the optimizer at a minimum has to discard the use of
> those indexes.
>
>>
>> 2. What'd be the behavior of the query planner in the case I have
>> hundreds or thousands of child tables, possibly in a multilevel hierarchy
>> (let's say, partitioning by year, month and company).
>
> Again, test it. Generally speaking the number of child tables directly
> correlates to planning time. Most experience that 60-100 tables is
> really the highest you can go.
>
> It all depends on actual implementation and business requirements
> however.
>
> Sincerely,
>
> Joshua D. Drake

I expect that a more complex schema will imply higher workloads
on the query planner. What I don't know is how the increase in the
workload will happen: linearly, sublinearly, polinomially or what?

Significant testing would require a prototype implementation with
an almost complete feed of data from the current solution.
But I'm at the feasibility study stage and have not enough resources
for that.

Thanks anyway for the insights, Joshua.
Does the 60-100 tables limit applies to a single level
of inheritance? Or is it more general?

-- 
NotOrAnd Information Technologies
Vincenzo Romano
--
NON QVIETIS MARIBVS NAVTA PERITVS

In response to

Responses

pgsql-performance by date

Next:From: Josh BerkusDate: 2010-07-29 17:39:14
Subject: Re: Pooling in Core WAS: Need help in performance tuning.
Previous:From: Joshua D. DrakeDate: 2010-07-29 17:12:06
Subject: Re: On Scalability

pgsql-hackers by date

Next:From: Joshua D. DrakeDate: 2010-07-29 17:39:44
Subject: Re: On Scalability
Previous:From: Robert HaasDate: 2010-07-29 17:20:20
Subject: Re: string_to_array has to be stable?

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group