Re: Vacuum: allow usage of more than 1GB of work mem

On Tue, Apr 11, 2017 at 4:38 PM, Claudio Freire <klaussfreire(at)gmail(dot)com> wrote:
> In essence, the patch as it is proposed, doesn't *need* a binary
> search, because the segment list can only grow up to 15 segments at
> its biggest, and that's a size small enough that linear search will
> outperform (or at least perform as well as) binary search. Reducing
> the initial segment size wouldn't change that. If the 12GB limit is
> lifted, or the maximum segment size reduced (from 1GB to 128MB for
> example), however, that would change.
>
> I'd be more in favor of lifting the 12GB limit than of reducing the
> maximum segment size, for the reasons above. Raising the 12GB limit
> has concrete and readily apparent benefits, whereas using bigger (or
> smaller) segments is far more debatable. Yes, that will need a binary
> search. But, I was hoping that could be a second (or third) patch, to
> keep things simple, and benefits measurable.

To me, it seems a bit short-sighted to say, OK, let's use a linear
search because there's this 12GB limit so we can limit ourselves to 15
segments. Because somebody will want to remove that 12GB limit, and
then we'll have to revisit the whole thing anyway. I think, anyway.

What's not clear to me is how sensitive the performance of vacuum is
to the number of cycles used here. For a large index, the number of
searches will presumably be quite large, so it does seem worth
worrying about performance. But if we just always used a binary
search, would that lose enough performance with small numbers of
segments that anyone would care? If so, maybe we need to use linear
search for small numbers of segments and switch to binary search with
larger numbers of segments.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company