Re: SSI patch version 14

On 09.02.2011 00:04, Kevin Grittner wrote:
> (1) When HTABs are created, there is the max_size, which is what
> the PredicateLockShmemSize function must use in its calculations,
> and the init_size, which is what will initially be allocated (and
> so, is probably what you see in the usage at the end of the
> InitPredLocks function). That's normally set to half the maximum.

Oh, I see.

> (2) The predicate lock and lock target initialization code was
> initially copied and modified from the code for heavyweight locks.
> The heavyweight lock code adds 10% to the calculated maximum size.
> So I wound up doing that for PredicateLockTargetHash and
> PredicateLockHash, but didn't do it for SerializableXidHassh.
> Should I eliminate this from the first two, add it to the third, or
> leave it alone?

I'm inclined to eliminate it from the first two. Even in
LockShmemSize(), it seems a bit weird to add a safety margin, the sizes
of the lock and proclock hashes are just rough estimates anyway.

> So if the space was all in HTABs, you might expect shmemsize to be
> 110% of the estimated maximum, and actual (at the end of the init
> function) to be 50% of the estimated maximum. So the shmemsize
> would be (2.2 * actual) at that point. The difference isn't that
> extreme because the list-based pools now used for some structures
> are allocated at full size without padding.
>
> In addition to the omission of the RWConflictPool (which is a
> biggie), the OldSerXidControlData estimate was only for a *pointer*
> to it, not the structure itself. The attached patch should correct
> the shmemsize numbers.

The actual and estimated shmem sizes still didn't add up, I still saw
actual usage much higher than estimated size, with max_connections=1000
and max_predicate_locks_per_transaction=10. It turned out to be because:

* You missed that RWConflictPool is sized five times as large as
SerializableXidHash, and

* The allocation for RWConflictPool elements was wrong, while the
estimate was correct.

With these changes, the estimated and actual sizes match closely, so
that actual hash table sizes are 50% of the estimated size as expected.

I fixed those bugs, but this doesn't help with the buildfarm members
with limited shared memory yet.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com