Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

Hello Tom,

>> Here is a POC which defines an internal interface for a PRNG, and use it
>> within pgbench, with several possible implementations which default to
>> rand48.
>
> I seriously dislike this patch. pgbench's random support is quite
> overengineered already IMO, and this proposes to add a whole batch of
> new code and new APIs to fix a very small bug.

My intention is rather to discuss postgres' PRNG, in passing. Full success
on this point:-)

>> I must admit that I have a grudge against standard rand48:
>
> I think this is nonsense, particularly the claim that anything in PG
> cares about the lowest-order bits of random doubles. I'm aware that
> there are applications where that does matter, but people aren't doing
> high-precision weather simulations in pgbench.

Sure. My point is not that it is an actual issue for pgbench, but as the
same PRNG is used more or less everywhere in postgres, I think that it
should be a good one rather than a known bad one.

Eg, about double:

\set i debug(random(1, POWER(2,49)) % 2)

Always return 1 because of the 48 bit precision, i.e. the output is never
even.

\set i debug(random(1, POWER(2,48)) % 2)

Return 0 1 0 1 0 1 0 1 0 1 0 1 0 1 ... because it is a LCG.

\set i debug(random(1, POWER(2,48)) % 4)

Cycles over (3 2 1 0)*

\set i debug(random(1, power(2, 47)) % 4)

Cycles over (0 0 1 1 2 2 3 3)*, and so on.

> BTW, did you look at the question of the range of zipfian?

Yep.

> I confirmed here that as used in the test case, it's generating a range
> way smaller than the other ones: repeating the insertion snippet 1000x
> produces stats like this: [...]

> I have no idea whether that indicates an actual bug, or just poor
> choice of parameter in the test's call. But the very small number
> of distinct outputs is disheartening at least.

Zipf distribution is highly skewed, somehow close to an exponential. To
reduce the decreasing probability the parameter must be closer to 1, eg
1.05 or something. However as far as the test is concerned I do not see
this as a significant issue. I was rather planning to submit a
documentation improvement to provide more precise hints about how the
distribution behaves depending on the parameter, and possibly reduce the
parameter used in the test in passing, but I see this as not very urgent.

--
Fabien.