Re: pg_dump / copy bugs with "big lines" ?

Daniel Verite wrote:

> My tests are OK too but I see an issue with the code in
> enlargeStringInfo(), regarding integer overflow.
> The bit of comment that says:
>
> Note we are assuming here that limit <= INT_MAX/2, else the above
> loop could overflow.
>
> is obsolete, it's now INT_MAX instead of INT_MAX/2.

Hmm, I think what it really needs to say there is UINT_MAX/2, which is
what we really care about. I may be all wet, but what I see is that the
expression
(((Size) 1) << (sizeof(int32) * 8 - 1)) - 1
which is what we use as limit is exactly half the unsigned 32-bit
integer range. So I would just update the constant in that comment
instead of removing it completely. (We're still relying on the loop not
to overflow in 32-bit machines, surely).

> There's a related problem here:
> newlen = 2 * str->maxlen;
> while (needed > newlen)
> newlen = 2 * newlen;
> str->maxlen is an int going up to INT_MAX so [2 * str->maxlen] now
> *will* overflow when [str->maxlen > INT_MAX/2].
> Eventually it somehow works because of this:
> if (newlen > limit)
> newlen = limit;
> but newlen is wonky (when resulting from int overflow)
> before being brought back to limit.

Yeah, this is bogus and your patch looks correct to me.

I propose this:

diff --git a/src/backend/lib/stringinfo.c b/src/backend/lib/stringinfo.c
index b618b37..a1d786d 100644
--- a/src/backend/lib/stringinfo.c
+++ b/src/backend/lib/stringinfo.c
@@ -313,13 +313,13 @@ enlargeStringInfo(StringInfo str, int needed)
* for efficiency, double the buffer size each time it overflows.
* Actually, we might need to more than double it if 'needed' is big...
*/
- newlen = 2 * str->maxlen;
+ newlen = 2 * (Size) str->maxlen;
while (needed > newlen)
newlen = 2 * newlen;

/*
* Clamp to the limit in case we went past it. Note we are assuming here
- * that limit <= INT_MAX/2, else the above loop could overflow. We will
+ * that limit <= UINT_MAX/2, else the above loop could overflow. We will
* still have newlen >= needed.
*/
if (newlen > limit)

--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services