Re: BUG #3571: call to decrypt causes segfault

"Ken Colson" <ken(dot)colson(at)sage(dot)com> writes:
> this statement:
> select decrypt(''::bytea,'password','bf')
> causes the postgresql backend to crash:
> This seems to be a 64bit problem.

Reproduced here in HEAD. The problem is here:

Program terminated with signal 11, Segmentation fault.
#0 0x00002aaaad2d41f0 in combo_decrypt (cx=0xb182f8,
data=0xac991c "~\177\177\177", dlen=0, res=0xb1838c "~\177\177\177@",
rlen=0x7fffc1f499e4) at px.c:293
293 pad = res[*rlen - 1];
(gdb) p res
$1 = (uint8 *) 0xb1838c "~\177\177\177@"
(gdb) p rlen
$2 = (unsigned int *) 0x7fffc1f499e4
(gdb) p *rlen
$3 = 0

What apparently is happening is that the compiler chooses to interpret
"res[-1]" as "res[0xFFFFFFFF]". On a 32-bit machine that wraps around
and you touch the previous byte, but on a 64-bit machine you touch
someplace in never-never land.

The problem clearly is that combo_decrypt()'s depadding code fails to
consider the possibility of a zero-length input, but I'm not entirely
sure how far up the food chain we ought to fix it --- perhaps
pg_decrypt() should not have bothered to light up the decryptor at all?
Also, what other pgcrypto routines might have similar bugs?

Marko, any time to work on this?

regards, tom lane