pg_verify_checksums vs windows

From: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
To: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Magnus Hagander <magnus(at)hagander(dot)net>
Subject: pg_verify_checksums vs windows
Date: 2018-08-29 11:31:35
Message-ID: CAA4eK1+LOnzod+h85FGmyjWzXKy-XV1FYwEyP-Tky2WpD5cxwA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

While trying to debug a recent bug report on hash indexes [1], I
noticed that pg_verify_checksums don't work on Windows (or at least in
my environment).

initdb -k ..\..\data
pg_verify_checksums.exe ..\..\Data
pg_verify_checksums: short read of block 0 in file
"..\..\Data/global/1136", got only 15 bytes

I have debugged and found that below code is the culprit.

scan_file(char *fn, int segmentno)
{
..
f = open(fn, 0);
..
int r = read(f, buf, BLCKSZ);

if (r == 0)
break;

if (r != BLCKSZ)
{
fprintf(stderr, _("%s: short read of block %d in file \"%s\", got only
%d bytes\n"),
progname, blockno, fn, r);
exit(1);
}
..
}

We are opening the file in text mode and trying to read the BLCKSZ
bytes, however, if there is any Control-Z char, it is treated as EOF.
This problem has been mentioned in the comments in c.h as follows:
/*
* NOTE: this is also used for opening text files.
* WIN32 treats Control-Z as EOF in files opened in text mode.
* Therefore, we open files in binary mode on Win32 so we can read
* literal control-Z. The other affect is that we see CRLF, but
* that is OK because we can already handle those cleanly.
*/

So, I think we need to open the file in binary mode as in other parts
of the code. The attached patch fixes the problem for me.

Thoughts?

[1] - https://www.postgresql.org/message-id/5d03686d-727c-dbf8-0064-bf8b97ffe850%402ndquadrant.com
--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

Attachment Content-Type Size
pg_verify_checksums_1.patch application/octet-stream 482 bytes

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Yugo Nagata 2018-08-29 11:33:43 Re: pg_verify_checksums -d option (was: Re: pg_verify_checksums -r option)
Previous Message Andrew Gierth 2018-08-29 11:31:09 Re: Catalog corruption