Re: BUG: pg_dump generates corrupted gzip file in Windows

From: Kuntal Ghosh <kuntalghosh(dot)2007(at)gmail(dot)com>
To: Craig Ringer <craig(at)2ndquadrant(dot)com>
Cc: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: BUG: pg_dump generates corrupted gzip file in Windows
Date: 2017-03-24 10:44:27
Message-ID: CAGz5QCLMY3KoU-Djwb_AKbDNwDyGPhSqnzZP43WYgnHQ6bxihg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Fri, Mar 24, 2017 at 2:17 PM, Kuntal Ghosh
<kuntalghosh(dot)2007(at)gmail(dot)com> wrote:
> On Fri, Mar 24, 2017 at 12:35 PM, Craig Ringer <craig(at)2ndquadrant(dot)com> wrote:
>> On 24 March 2017 at 14:07, Kuntal Ghosh <kuntalghosh(dot)2007(at)gmail(dot)com> wrote:
>>> On Fri, Mar 24, 2017 at 11:28 AM, Kuntal Ghosh
>>> <kuntalghosh(dot)2007(at)gmail(dot)com> wrote:
>>>> Hello,
>>>> In Windows, if one needs to take a dump in plain text format (this is
>>>> the default option, or can be specified using -Fp) with some level of
>>>> compression (-Z[0-9]), an output file has to
>>>> be specified. Otherwise, if the output is redirected to stdout, it'll
>>>> create a corrupted dump (cmd is set to ASCII mode, so it'll put
>>>> carriage returns in the file).
>>> To reproduce the issue, please use the following command in windows cmd:
>>>
>>> pg_dump -Z 9 test > E:\test_xu.backup
>>> pg_dump -Fp -Z 9 test > E:\test_xu.backup
>>
>> This is a known problem. It is not specific to PostgreSQL, it affects
>> any software that attempts to use stdin/stdout on Windows via cmd,
>> where it is not 8-bit clean.
>>
>> We don't just refuse to run with stdout as a destination because it's
>> perfectly sensible if you're not using cmd.exe. pg_dump cannot, as far
>> as I know, tell whether it's being invoked by cmd or something else.
> ASAICU, if we use binary mode, output is stored bit by bit. In ASCII
> mode, cmd pokes its nose and does CR / LF conversions on its own. So,
> whenever we want compression on a plain-text dump file, we can set the
> stdout mode to O_BINARY. Is it a wrong approach?
With the help from Ashutosh Sharma, I tested this in Windows
environment. Sadly, it still doesn't work. :( IMHO, we should document
the issue somewhere.

--
Thanks & Regards,
Kuntal Ghosh
EnterpriseDB: http://www.enterprisedb.com

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Langote 2017-03-24 10:48:39 Re: Partition-wise join for join between (declaratively) partitioned tables
Previous Message Heikki Linnakangas 2017-03-24 10:44:24 Re: Re: [COMMITTERS] pgsql: Fix and simplify check for whether we're running as Windows serv