Re: Table copy

From: Ron Johnson <ronljohnsonjr(at)gmail(dot)com>
To: "pgsql-generallists(dot)postgresql(dot)org" <pgsql-general(at)lists(dot)postgresql(dot)org>
Subject: Re: Table copy
Date: 2025-02-05 21:17:11
Message-ID: CANzqJaCutOig_e9eW_9tUeKxxs7sX5rUtBthfk9hKOEb5HcTPQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Could there have been a network hiccup? Or some sort of timeout?

If I needed to transfer 360GB of data, I'd probably do something old school
like:

1. write a PowerShell script to export a set of rows into a csv file, 7zip
compress it, then rsync or scp it to the target.
2. Write a bash script to decompress it then load the data into the PG
table.
3. Repeat (1) with the next set of data, and (2) until complete. Start the
second (1) while the first (2) is running.

That's how I migrated 12GB of Oracle data to PG (except of course bash, not
PowerShell).

On Wed, Feb 5, 2025 at 4:05 PM Andy Hartman <hartman60home(at)gmail(dot)com> wrote:

> nothing in log from mssql side and no anti-virus
>
> On Wed, Feb 5, 2025 at 2:06 PM Adrian Klaver <adrian(dot)klaver(at)aklaver(dot)com>
> wrote:
>
>>
>>
>> On 2/5/25 9:46 AM, Andy Hartman wrote:
>> > [6992] ERROR: unexpected EOF on client connection with an open
>> transaction
>> > 2025-02-05 12:19:44.919 EST [6992] CONTEXT: COPY sqlt_data_1_2022_03,
>> > line 24431524, column dataintegrity
>> > 2025-02-05 12:19:44.919 EST [6992] STATEMENT: COPY sqlt_data_1_2022_03
>> > (tagid, intvalue, floatvalue, stringvalue, datevalue, dataintegrity,
>> > t_stamp) FROM STDIN (FORMAT BINARY)
>> > 2025-02-05 12:19:44.919 EST [6992] FATAL: terminating connection
>> > because protocol synchronization was lost
>>
>> You need to look at the other end of the connection also, in other words
>> what is happening on the MS SQL Server 2016/Windows Server 2019 side?
>>
>> Also is there anti-virus software running on either end?
>>
>>
>>
>> --
>> Adrian Klaver
>> adrian(dot)klaver(at)aklaver(dot)com
>>
>

--
Death to <Redacted>, and butter sauce.
Don't boil me, I'm still alive.
<Redacted> lobster!

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Michał Kłeczek 2025-02-05 21:19:30 Re: Lookup tables
Previous Message Andy Hartman 2025-02-05 21:09:03 Re: Table copy