Re: Help - corruption issue?

On Tue, Apr 26, 2011 at 3:24 PM, Scott Marlowe <scott(dot)marlowe(at)gmail(dot)com> wrote:
> On Mon, Apr 25, 2011 at 8:50 PM, Phoenix Kiula <phoenix(dot)kiula(at)gmail(dot)com> wrote:
>>> On Tuesday, April 26, 2011, Tomas Vondra <tv(at)fuzzy(dot)cz> wrote:
>>>> Dne 25.4.2011 18:16, Phoenix Kiula napsal(a):
>>>>> Sorry, spoke too soon.
>>>>>
>>>>> I can COPY individual chunks to files. Did that by year, and at least
>>>>> the dumping worked.
>>>>>
>>>>> Now I need to pull the data in at the destination server.
>>>>>
>>>>> If I COPY each individual file back into the table, it works. Slowly,
>>>>> but seems to work. I tried to combine all the files into one go, then
>>>>> truncate the table, and pull it all in in one go (130 million rows or
>>>>> so) but this time it gave the same error. However, it pointed out a
>>>>> specific row where the problem was:
>>>>>
>>>>> COPY links, line 15272357:
>>>>> "16426447     9s2q7   9s2q7   N       http://www.amazon.com/gp/search?camp=1789&amp;creative=9325&amp;ie=UTF8&amp;i..."
>>>>> server closed the connection unexpectedly
>>>>>       This probably means the server terminated abnormally
>>>>>       before or while processing the request.
>>>>> The connection to the server was lost. Attempting reset: Failed.
>>>>>
>>>>> Is this any use at all?  Would appreciate any pointers!
>>>>
>>>> So the dump worked fina and it fails when loading it back into the DB?
>>>> Have you checked the output file (just see the tail). Can you post the
>>>> part that causes issues? Just the line 16426447 and few lines around.
>>>>
>>>> regards
>>>> Tomas
>>
>> From the old server:
>> Yearly COPY files worked. Pg_dumpall was giving problems.
>>
>> In the new server:
>> COPY FROM worked. All files appear to have been copied. Then I create
>> the primary key index, and another index. Many records are there, but
>> many are not there! There's no error, just that some records/rows just
>> didn't make it.
>
> Are you sure you're getting all the data out of the source (broken)
> database you think you are?  Are you sure those rows are in the dump?

Actually I am not. Some rows are missing.

Will a COUNT(*) on the two databases -- old and new -- be sufficient
and reliable information about the number of rows that went AWOL?