Skip site navigation (1) Skip section navigation (2)

Re: [GENERAL] OIDs - file objects, are damaged by PostgreSQL.

From: Richard Huxton <dev(at)archonet(dot)com>
To: Purusothaman A <purusothaman(dot)a(at)gmail(dot)com>
Cc: pgsql-general(at)postgresql(dot)org, pgsql-admin(at)postgresql(dot)org
Subject: Re: [GENERAL] OIDs - file objects, are damaged by PostgreSQL.
Date: 2007-05-23 15:47:40
Message-ID: 4654621C.8010700@archonet.com (view raw or flat)
Thread:
Lists: pgsql-adminpgsql-general
Purusothaman A wrote:
> Dear Richard Huxton,
> 
> Thanks for your quick reply.
> 
> only the first 3 values(HX, MASK, Rockey4ND) are file object's oid value.
> the other two are are not oid values.

Umm - OK. Can I suggest perhaps having different tables for different 
types of data?

> I have shown original output values displayed by postgresql client.
> 
> I can explain more.
> 
> 1. HX is a XML file. after downloading that file I opened that file in word
> pad application.
> In that I have noticed that nearly 20 characters of last line lost.
> 2. Rockey4ND is a dll file. I was unable to use that dll in my application.
> 
> In both cases, I checked file size. Corrupted files are smaller when 
> compare
> to the original one what I uploaded to postgresql.

OK, so the data seems OK up until that point?

> Usually this problem arises only after the database become large.

I can't think of any reason why that would make a difference. But, this 
does give us a clue. If you have successfully downloaded these files 
before, that rules out certain forms of failure.

> Any suggestion to rectify this problem would be nice of you.

Reading through recent release notes, I can't see anything mentioning 
lo_import/export, large objects or similar.

http://www.postgresql.org/docs/8.2/static/release-8-2-4.html

Well, your data is stored in pg_largeobject. If you run a SELECT you can 
see how it's broken into chunks.
   SELECT loid,pageno,length(data) FROM pg_largeobject ;

On my system, a full chunk is 2048 bytes long. What does the last chunk 
of your HX object (101800) look like? Is it a full chunk? Does it end 
where your downloaded file ends?

If the data is OK in pg_largeobject then we know we have a problem with 
lo_export-ing or saving to a file.

If not, then we know we have a problem with something deleting or 
overwriting chunks in pg_largeobject. That would surprise me, because I 
don't think there's anything special about pg_largeobject - it's just a 
table with chunks of bytea data in it.

Just to recap - you're using lo_import() and lo_export from C (or at 
least via libpq) to read/write these files directly to your filesystem. 
You've not been seeing crashes and you don't think you've got hardware 
problems.

-- 
   Richard Huxton
   Archonet Ltd

In response to

Responses

pgsql-admin by date

Next:From: Jair Elton BatistaDate: 2007-05-24 02:46:21
Subject: Performance Monitor
Previous:From: Purusothaman ADate: 2007-05-23 14:56:21
Subject: Re: [GENERAL] OIDs - file objects, are damaged by PostgreSQL.

pgsql-general by date

Next:From: Joshua D. DrakeDate: 2007-05-23 15:51:16
Subject: Re: Tsearch2: upgrading from postgres 8.1.x to 8.2.x
Previous:From: Alvaro HerreraDate: 2007-05-23 15:22:26
Subject: Re: Faster data type for one-length values

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group