Re: pg_dump and large files - is this a problem?

From: Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
To: Philip Warner <pjw(at)rhyme(dot)com(dot)au>
Cc: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Peter Eisentraut <peter_e(at)gmx(dot)net>, Giles Lean <giles(at)nemeton(dot)com(dot)au>
Subject: Re: pg_dump and large files - is this a problem?
Date: 2002-10-23 14:24:58
Message-ID: 200210231424.g9NEOwu10965@candle.pha.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Philip Warner wrote:
> At 01:02 AM 23/10/2002 -0400, Bruce Momjian wrote:
>
> >OK, you are saying if we don't have fseeko(), there is no reason to use
> >off_t, and we may as well use long. What limitations does that impose,
> >and are the limitations clear to the user.
>
> What I'm saying is that if we have not got fseeko then we should use any
> 'seek-class' function that returns a 64 bit value. We have already made the
> assumption that off_t is an integer; the same logic that came to that
> conclusion, applies just as validly to the other seek functions.

Oh, I see, so try to use fsetpos/fgetpos? I can write wrappers for
those to look like fgetpos/fsetpos and put it in /port.

> Secondly, if there is no 64 bit 'seek-class' function, then we should
> probably use a size_t, but a long would probably be fine too. I am not
> particularly attached to this part; long, int etc etc. Whatever is most
> likely to return an integer and work with whatever function we choose.
>
> As to implications: assuming they are all integers (which as you know I
> don't like), we should have no problems.
>
> If a system does not have any function to access 64 bit file offsets, then
> I'd say they are pretty unlikely to have files > 2GB.

OK, my OS can handle 64-bit files, but has only fgetpos/fsetpos, so I
could get that working. The bigger question is what about OS's that
have 64-bit off_t/files but don't have any seek-type functions. I did
research to find mine, but what about others that may have other
variants?

I think you are right that we have to not use off_t and use long if we
can't find a proper 64-bit seek function, but what are the failure modes
of doing this? Exactly what happens for larger files?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Greg Copeland 2002-10-23 14:30:34 Re: Memory leaks
Previous Message Hans-Jürgen Schönig 2002-10-23 14:24:27 PREPARE / EXECUTE