Re: pg_dump and large files - is this a problem?

From: "Zeugswetter Andreas SB SD" <ZeugswetterA(at)spardat(dot)at>
To: "Bruce Momjian" <pgman(at)candle(dot)pha(dot)pa(dot)us>
Cc: "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>, "Peter Eisentraut" <peter_e(at)gmx(dot)net>, "Philip Warner" <pjw(at)rhyme(dot)com(dot)au>, "PostgreSQL-development" <pgsql-hackers(at)postgresql(dot)org>, "Giles Lean" <giles(at)nemeton(dot)com(dot)au>
Subject: Re: pg_dump and large files - is this a problem?
Date: 2002-10-28 16:35:57
Message-ID: 46C15C39FEB2C44BA555E356FBCD6FA4961ED4@m0114.s-mxs.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers


> > > The question is *which* seek APIs we need to support. Are there any
> > > besides fseeko() and fgetpos()?
> >
> > On AIX we have
> > int fseeko64 (FILE* Stream, off64_t Offset, int Whence);
> > which is intended for large file access for programs that do NOT
> > #define _LARGE_FILES
> >
> > It is functionality that is available if _LARGE_FILE_API is defined,
> > which is the default if _LARGE_FILES is not defined.
> >
> > That would have been my preferred way of handling large files on AIX
> > in the two/three? places that need it (pg_dump/restore, psql and backend COPY).
> > This would have had the advantage that off_t is not 64 bit in all other places
> > where it is actually not needed, no ?
>
> OK, I am focusing on AIX now. I don't think we can go down the road of
> saying where large file support is needed or not needed. I think for
> each platform either we support large files or we don't. Is there a way
> to have off_t be 64 bits everywhere, and if it is, why wouldn't we just
> enable that rather than poke around figuring out where it is needed?

if _LARGE_FILES is defined, off_t is 64 bits on AIX (and fseeko works).
The problem with flex is, that the generated c file does #include <unistd.h>
before we #include "postgres.h".
In this situation _LARGE_FILES is not defined for unistd.h and unistd.h
chooses to define _LARGE_FILE_API, those two are not compatible.

If a general off_t of 64 bits is no performance problem, we should focus
on fixing the #include <unistd.h> issue, and forget what I wanted/hinted.
Peter E. has a patch for this in his pipeline. I can give it a second try
tomorrow.

Sorry for the late answer, I am very pressed currently :-(
Andreas

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message snpe 2002-10-28 17:14:24 Re: 7.3b3 build --with-perl fails on RH 8.0
Previous Message Dave Prosser 2002-10-28 16:32:45 Re: [Fwd: Re: UnixWare 7.1.3 (BETA), C99 compiler,