Re: Is my database now too big?

"Darren Reed" <darrenr+postgres(at)fastmail(dot)net> writes:
> * pg_dump fails to run, causing an "out of memory" error (I don't
> understand
> why this happens, but it does);

Let's see the exact command to pg_dump and the exact failure message.
It might be useful to add -v so we can get some sense of where in the
process it fails.

> * restarting postgres across a reboot results in tables disappearing
> without any error messages being given out (this is *really
> disturbing*.)

Hm, system reboot you mean? To be honest I think this is likely pilot
error, along the lines of the postmaster you are running after the
reboot is using a different data directory than the one you were talking
to before. Comparing the output of "SHOW data_directory;" before and
after might be illuminating.

I remember a vaguely similar incident awhile back in which it turned out
that the DBA had put the data directory on a soft-mounted NFS server,
which sometimes hadn't come up by the time Postgres started, and so
sometimes he got the NFS server's version of the database and sometimes
he got a version that was on the local disk underneath the NFS mount
point. *That* was a mess :-(, and that was one reason why the PGDG
and Red Hat init scripts for PG no longer do an automatic initdb if
they don't find a valid data directory where it's supposed to be.

Exactly how are you starting the postmaster, anyway, and what is the
underlying platform here? Did you roll-your-own Postgres build or
is this a prepackaged distribution? If the latter, whose?

While I'm asking questions, is it just the tables that disappear or
is it all your database objects (including functions etc)? Do
*all* your tables disappear or just some of them? What do you
mean by "disappear" exactly --- do you get 'relation "foo" does not
exist', or some weirder error, or are the tables present but empty?

> "wc -l" of the output from the last successuful dump_all is around
> 8million lines, spread across half a dozen or so tables.

As already pointed out, this database is tiny. I'm not sure what your
problem is, but I'm quite sure it's not "database is too big".

> I'm currently using 8.1.3, will I fare any better with a more recent
> version?

You should certainly be on 8.1.10 just on general principles, though
I don't immediately recall any bugs that seem like they could be
related. There have been plenty of bugs fixed since 8.1.3 though.

regards, tom lane