Skip site navigation (1) Skip section navigation (2)

Re: [BUGS] General Bug Report: Files greater than 1 GB are created while sorting

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Bruce Momjian <maillist(at)candle(dot)pha(dot)pa(dot)us>
Cc: Doug Mitchell <doug(at)mitchcraft(dot)com>, pgsql-bugs(at)postgreSQL(dot)org
Subject: Re: [BUGS] General Bug Report: Files greater than 1 GB are created while sorting
Date: 1999-07-08 14:24:29
Message-ID: 28965.931443869@sss.pgh.pa.us (view raw or flat)
Thread:
Lists: pgsql-bugs
Bruce Momjian <maillist(at)candle(dot)pha(dot)pa(dot)us> writes:
> I have renamed these sort temp tables to pg_sorttemp so they will not be
> confused with actual temp tables.

I didn't realize that the names generated for temp tables were so close
to those generated for temp files.  Changing one or the other does seem
like a good idea.  But I do not like "pg_sorttemp" because fd.c's
temporary-file mechanism is used for more things than just sorting.
Hash joins, for example.  Can we think of a better name?

Alternatively, how about including the user-given name for a temp table
into its real name?  That would be helpful for debugging, I'm sure.
I'm thinking of something like

	snprintf(newrelname, NAMEDATALEN, "pg_temp.%d.%u.%s",
		 (int) MyProcPid, uniqueId++, userrelname);

(relying on snprintf to truncate the user name if too long, here).


> You are safe up to 2 gigs, and at that point, the OS will can cause a
> problem.  The new naming should make the cause clearer.  Don't know if
> we can get this done in 6.5.1 because the change to segment these
> requires some work.  Looks like the psort code goes right to fd/*,
> bypassing the storage manager.

Yes, it will take some thought to figure out how to handle multi-segment
temp files without cluttering the code too badly.  I think it can be
handled inside fd.c, though.

Note that under ordinary circumstances, the data being processed by a
sort or hash join will be written into several temp files that each get
just a fraction of the data; so you would not actually see a problem
until you got to several-times-2-Gig total data volume.

			regards, tom lane

Responses

pgsql-bugs by date

Next:From: PostmasterDate: 1999-07-08 14:47:42
Subject: Please update and reply
Previous:From: secretDate: 1999-07-08 14:00:58
Subject: Re: [BUGS] General Bug Report: GROUP BY with NULL not done properly(Oracle8& DB/2 do this completely different)

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group