From: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
---|---|
To: | pgsql-hackers(at)postgreSQL(dot)org |
Subject: | mdnblocks is an amazing time sink in huge relations |
Date: | 1999-10-12 04:12:28 |
Message-ID: | 15265.939701548@sss.pgh.pa.us |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
I have been doing some testing with multi-segment relations.
(Not having sixty gigabytes laying around, like some folk I've been
talking to recently, I rebuilt with a very small RELSEG_SIZE for
testing...) I have discovered that performance goes to zero as the
number of segments gets large --- in particular, if the number of
segments exceeds the number of kernel file descriptors that fd.c
thinks it can use, you can take a coffee break while inserting a
few tuples :-(. The main problem is mdnblocks() in md.c, which
is called for each tuple inserted; it insists on touching every
segment of the relation to verify its length via lseek(). That's an
unreasonable number of kernel calls in any case, and if you add a file
open and close to each touch because of file-descriptor thrashing,
it's coffee break time.
It seems to me that mdnblocks should assume that all segments
previously verified to be of length RELSEG_SIZE are still of that
length, and only do an _mdnblocks() call on the last element of the
segment chain. That way the number of kernel interactions needed
is a constant independent of the number of segments in the table.
(This doesn't make truncation of the relation by a different backend
any more or less dangerous; we're still hosed if we don't get a
notification before we try to do something with the relation.
I think that is fixed, and if it's not this doesn't make it worse.)
Any objections?
regards, tom lane
From | Date | Subject | |
---|---|---|---|
Next Message | The Hermit Hacker | 1999-10-12 04:16:26 | Re: [HACKERS] Re: [INTERFACES] Next release is 7.0(?) |
Previous Message | The Hermit Hacker | 1999-10-12 04:07:57 | Re: [HACKERS] Features for next release |