RE: [HACKERS] mdnblocks is an amazing time sink in huge relations

>
> > Currently only the first segment is opened when mdopen() etc is
> > called. Would you change to check all segments at first open ?
>
> No, I don't see a need to change that logic. I was thinking that
> since mdnblocks adds another link to the chain whenever it sees that
> the current segment is exactly RELSEG_SIZE, it could just assume that
> if the next-segment link is not NULL then this segment must be of
> size RELSEG_SIZE and it doesn't need to check that again. It'd only
> need to do an actual check on the last chain element (plus any elements
> it adds during the current call, of course). We'd rely on the
> higher-level interlock to close and reopen the virtual file when a
> truncate has happened.
>

You are right.
The last segment is always lseeked.
And does this mean that the first mdnblocks() opens all segments and
checks them ?

> > If there is a relation which has multi-segment of size 0,which would
> > be the last segment ?
>
> Only the last segment can have any size other than RELSEG_SIZE, I think.
>

This may be another issue.
I have been suspicious about current implementation of md.c.
It relies so much on information about existent phisical files.

How do you think about the following ?

1. Partial blocks(As you know,I have changed the handling of this
kind of blocks recently).
2. If a backend was killed or crashed in the middle of execution of
mdunlink()/mdtruncate(),half of segments wouldn't be unlink/
truncated.
3. In cygwin port,mdunlink()/mdtruncate() may leave segments of 0
length.
4. We couldn't mdcreate() existent files and coudn't mdopen()/md
unlink() non-existent files. So there are some cases that we
could neither CREATE TABLE nor DROP TABLE.

I have no solution but seems the count of valid segments and blocks
should be held in other places.

Regards.

Hiroshi Inoue
Inoue(at)tpf(dot)co(dot)jp