Re: mdnblocks() sabotages error checking in _mdfd_getseg()

From: Andres Freund <andres(at)anarazel(dot)de>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: mdnblocks() sabotages error checking in _mdfd_getseg()
Date: 2015-12-10 16:36:52
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 2015-12-10 11:10:10 -0500, Robert Haas wrote:
> (More broadly, as Kevin was pointing out to me yesterday, md.c looks
> like it could do with a face lift. Keeping a linked list of 1GB
> segments and chasing down the list to find the length of the file may
> have been fine when relations over 1GB were rare, but that's now
> routine. Some relations may be over 1TB, and using a linked list of
> 1000 entries to keep track of all of those segments does not seem like
> an especially good choice.

Yes, that sucks. I've posted a patch for that at
- I don't have access to to either the workload or a good testing
machine anymore though, so I've kinda lost interest for a while.

I'll try to push the patch with the renaming suggested downthread by Tom

> In fact, having no way to get the relation length other than scanning
> 1000 files doesn't seem like an especially good choice even if we used
> a better data structure. Putting a header page in the heap would make
> getting the length of a relation O(1) instead of O(segments), and for
> a bonus, we'd be able to reliably detect it if a relation file
> disappeared out from under us. That's a difficult project and
> definitely not my top priority, but this code is old and crufty all
> the same.)

The md layer doesn't really know whether it's dealing with an index, or
with an index, or ... So handling this via a metapage doesn't seem
particularly straightforward.


In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message Robert Haas 2015-12-10 16:37:23 Re: Patch: ResourceOwner optimization for tables with many partitions
Previous Message Andres Freund 2015-12-10 16:32:30 Re: Error with index on unlogged table