Re: Patch queue triage

For those who have forgotten the progress we have made toward 8.3, here
are the open patches we had for 8.3 as of May 1, 2006:

---------------------------------------------------------------------------

Tom Lane wrote:
> Bruce Momjian <bruce(at)momjian(dot)us> writes:
> > FYI, Tom, Heikki, I need one of you to post the list of patches and
> > where we think we are on each one, even if the list is imperfect.
>
> This message is an attempt to sort out which patch queue entries have
> no hope of getting into 8.3 (and so we shouldn't spend more time on them
> right now), which ones can get in but are waiting on their authors for
> more work, and which ones are ready for immediate review.
>
> You'll notice that numerically quite a lot of the patches fall into the
> "dead" category. This isn't actually all that surprising because if
> they were apply-able they'd have gotten in. (It's not like we've
> completely neglected applying patches for the last six months...)
> However, many of the remaining live patches are huge ones, like HOT and
> delayed commit, and are going to consume considerable review effort
> (again, if they didn't, they might have been in already).
>
> The bottom line is that we are desperately in need of more review
> manpower. I'm pleased to report that Heikki Linnakangas has promised to
> devote most of the next few weeks to helping review, and has already
> picked out some patches he feels qualified to work on, as you'll see
> below. I have also been coordinating reviewing assignments off-list with
> Neil and Magnus to avoid duplication of effort. But I'd like to encourage
> everyone who's done any backend hacking to look at the live patches not
> shown as assigned to someone specific. The more eyeballs the better;
> anything you catch is something someone else might miss. Also, several
> of the open patches are in need of more performance testing, so if you
> can help out in that fashion please do so.
>
> With that, the patches:
>
>
> * Re: [pgsql-patches] [PATCHES] [HACKERS] [Fwd: Index Advisor]
> /Gurjeet Singh/
>
> I don't think this can be applied in anything like its current state.
> The internal interface to the planner is not very good, and ditto for the
> user API. What I would suggest trying to do is get a set of plugin hooks
> defined for the planner that would allow the advisor to be implemented
> entirely as an external module, and then let it be worked on as a
> pgfoundry project. I have some ideas about appropriate design of the
> plugin hooks (as mentioned in my review message of a month ago), but I
> have to say that getting that done for 8.3 is lower-priority than some of
> the other patches.
>
> * [pgsql-patches] Ctid chain following enhancement
> /Pavan Deolasee/
>
> I'm not very excited about this --- it seems to me to complicate the code
> in some places that are not in fact performance-critical. While it
> doesn't seem likely to break things, I'm not in favor of reducing code
> readability unless a measurable performance improvement can be shown.
> Can we have some tests showing this is worthwhile?
>
> * [PATCHES] Error correction for n_dead_tuples
> /ITAGAKI Takahiro/
>
> Waiting for OldestXmin patch to be accepted or rejected. However,
> regardless of the fate of that patch, I'm concerned that this one creates
> an open-loop behavior in which the n_dead_tuples estimate will diverge
> arbitrarily far from reality over time. I criticized the original
> proposal on that basis, and I'm not convinced this version fixes it,
> because of the fact that stats counter updates occur much later than the
> actions they count. (My recent patch to rate-limit tabstat messages made
> that problem worse, but it existed anyway.) What might make sense is for
> vacuum to count the number of dead-but-not-removable tuples it skips over,
> and apply that as the value of n_dead_tuples on receipt of the vacuum
> message (instead of setting to zero as now). This is likely to be wrong
> with respect to the actions of transactions running concurrently with the
> vacuum, but I think so is the proposed patch; and at least in this form
> the error certainly cannot accumulate across vacuum cycles.
>
> * [PATCHES] Table function support /Pavel Stehule/
>
> Neil has promised to review this. AFAIK there are no showstoppers but
> there are some disagreements on the details of the functionality.
>
> * [HACKERS] Grouped Index Tuples /Heikki Linnakangas/
> * [HACKERS] Grouped Index Tuples / Clustered Indexes
> /Heikki Linnakangas/
>
> Needs review. I'm not sure how many people besides Heikki have really
> looked at this (I know I haven't).
>
> * Re: [PATCHES] POSIX shared memory support
> /Chris Marcellino/
>
> I'm not convinced that we want this at all. The original proposal was
> to provide an alternative for platforms without SysV shmem support,
> but the working versions of the patch fail to remove the SysV dependency,
> and the portability of this code is itself quite unproven. Do we really
> want to take on maintenance of yet-another shmem implementation just to
> let people be lazy about changing their SHMMAX settings? As best I can
> tell the main problem in that department is Darwin, and it'd be a whole
> lot simpler if Apple just raised their darn default SHMMAX to something
> that's sensible for the 21st century.
>
> [BTW, there was a patch submitted recently to use SysV semaphores instead
> of Posix on Darwin versions that have 'em; why isn't that in the queue?
> It makes sense to me to do that to save on file descriptors ...]
>
> * [HACKERS] Automatic adjustment of bgwriter_lru_maxpages (was:
> Dead Space Map version 2) /ITAGAKI Takahiro/
>
> Heikki is reviewing this. AIUI there are no known showstoppers but
> we'd like more performance measurements, plus there are interactions
> with other open patches to investigate.
>
> * [HACKERS] Stream bitmaps /Heikki Linnakangas/
>
> I think this is on hold unless a finished bitmap-index patch shows up;
> however there was some discussion of using this in support of clustered
> indexes, so maybe it's live anyway? Heikki?
>
> * [PATCHES] Heap page diagnostic/test functions (v2)
> /Simon Riggs/
>
> Heikki is reviewing this and refactoring it as a contrib module.
>
> * [PATCHES] non-recursive WITH clause support
> /Gregory Stark/
>
> I think the consensus is that we should wait for a more complete WITH
> implementation to be submitted, since this one creates compatibility
> issues without any real return in the form of added functionality.
>
> * Re: [HACKERS] Bug: Buffer cache is not scan resistant
> /Simon Riggs/
>
> Heikki is reviewing this one, in conjunction with the sync-scan patch.
> Performance testing is needed for both.
>
> * [PATCHES] Updated bitmap index patch /Gavin Sherry/
>
> We're still waiting to see if a finished patch will be submitted.
> I'm very close to saying this has to wait for 8.4, though.
>
> * Re: [HACKERS] [PATCHES] Bitmapscan changes
> /Heikki Linnakangas/
>
> I had objected to this on the grounds that it seemed to be covering
> only a narrow territory between HOT and bitmap indexes, but given the
> probability that one or even both of those won't make it, we need to
> give this one a second look. So: live, needs review.
>
> * Re: [PATCHES] Synchronized Scan WIP patch
> /Simon Riggs/
>
> Heikki is reviewing this one. Also I believe Greg Smith is doing more
> performance testing.
>
> * Re: [HACKERS] [PATCHES] xml2 contrib patch supporting default XML
> namespaces /Mike Rylander/
>
> I would like to get Peter to deal with this and the other XML-related
> patch.
>
> * [HACKERS] Indexam interface proposal /Heikki Linnakangas/
>
> AFAICS this discussion died out with no actual patch submitted.
>
> * [PATCHES] Load distributed checkpoint V3
> /ITAGAKI Takahiro/
>
> Heikki is reviewing this one. More performance testing is needed,
> I think.
>
> * [HACKERS] Group Commit /Heikki Linnakangas/
>
> I believe this is on hold for 8.4; in any case it should wait till
> after Simon's deferred-commit patch is in and we have more info about
> the performance implications of that.
>
> * [PATCHES] Patch for circular buffer in tuplestore to optimize
> merge joins (v1) /stark/
>
> I will handle this --- AFAIK there are no showstopper objections.
>
> * [PATCHES] Concurrent psql v4 [WIP] /stark/
>
> This is waiting on the author to change the API per list discussions; we
> can't apply something that is about to have some commands removed ...
>
> * Re: [PATCHES] Recalculating OldestXmin in a long-running vacuum
> /Heikki Linnakangas/
>
> This is on hold until the other vacuum-related patches are accepted
> or rejected; at that point we need to redo the performance tests.
> As things stand it should *not* be accepted because the only available
> test shows a performance decrease, but maybe that will change after
> the other patches go in. (In any case it'd be interesting to find
> out why there's no performance improvement, but it seems unlikely
> we have time for that investigation for 8.3.)
>
> * Re: [HACKERS] Modifying TOAST thresholds /Tom Lane/
>
> At this point it seems nothing will be done about this issue for 8.3.
>
> * [PATCHES] plpgpsm /Pavel Stehule/
>
> I think this has to be held for 8.4: it was submitted too late for the 8.3
> feature deadline, and in fact I don't recall that there was any community
> discussion/consensus on accepting it into core at all. Another big
> problem is that it largely copies-and-pastes the plpgsql code, which I
> think is an unacceptable maintenance burden in the long run. We need to
> consider whether we can't refactor to avoid code duplication.
>
> * [HACKERS] tsearch_core patch for inclusion
> /Teodor Sigaev/
>
> Have we resolved whether the API and the dump/restore strategy are
> acceptable? The code needs review too, but not till we've settled
> on the basic question whether we like the feature set.
>
> * Re: [PATCHES] [HACKERS] Full page writes improvement, code update
> /Koichi Suzuki/
>
> I feel that we have to insist that this be modified to not create any WAL
> bloat in the pre-compression form. That will be more efficient and will
> avoid the whole argument about whether a switch is needed. There was also
> doubt about whether the external programs (pg_compresslog etc) were ready
> for prime time. At this point we could accept a patch that makes whatever
> small tweaks are needed to ensure that an incremental-format WAL record
> can be extracted from a full-page-write record, at least for the most
> common WAL record types for which compression is actually important. I
> think the actual compression/decompression programs could undergo further
> development on pgfoundry with an eye to merging them into core before 8.4
> release.
>
> * [PATCHES] Dead Space Map version 3 (simplified)
> /ITAGAKI Takahiro/
>
> I want to reject this in its current form. It's been clear almost since
> the FSM code was first released to the field that its dependence on a
> fixed-size shared memory area was a serious design blunder (one for which
> I take full blame; I thought it'd be workable, I was wrong). We can't
> sanely take a patch that repeats that mistake. It seems fairly clear that
> DSM could be redesigned to be stored in "special space" in every Nth heap
> page and thereby avoid the problem. We should do that and look at whether
> FSM couldn't be integrated into the same data structure. (I'm not
> entirely sure how to handle FSM for indexes in this fashion, but maybe we
> should just throw that problem back onto the shoulders of the index AMs;
> for instance make them keep linked lists of free index pages.)
> Obviously this won't happen for 8.3, but I don't think DSM as it stands
> would be a net plus for users.
>
> * Re: [PATCHES] COPY-able sql log outputs /FAST PostgreSQL/
>
> Are we agreed on the API for this? AFAIR there were no showstoppers
> but there was some considerable discussion about API details.
>
> * [PATCHES] Updateable cursors patch /FAST PostgreSQL/
>
> This is incomplete, and I fear at this point has to be held over to 8.4.
>
> * [PATCHES] Preliminary GSSAPI Patches /Henry B. Hotz/
>
> Magnus is reviewing this one.
>
> * [PATCHES] Logging checkpoints and other slowdown causes
> /Greg Smith/
>
> Heikki is reviewing this one.
>
> * Re: [PATCHES] [Fwd: Deferred Transactions, Transaction Guarantee
> and COMMITwithout waiting] /Simon Riggs/
>
> Simon is on the hook to submit an updated patch. I hope this one
> makes it in, as it looks like a really nice performance improvement
> for those who can use it; but the original patch seems overcomplicated.
>
> * Re: [HACKERS] Arrays of Complex Types /David Fetter/
>
> Waiting on the author to respond to open issues; in particular the
> problem of not making array types for system catalogs.
>
> * Re: [HACKERS] Implicit casts to text /Peter Eisentraut/
>
> I had volunteered to finish this, but don't be surprised if it gets
> put off till 8.4.
>
> * Re: [GENERAL] dropping role w/dependent objects
> /Ed L./
>
> Patch seems ok, comment maybe overkill. Will apply this shortly.
>
> * Re: [HACKERS] pg_index updates and SI invalidation
> /Pavan Deolasee/
>
> Needs some review, I don't think there are any showstoppers though.
>
> * [PATCHES] HOT Patch - Ready for review /Pavan Deolasee/
>
> This needs a *lot* of review. Can we break it down into more manageable
> chunks? I'm not sure that anyone's got a full grasp of the implications
> of this patch, and that's a scary thought.
>
> * [PATCHES] Minor recovery changes /Simon Riggs/
>
> The submission mentions open issues --- when will those be resolved?
> Should we apply the patch-so-far anyway?
>
> * Re: [PATCHES] UTF8MatchText /ITAGAKI Takahiro/
>
> I'd like someone who uses UTF8 more than I do to double-check whether this
> is sane. Also, can we avoid having so much code duplication between the
> various forms of LIKE?
>
> * [PATCHES] Clear up strxfrm() in UTF-8 with locale on Windows
> /ITAGAKI Takahiro/
>
> Someone else needs to look at this; I can't test it. Magnus?
>
> * Re: [PATCHES] xpath_array with namespaces support
> /Nikolay Samokhvalov/
>
> I would like to get Peter to deal with this and the other XML-related
> patch.
>
> * Re: [PATCHES] CREATE TABLE LIKE INCLUDING INDEXES support
> /Trevor Hardcastle/
>
> Neil will review this.
>
> * [PORTS] m32r tas() implementation /Martin Pitt/
>
> This lacks a configure probe for sys/tas.h, but I have no other
> objection to it.
>
> * [HACKERS] [PATCH] A crash and subsequent recovery of the master
> can cause theslave to get out-of-sync /Florian G. Pflug/
>
> This seems like a real problem that needs review. The given patch
> was done hastily and might or might not be OK, though.
>
> * [COMMITTERS] pgsql: Remove some of the most blatant brain-fade in
> the recent guc /Tom Lane/
>
> The recent guc patch to reset variables when they are removed from
> postgresql.conf was broken. Again :-(. It needs to be fixed
> or reverted. (Why are we having such problems here? Maybe the
> GUC code as a whole is too complex to understand?)
>
> * [HACKERS] Avoiding unnecessary reads in recovery
> /Heikki Linnakangas/
>
> I think we need to apply this, or else do something else with the
> wired-in assumption about full page writes that was noted in the
> discussion.
>
> * Re: [PATCHES] [HACKERS] autovacuum does not start in HEAD
> /ITAGAKI Takahiro/
>
> I assume Alvaro will deal with this.
>
> * Re: [PATCHES] LIMIT/SORT optimization /Gregory Stark/
> * [PATCHES] updated SORT/LIMIT patch /Gregory Stark/
>
> I will look at this. I recall being concerned about whether there
> wasn't a cleaner way to introduce the required inter-node communication.
>
> * Re: [PATCHES] New version of GENERATED/IDENTITY, was Re: parser
> dilemma /Zoltan Boszormenyi/
>
> I reviewed this earlier and can deal with the update.
>
>
> There is one large item that is not in the patch queue at all, because
> it's not been written, and that's the change to support per-function
> settings of search_path. I think this is critical to get done for 8.3,
> but adding a column to pg_proc will create merge issues for any patch that
> adds anything to pg_proc, and a lot of the open ones do. So I've been
> holding off writing it until more of them are in. I think it's going to
> be straightforward so I'm not too worried about the delay, but it needs to
> happen.
>
> regards, tom lane
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: Have you checked our extensive FAQ?
>
> http://www.postgresql.org/docs/faq

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +