Re: COPY enhancements

On Wed, 7 Oct 2009, Emmanuel Cecchet wrote:

> I think there is a misunderstanding about what the current patch is
> about...the patch does NOT include logging errors into a file (a feature
> we can add later on (next commit fest?))

I understand that (as one of the few people who has read the patch it
seems), but as you can might guess from the feedback here that's not the
way I expect your patch is actually going to be handled. Something that
logs only to a table without the interface for the file output at least
specified isn't the sort of thing that this group tends to go along with
very well. Since as it is we haven't even nailed down how the file output
is even going to look or work yet, the table logging isn't very likely to
go in unless it's at least clear that the future plans there will cleanly
stack on top. That's what people are alluding to mentioning the whole
roadmap concept for all this.

I get the impression you were hoping to get another chunk of this commited
on this round. I would guess that realistically it's going to take at
least a few more weeks for the rest of this to get nailed down, and that a
really solid feature should be ready by the next CF instead. You should
actually be proud of the progress you spurred on the COPY options stuff
that's in there now, that idea has been floating around for a while now
but until your patch there wasn't any momentum behind doing something
about it. The problem you're facing now is that so many people have been
thinking about this particular area for so long without any code to talk
about that you're now stuck with all that pent up uncoded design to clear.

> I can put the auto-partitioning in a separate patch if that helps but this
> patch relies on error logging to log possible errors in the routing of
> tuples.

Understood. I know you've gotten some other coding feedback here, and
you've been very good about taking that and producing updated versions
which is appreciated. I would recommend that the next time you resubmit
this, if you could break the logging and partitioning patches into a pair
that depend on one another, that would make life easier for everyone else
(albeit probably a harder for you). At that point we should be set to
have some others take over some of that tweaking, with myself, Selena, and
Jeff all interested and capable of helping out here.

> If you prefer to postpone the auto-partitioning to the next commit fest,
> I can strip it from the current patch and re-submit it for the next fest
> (but it's just 2 isolated methods really easy to review).

That was one of points I was trying to make--that would be an easier to
review by itself, but as it is people interested in it can't consider it
without also staring at the logging stuff. And people who are focusing on
the logging bits find it distracting, so nobody is really happy with the
current combined patch.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD