Re: Documentation improvements for partitioning

On 24 February 2017 at 02:36, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:

>> While its true that the patch had syntax documentation, there was no
>> user design documentation which explained how it worked to allow
>> objective review. Had I been able to provide input without reading
>> every email message, I would have done so earlier.
>
> But I don't agree that it was impossible for you to provide input
> earlier without reading every email message, nor do I agree that it is
> unreasonable to expect to people who want to provide input to read the
> relevant threads.

>> The features I consider very important in the first release are
>> 1. Declarative syntax (we have this!)
>> 2. Tuple routing on INSERT/COPY (we have this!)
>> 3. O(1) partition elimination for simple = queries
>> 4. PRIMARY KEY defined using a) distinct set of UNIQUE constraints on
>> each partition, b) partition key
>>
>> 2 and 3 are intimately connected because they would both use the same
>> in-memory data for bsearch, so the code should be almost identical.
>>
>> 4 is important for Foreign Keys and Logical Replication
>>
>> As missing items, features 3 and 4 seem achievable in this release,
>> potentially in restricted form.
>
> Simon, this is ridiculous. We're 4 or 5 days away from the start of
> the last CommitFest. We have time to fix bugs and improve
> documentation and maybe tweak things here and there, but 3 and 4 are
> significant development projects. There isn't time to do that stuff
> right now and get it right.

Agreed, you beat me to it. I've spent the last few hours trying to see
what I could personally pull out of the fire to assist. Those things
aren't that big, but they are bigger than we have time for now. Had I
known about that a few months back... well, we are where we are.

The good news is that logical replication DOES work with partitioning,
but only for a Publication with PUBLISH INSERT, pushing from a normal
table to a partitioned one. Which is useful enough for first release.

The work on having UPDATE work between partitions can be used to make
updates and deletes work in logical replication. That might yet be
made to work in this release, and I think we should look into it, but
I think it can be left til next release if we try.

> You don't get to show up more than two
> months after the feature is committed and start complaining that it
> doesn't include all the things you want. Which things ought to be
> included in the initial patch was under active discussion about a year
> ago this time.

I've provided lots of input across many years, for example my
explanation of how to do tuple routing is very clearly the basis of
the design of the current feature. You *knew* I would have feedback
and if it it hadn't appeared yet you knew it would come.

The "two months after the feature is committed" is a direct result of
the lack of docs. Note that within hours of reading the docs I'd given
feedback. How could I give feedback earlier? Well, I tried. I've flown
to Japan to ensure I could talk to Amit in person about this feature.
Your absence at two consecutive developer meetings hasn't helped
there. It is you that needs to show up more. I'm sure we both have
family and work reasons not to attend them.

The need for design feedback is exactly why the docs for logical
replication were published in June, six months before logical
replication was committed.

With repect, you are making a few mistakes. The first is to imagine
that review comments are negative or complaints; with the right
viewpoint they could easily be seen as helping people to see what has
been missed, yet you repeatedly see them as personal attacks and throw
words back. Oh sure, I've done that myself in earlier days. The second
is to imagine that discussing things on Hackers in multiple threads,
spanning many months with long, detailed emails and rapid replies is
something that anybody could have followed if they wished. And the
third is to imagine I have no right to review; I will watch and see if
you deploy this same "You don't get to show up.." argument against Tom
or Noah when they point out holes in late reviews, though we already
both know you won't. I see you using that yourself, objecting
frequently against patches large and small if they do not meet your
exacting standards, yet you have spoken multiple times against my
right to do that.

Perhaps we'll have time to scowl at each other in India. I'll look
forward to that. ;-)

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services