Re: Support for N synchronous standby servers - take 2

From: "Amir Rohan" <amir(dot)rohan(at)mail(dot)com>
To: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Support for N synchronous standby servers - take 2
Date: 2015-09-23 04:11:15
Message-ID: trinity-4258acb0-b75a-4bf1-9657-f8237883cd0d-1442981475318@3capp-mailcom-lxa02
Views: Raw Message | Whole Thread | Download mbox
Lists: pgsql-hackers

<html><head></head><body><div style="font-family: Verdana;font-size: 12.0px;"><div>
<div>&gt;On 07/16/15, Robert Haas wrote:<br/>
&gt;&nbsp;&nbsp; &nbsp;<br/>
&gt;&gt;&gt; * Developers will immediately understand the format<br/>
&gt;&gt;I doubt it.&nbsp; I think any format that we pick will have to be carefully<br/>
&gt;&gt;documented.&nbsp; People may know what JSON looks like in general, but they<br/>
&gt;&gt;will not immediately know what bells and whistles are available in<br/>
&gt;&gt;this context.<br/>
&gt;&gt;&gt; * Easy to programmatically manipulate in a range of languages<br/>
&gt;&gt; &lt;...&gt; I think it will be rare to need to parse the postgresql.conf string,<br/>
&gt;&gt; manipulate it programatically, and then put it back.<br/>
&gt;On Sun, Jul 19, 2015 at 4:16 PM, Tom Lane &lt;tgl(at)sss(dot)pgh(dot)pa(dot)us&gt; wrote:<br/>
&gt;&gt; Josh Berkus &lt;josh(at)agliodbs(dot)com&gt; writes:<br/>
&gt;&gt;&gt; On 07/17/2015 04:36 PM, Jim Nasby wrote:<br/>
&gt;&gt;&gt;&gt; I&#39;m guessing it&#39;d be really ugly/hard to support at least this GUC being<br/>
&gt;&gt;&gt;&gt; multi-line?<br/>
&gt;&gt;&gt; Mind you, multi-line GUCs would be useful otherwise, but we don&#39;t want<br/>
&gt;&gt;&gt; to hinge this feature on making that work.<br/>
&gt;&gt; Do we really want such a global reduction in friendliness to make this<br/>
&gt;&gt; feature easier?<br/>
&gt;Maybe shoehorning this into the GUC mechanism is the wrong thing, and<br/>
&gt;what we really need is a new config file for this.&nbsp; The information<br/>
&gt;we&#39;re proposing to store seems complex enough to justify that.</div>


<div>It seems like:</div>

<div>1) There&#39;s a need to support structured data in configuration for future</div>

<div>needs as well, it isn&#39;t specific to this feature.<br/>
2) There should/must be a better way to validate configuration then<br/>
to restarting the server in search of syntax errors.</div>


<div>Creating a whole new configuration file for just one feature *and* in a different
<div>format seems suboptimal.&nbsp; What happens when the next 20 features need structured</div>

<div>config data, where does that go? will there be additional JSON config files *and* perhaps</div>

<div>new mini-language values in .conf as development continues?&nbsp; How many dedicated</div>

<div>configuration files is too many?</div>

<div>Now, about JSON.... (Earlier Upthread):</div>

On 07/01/15, Peter Eisentraut wrote:</div>

<div>&gt; On 6/26/15 2:53 PM, Josh Berkus wrote:<br/>
&gt; &gt; I would also suggest that if I lose this battle and<br/>
&gt; &gt; we decide to go with a single stringy GUC, that we at least use JSON<br/>
&gt; &gt; instead of defining our out, proprietary, syntax?<br/>
&gt; &nbsp;<br/>
&gt; Does JSON have a natural syntax for a set without order?</div>


<div>No. Nor Timestamps. It doesn&#39;t even distingush integer from float</div>

<div>(Though parsers do it for you in dynamic languages). It&#39;s all because</div>

<div>of its unsightly javascript roots.<br/>

<div>The current patch is now forced by JSON to conflate sets and lists, so</div>

<div>un/ordered semantics are no longer tied to type but to the specific configuration keys.</div>

<div>So, If a feature ever needs a key where the difference between set and list matters</div>

<div>and needs to support both, you&#39;ll need seperate keys (both with lists, but meaning different things)</div>

<div>or a separate &quot;mode&quot; key or something. Not terrible, just iffy.</div>


<div>Other have found JSON unsatisfactory before. For example, the clojure community</div>

<div>has made (at least) two attempts at alternatives, complete with the meh adoption</div>

<div>rates you&#39;d expect despite being more capable formats:</div>




<div>There&#39;s also YAML, TOML, etc&#39;, none as universal as JSON. But to reiterate, JSON itself</div>

<div>has Lackluster type support (no sets, no timestamps), is verbose, iseasy to malform when editing</div>

<div>(missed a curly brace, shouldn&#39;t use a single quote), isn&#39;t extensible, and my personal pet peeve</div>

<div>is that it doesn&#39;t allow non-string or bare-string keys in maps (a.k.a &quot;death by double-quotes&quot;).</div>

<div>Python has the very natural {1,2,3} syntax for sets, but of course that&#39;s not part of JSON.</div>


<div>If&nbsp; JSON wins out despite all this, one alternative not discussed is to extend</div>

<div>the .conf parser to accept json dicts as a fundamental type. e.g.:</div>



<div>data_directory = &#39;ConfigDir&#39;&nbsp; &nbsp;<br/>
port = 5432<br/>
work_mem = 4MB<br/>
hot_standby = off<br/>
client_min_messages = notice<br/>
log_error_verbosity = default<br/>
autovacuum_analyze_scale_factor = 0.1<br/>
synch_standby_config = {<br/>
&nbsp; &quot;sync_info&quot;: {<br/>
&nbsp;&nbsp;&nbsp; &quot;nodes&quot;: [<br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; {<br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &quot;priority&quot;: 1,<br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &quot;group&quot;: &quot;cluster1&quot;<br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; },<br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &quot;A&quot;<br/>
&nbsp;&nbsp;&nbsp; ],<br/>
&nbsp;&nbsp;&nbsp; &quot;quorum&quot;: 3<br/>
&nbsp; },<br/>
&nbsp; &quot;groups&quot;: {<br/>
&nbsp;&nbsp;&nbsp; &quot;cluster1&quot;: [<br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &quot;B&quot;,<br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &quot;C&quot;<br/>
&nbsp;&nbsp;&nbsp; ]<br/>
&nbsp; }<br/>


<div>This *will* break someone&#39;s perl I would guess. Ironically, those scripts wouldn&#39;t have broken if</div>

<div>some structured format were in use for the configuration data when they were written...</div>

<div>&#96;postgres --describe-config&#96; is also pretty much tied to a line-oriented configuration.</div>






<div>MIA configuration validation tool/switch should probably get a thread too.</div>


Attachment Content-Type Size
unknown_filename text/html 6.3 KB


Browse pgsql-hackers by date

  From Date Subject
Next Message David Rowley 2015-09-23 05:11:35 Re: PATCH: use foreign keys to improve join estimates v1
Previous Message Charles Clavadetscher 2015-09-23 02:36:05 Re: unclear about row-level security USING vs. CHECK