Re: Replication

From: Mike Christensen <mike(at)kitchenpc(dot)com>
To: Arndt Lehmann <arndt(dot)lehmann(at)gmail(dot)com>
Cc: pgsql-general(at)postgresql(dot)org
Subject: Re: Replication
Date: 2009-06-23 17:37:34
Message-ID: 7aa638e00906231037v64f8a296pdfbcc2b3bc95619e@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

>
> Hi Mike
>
> thanks for your interest in rubyrep. I developed rubyrep. Let me
> answer your questions.
>
> On Jun 23, 4:16 pm, m(dot)(dot)(dot)(at)kitchenpc(dot)com (Mike Christensen) wrote:
> > There will be a set of triggers for each replication. Since MySql
> doesn't
> > support more than one trigger on a table, this approach won't work which
> I
> > guess is their way of saying "We're database independent, as long as you
> use
> > either Postgres or MySql oh and btw we have no replication story above 2
> > nodes on MySQL"
> The first statement on the rubyrep project website says that rubyrep
> provides database independent - currently supporting PostgreSQL and
> MySQL - master-master replication [i. e. 2 databases].
> That statement is 100% correct.
> Regarding multi-master replication (i. e. more than 2 databases) the
> according sub page says that it only works for PostgreSQL.
> I intended both statements to be very clear and *not* misleading. If
> you think they are not, I am interested in hearing your improvement
> suggestions.
>
>
Hi Arndt - Sorry, my email was just being a little on the sarcastic side. I
don't think your statements are misleading, just a bit marketed. To me,
something "database independent" should be able to be configured to work on
any DB, not just two. And the fact that there's a pretty big limitation in
one of those databases really limits your story for MySQL. I can't say I'm
too concerned, as like many others on this thread I'm not the biggest fan of
MySQL (I tried it for about a day and kept saying "Wait, it can't do
/that/?" too many times so I gave up)

I think unless you want to either develop a full replication story for MySQL
or add support for other software such as IBM and Oracle, you might just
want to focus all your energy on being the best Postgres replication story
around and market as being just that. It would definitely keep the support
costs down having to deal with random bugs on mySQL and trying to please
everybody.

>
> >
> > Also, if database C goes down, then everything goes kaboom, right? Even
> if
> > you did A replicates with B, B replicates with C, if one database goes
> down
> > your chain is broken. I'm worried about this scenario,
> Assuming that A replicates with C and C with B then once C goes down,
> indeed replication will stop.
> However nothing goes "kaboom". All changes in either A or B are still
> tracked in the according queue tables. As soon as C comes up again,
> all pending changes will be replicated.
> If let's say C goes down, then replication between A and B will not be
> affected.
> Same if A goes down: replication between B and C will continue.
> >and any perf
> > implications with having a whole bunch of triggers on a table. Maybe
> > someone can comment.
> The triggers are designed to be as "lean" as possible. Actually all
> they do is to add the primary key of a created / modified / deleted
> row into the queue table. I don't think that this will cause any
> performance impact.
> (The actual work is done by the rubyrep process which applies all
> changes. That process can be run on a totally different server to
> avoid impacting the database server performance.)
>
>
Thanks for clarifying. After sending the email I put some thought into what
a true fault tolerant replication story would entale and it seems like it'd
be a complete nightmare to design. Does anyone actually have that (any node
can go down and the others still replicate amongst themselves?)

On my system, if the master DB goes down and some data is out of date by a
few minutes or an hour or so shouldn't cause too many problems so long as
"half a transaction" isn't replicated or anything silly like that. The only
scenario I could come up with that would be annoying is if I user created a
new account on the system, then switched over to another webserver/DB node
and all of a sudden it said their user account didn't exist.

The "left/right" database idea I can see is a very straight forward
architecture and easy to setup and maintain, I'm just worried if you had a
/huge/ cluster (Like Facebook size) then it really puts a massive strain on
your master DB, since with every piece of data that changes, keys have to be
inserted in dozens of tables. If seems if you could optimize this
redundancy and have all the triggers using the same table, it might scale
better. However, I haven't done any conclusive studies to back this
statement up :)

Keep up the good work! One of these days I have to start looking into
replication stories before my site goes live, and rubyrep will for sure be
on the list.

Mike

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Tom Lane 2009-06-23 17:43:31 Re: possible bug on age() function (8.2.4 , 8.3.6)
Previous Message Merlin Moncure 2009-06-23 17:31:59 Re: Replication