Re: logical replication - still unstable after all these months

On 2017-05-27 01:35, Mark Kirkwood wrote:
> On 26/05/17 20:09, Erik Rijkers wrote:
>
>>
>> The idea is simple enough:
>>
>> startup instance1
>> startup instance2 (on same machine)
>> primary: init pgbench tables
>> primary: add primary key to pgbench_history
>> copy empty tables to replica by dump/restore
>> primary: start publication
>> replica: start subscription
>> primary: run 1-minute pgbench
>> wait till the 4 md5's of primary pgbench tables
>> are the same as the 4 md5's of replica pgbench
>> tables (this will need a time-out).
>> log 'ok' or 'not ok'
>> primary: clean up publication
>> replica: clean up subscription
>> shutdown primary
>> shutdown replica
>>
>> this whole thing 100x

Here is what I have:

instances.sh:
starts up 2 assert enabled sessions

instances_fast.sh:
alternative to instances.sh
starts up 2 assert disabled 'fast' sessions

testset.sh
loop to call pgbench_derail2.sh with varying params

pgbench_derail2.sh
main test program
can be called 'standalone'
./pgbench_derail2.sh $scale $clients $duration $date_str
so for instance this should work:
./pgbench_derail2.sh 25 64 60 20170527_1019
to remove publication and subscription from sessions, add a 5th
parameter 'clean'
./pgbench_derail2.sh 1 1 1 1 'clean'

pubsub.sh
displays replication state. also called by pgbench_derail2.sh
must be in path

result.sh
display results
I keep this in a screen-session as:
watch -n 20 './result.sh 201705'

Peculiar to my setup also:
server version at compile time stamped with date + commit hash
I misuse information_schema.sql_packages at compile time to store
patch information
instances are in $pg_stuff_dir/pg_installations/pgsql.<project name>

So you'll have to outcomment a line here and there, and adapt paths,
ports, and things like that.

It's a bit messy, I should have used perl from the beginning...

Good luck :)

Erik Rijkers