Re: User defined data types in Logical Replication

At Mon, 19 Mar 2018 12:50:55 +0900, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote in <CAD21AoD=kiDVxfZ2aT_Oeg7+5etkxg0eqmsRE-gcUbptKNir6g(at)mail(dot)gmail(dot)com>
> On Fri, Mar 16, 2018 at 10:24 AM, Alvaro Herrera
> <alvherre(at)alvh(dot)no-ip(dot)org> wrote:
> > Masahiko Sawada wrote:
> >> On Thu, Mar 15, 2018 at 9:41 AM, Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org> wrote:
> >
> >> > I think this is a worthwhile test, but IMO it should be improved a bit
> >> > before we include it. Also, we can come up with a better name for the
> >> > test surely, not just refer to this particular bug. Maybe "typemap".
> >>
> >> It might be useful if we have the facility of TAP test to check the
> >> log message and regexp-match the message to the expected string.
> >
> > Something similar to PostgresNode::issues_sql_like() perhaps?
> >
>
> Yeah, I didn't know that but I think it's a good idea. Unlike
> issues_sql_like() we don't issue anything to the subscriber. So maybe
> we need truncate logfile before insertion and verify logfile of
> particular period. The test code would be like follows.
>
> $node_subscriber->safe_psql('postgres', 'CREATE SUBSCRIPTION...");
> truncate $node_subscriber->logfile, 0;
> $node_publisher->safe_psql('postgres', 'INSERT .. ')
> my $log = TestLib::slurp_file($node_subscriber->logfile);
>
> # Verify logs
> like($log, qr/processing remote data for replication target relation
> "public.test" column "b", remote type dummyint, local type dummyint/,
> 'callback function of datatype conversion1');
> like($log, qr/processing remote data for replication target relation
> "public.test" column "a", remote type dummytext, local type
> dummytext/, 'callback function of datatype conversion2');
>
> Thoughts?

FWIW something like is in a currently proposed patch.

https://www.postgresql.org/message-id/20180129.194023.228030941.horiguchi.kyotaro@lab.ntt.co.jp

0003 contains the following function.

+# find $pat in logfile of $node after $off-th byte
+sub find_in_log
+{
+ my ($node, $pat, $off) = @_;
+
+ $off = 0 unless defined $off;
+ my $log = TestLib::slurp_file($node->logfile);
+ return 0 if (length($log) <= $off);
+
+ $log = substr($log, $off);
+
+ return $log =~ m/$pat/;
+}

It is used as the follows.

+$logstart = get_log_size($node_standby);
+$node_standby->start;
+
+my $failed = 0;
+for (my $i = 0 ; $i < 10000 ; $i++)
+{
+ if (find_in_log($node_standby,
+ "requested WAL segment [0-9A-F]+ has already been removed",
+ $logstart))
+ {
+ $failed = 1;
+ last;
+ }
+ usleep(100_000);
+}
+ok($failed, 'check replication has been broken');

regards,

--
Kyotaro Horiguchi
NTT Open Source Software Center