[BUG] postgresql replaying WAL slowly when master drop a large number of relations in one transaction

From: 158306855 <anderson2013(at)qq(dot)com>
To: pgsql-bugs <pgsql-bugs(at)postgresql(dot)org>
Subject: [BUG] postgresql replaying WAL slowly when master drop a large number of relations in one transaction
Date: 2018-05-30 02:38:01
Message-ID: tencent_2335C979BB4694EEB29EA98ECD4E7A4FB907@qq.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

HI pg team:

I found a point that could be improved that is postgresql replaying WAL slowly when master drop a large number of tables in one transaction.

1. When postgresql drop some relation, replication replay it in function xact_redo_commit
2. We can see that every relation run function smgrdounlink one by one, this looks ok, code like this:
for (i = 0; i < parsed->nrels; i++)
{
SMgrRelation srel = smgropen(parsed->xnodes[i], InvalidBackendId);
ForkNumber fork;

for (fork = 0; fork <= MAX_FORKNUM; fork++)
XLogDropRelation(parsed->xnodes[i], fork);
smgrdounlink(srel, true);
smgrclose(srel);
}

3 But, the problem drop relation buffers by function DropRelFileNodesAllBuffers, that call by smgrdounlink. Each relation search the entire shared_buffer. This is unnecessary and can be optimized.
DropRelFileNodesAllBuffers code:
for (i = 0; i < NBuffers; i++)
{

....
}

4 When master drop a large number of relations in one transaction, replaying WAL slowly.
Replay of this record on the replication node consumes CPU and execution time much longer than the master node.

5 The function xact_redo_abort also has this problem.

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message PG Bug reporting form 2018-05-30 04:02:50 BUG #15217: Valgrind - writing uninitialised byte(s) in FileWrite
Previous Message PG Bug reporting form 2018-05-29 20:46:53 BUG #15216: for test new Web ERP/CRM