Optimization for hot standby XLOG_STANDBY_LOCK redo

From: 邱宇航 <iamqyh(at)gmail(dot)com>
To: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Optimization for hot standby XLOG_STANDBY_LOCK redo
Date: 2020-04-30 10:37:27
Message-ID: 2403B554-09D6-4006-9A1F-866E2E6BEF46@gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

I noticed that in hot standby, XLOG_STANDBY_LOCK redo is sometimes block by another query, and all the rest redo is blocked by this lock getting operation, which is not good and often happed in my database, so the hot standby will be left behind and master will store a lot of WAL which can’t be purged.

So here is the idea:
We can do XLOG_STANDBY_LOCK redo asynchronously, and the rest redo will continue.
And I wonder will LogStandbySnapshot influence the consistency in hot standby, for the redo is not by order. And how to avoid this.

// ------ startup ------
StartupXLOG()
{
while (readRecord())
{
check_lock_get_state();
if (record.tx is in pending tbl):
append this record to the pending lock for further redo.
redo_record();
}
}

check_lock_get_state()
{
for (tx in pending_tx):
if (tx.all_lock are got):
redo the rest record for this tx
free this tx
}

standby_redo
{
if (XLOG_STANDBY_LOCK redo falied)
{
add_lock_to_pending_tx_tbl();
}
}

// ------ worker process ------
main()
{
while(true)
{
for (lock in pending locks order by lsn)
try_to_get_lock_from_pending_tbl();
}
}

regards.
Yuhang

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Kapila 2020-04-30 10:45:13 Re: WIP/PoC for parallel backup
Previous Message Sumanta Mukherjee 2020-04-30 09:18:23 Re: WIP/PoC for parallel backup