Re: Wrong statistics for size of XLOG_SWITCH during pg_waldump.

From: "movead(dot)li(at)highgo(dot)ca" <movead(dot)li(at)highgo(dot)ca>
To: Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>, andres <andres(at)anarazel(dot)de>
Cc: michael <michael(at)paquier(dot)xyz>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>, ahsan(dot)hadi <ahsan(dot)hadi(at)highgo(dot)ca>
Subject: Re: Wrong statistics for size of XLOG_SWITCH during pg_waldump.
Date: 2020-10-15 04:56:02
Message-ID: 2020101512560020200319@highgo.ca
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Thanks for all the suggestions.

>Yeah. In its current shape, it means that only pg_waldump would be
>able to know this information. If you make this information part of
>xlogdesc.c, any consumer of the WAL record descriptions would be able
>to show this information, so it would provide a consistent output for
>any kind of tools.
I have change the implement, move some code into xlog_desc().

>On top of that, it seems to me that the calculation used in the patch
>is wrong in two aspects at quick glance:
>1) startSegNo and endSegNo point always to two different segments with
>a XLOG_SWITCH record, so you should check that ReadRecPtr is not at a
>segment border instead before extracting SizeOfXLogLongPHD, no?
Yes you are right, and I use 'record->EndRecPtr - 1' instead.

>2) This stuff should also check after the case of a WAL *page* border
>where you'd need to adjust based on SizeOfXLogShortPHD instead.
The 'junk_len -= SizeOfXLogLongPHD' code is considered for when the
remain size of a wal segment can not afford a XLogRecord struct for
XLOG_SWITCH, it needn't care *page* border.

>I'm not sure the exact motive of this proposal, but if we show the
>wasted length in the stats result, I think it should be other than
>existing record types.
Yes agree, and now it looks like below as new patch:

movead(at)movead-PC:/h2/pg/bin$ ./pg_waldump -p ../walbk/ -s 0/3000000 -e 0/6000000 -z
Type N (%) Record size (%) FPI size (%) Combined size (%)
---- - --- ----------- --- -------- --- ------------- ---
XLOG 5 ( 31.25) 300 ( 0.00) 0 ( 0.00) 300 ( 0.00)
XLOGSwitchJunk 3 ( 18.75) 50330512 (100.00) 0 ( 0.00) 50330512 (100.00)

movead(at)movead-PC:/h2/pg/bin$ ./pg_waldump -p ../walbk/ -s 0/3000000 -e 0/6000000 --stat=record
XLOG/SWITCH 3 ( 18.75) 72 ( 0.00) 0 ( 0.00) 72 ( 0.00)
XLOG/SWITCH_JUNK 3 ( 18.75) 50330512 (100.00) 0 ( 0.00) 50330512 (100.00)

The shortcoming now is I do not know how to handle the 'count' of SWITCH_JUNK
in pg_waldump results. Currently I regard SWITCH_JUNK as one count.

>By the way, I noticed that --stats=record shows two lines for
>Transaction/COMMIT. The cause is that XLogDumpCountRecord assumes the
>all four bits of xl_info is used to identify record id.
Thanks,I didn't notice it before, and your patch added into v3 patch attached.

Regards,
Highgo Software (Canada/China/Pakistan)
URL : www.highgo.ca
EMAIL: mailto:movead(dot)li(at)highgo(dot)ca

Attachment Content-Type Size
fix_waldump_size_for_wal_switch_v3.patch application/octet-stream 5.5 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Masahiko Sawada 2020-10-15 05:28:57 Re: Add Information during standby recovery conflicts
Previous Message Greg Nancarrow 2020-10-15 04:25:14 Re: Parallel INSERT (INTO ... SELECT ...)