Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

Hi Bharath,

Due to recent commits on master, the pg_walinpect module is not
compiling. Kindly update the patch.

pg_walinspect.c: In function ‘GetXLogRecordInfo’:
pg_walinspect.c:362:39: error: ‘XLogReaderState’ {aka ‘struct
XLogReaderState’} has no member named ‘max_block_id’
362 | for (block_id = 0; block_id <= record->max_block_id; block_id++)
| ^~
pg_walinspect.c:382:29: error: ‘XLogReaderState’ {aka ‘struct
XLogReaderState’} has no member named ‘blocks’
382 | uint8 bimg_info = record->blocks[block_id].bimg_info;
| ^~
pg_walinspect.c:385:21: error: ‘XLogReaderState’ {aka ‘struct
XLogReaderState’} has no member named ‘blocks’
385 | fpi_len += record->blocks[block_id].bimg_len;
| ^~
pg_walinspect.c:402:16: error: ‘XLogReaderState’ {aka ‘struct
XLogReaderState’} has no member named ‘blocks’
402 | record->blocks[block_id].hole_offset,
| ^~
pg_walinspect.c:403:16: error: ‘XLogReaderState’ {aka ‘struct
XLogReaderState’} has no member named ‘blocks’
403 | record->blocks[block_id].hole_length,
| ^~
pg_walinspect.c:405:16: error: ‘XLogReaderState’ {aka ‘struct
XLogReaderState’} has no member named ‘blocks’
405 | record->blocks[block_id].hole_length -
| ^~
pg_walinspect.c:406:16: error: ‘XLogReaderState’ {aka ‘struct
XLogReaderState’} has no member named ‘blocks’
406 | record->blocks[block_id].bimg_len,
| ^~
pg_walinspect.c:414:16: error: ‘XLogReaderState’ {aka ‘struct
XLogReaderState’} has no member named ‘blocks’
414 | record->blocks[block_id].hole_offset,
| ^~
pg_walinspect.c:415:16: error: ‘XLogReaderState’ {aka ‘struct
XLogReaderState’} has no member named ‘blocks’
415 | record->blocks[block_id].hole_length);
| ^~
make: *** [../../src/Makefile.global:941: pg_walinspect.o] Error 1

Thanks & Regards,
Nitin Jadhav

On Thu, Mar 17, 2022 at 1:54 PM Bharath Rupireddy
<bharath(dot)rupireddyforpostgres(at)gmail(dot)com> wrote:
>
> On Thu, Mar 17, 2022 at 10:48 AM Kyotaro Horiguchi
> <horikyota(dot)ntt(at)gmail(dot)com> wrote:
> >
> > It still suggests unspecifiable end-LSN..
> >
> > > select * from pg_get_wal_records_info('4/4B28EB68', '4/4C000060');
> > > ERROR: cannot accept future end LSN
> > > DETAIL: Last known WAL LSN on the database system is 4/4C000060.
>
> Thanks Kyotaro-san. We can change the detail message to show (current
> flush lsn/last replayed lsn - 1), that's what I've done in v11 posted
> upthread at [1]. The problem is that all the pg_walinspect functions
> would wait for the first valid record in read_local_xlog_page() via
> InitXLogReaderState()->XLogFindNextRecord(), see[2].
>
> We have two things to do:
> 1) Just document the behaviour "pg_walinspect functions will wait for
> the first valid WAL record if there is none found after the specified
> input LSN/start LSN.". This seems easier but some may see it as a
> problem.
> 2) Have read_local_xlog_page_2 which doesn't wait for future WAL LSN
> unlike read_local_xlog_page and like pg_waldump's WALDumpReadPage. It
> requires a new function read_local_xlog_page_2 that almost looks like
> read_local_xlog_page except wait (pg_usleep) loop, we can avoid code
> duplication by moving the read_local_xlog_page code to a static
> function read_local_xlog_page_guts(existing params, bool wait):
>
> read_local_xlog_page(params)
> read_local_xlog_page_guts(existing params, false);
>
> read_local_xlog_page_2(params)
> read_local_xlog_page_guts(existing params, true);
>
> read_local_xlog_page_guts:
> if (wait) wait for future wal; ---> existing pg_usleep code in
> read_local_xlog_page.
> else return;
>
> I'm fine either way, please let me know your thoughts on this?
>
> [1] https://www.postgresql.org/message-id/CALj2ACU8XjbYbMwh5x6hEUJdpRoG9%3DPO52_tuOSf1%3DMO7WtsmQ%40mail.gmail.com
> [2]
> postgres=# select pg_current_wal_flush_lsn();
> pg_current_wal_flush_lsn
> --------------------------
> 0/1624430
> (1 row)
>
> postgres=# select * from pg_get_wal_record_info('0/1624430');
> ERROR: cannot accept future input LSN
> DETAIL: Last known WAL LSN on the database system is 0/162442F.
> postgres=# select * from pg_get_wal_record_info('0/162442f'); --->
> waits for the first valid record in read_local_xlog_page.
>
> Regards,
> Bharath Rupireddy.
>
>