From: | "matsumura(dot)ryo(at)fujitsu(dot)com" <matsumura(dot)ryo(at)fujitsu(dot)com> |
---|---|
To: | 'Kyotaro Horiguchi' <horikyota(dot)ntt(at)gmail(dot)com> |
Cc: | "bossartn(at)amazon(dot)com" <bossartn(at)amazon(dot)com>, "masao(dot)fujii(at)gmail(dot)com" <masao(dot)fujii(at)gmail(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org> |
Subject: | RE: archive status ".ready" files may be created too early |
Date: | 2020-07-06 04:02:23 |
Message-ID: | OSAPR01MB5027F9C2007CC45A9EA65CF8E8690@OSAPR01MB5027.jpnprd01.prod.outlook.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Hello, Horiguchi-san
Thank you for your comment and patch.
At Thursday, June 25, 2020 3:36 PM(JST), "Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>" wrote in
> I think we don't need most of that shmem stuff. XLogWrite is called
I wanted no more shmem stuff too, but other ideas need more lock
that excludes inserter and writer each other.
> after WAL buffer is filled up to the requested position. So when it
> crosses segment boundary we know the all past corss segment-boundary
> records are stable. That means all we need to remember is only the
> position of the latest corss-boundary record.
I could not agree. In the following case, it may not work well.
- record-A and record-B (record-B is a newer one) is copied, and
- lastSegContRecStart/End points to record-B's, and
- FlushPtr is proceeded to in the middle of record-A.
In the above case, the writer should notify segments before record-A,
but it notifies ones before record-B. If the writer notifies
only when it flushes the latest record completely, it works well.
But the writer may not be enable to notify any segment forever when
WAL records crossing-segment-boundary are inserted contiunuously.
So I think that we must remeber all such cross-segement-boundary records's EndRecPtr in buffer.
> If we call XLogMarkEndRecPtrIfNeeded() there, the function is called
> every time a record is written, most of which are wasteful.
> XLogInsertRecord already has a code block executed only at every page
> boundary.
I agree.
XLogMarkEndRecPtrIfNeeded() is moved into the code block before updating
LogwrtRqst.Write for avoiding passing-each-other with writer.
> Now we can identify stable portion of WAL stream. It's enough to
> prevent walsender from sending data that can be overwritten
> afterwards. GetReplicationTargetRecPtr() in the attached does that.
I didn't notice it.
I agree basically, but it is based on lastSegContRecStart/End.
So, first of all, we have to agree what should be remebered.
Regards
Ryo Matsumura
From | Date | Subject | |
---|---|---|---|
Next Message | Michael Paquier | 2020-07-06 04:11:33 | Re: Cache lookup errors with functions manipulation object addresses |
Previous Message | Amul Sul | 2020-07-06 03:40:26 | Re: Cleanup - adjust the code crossing 80-column window limit |