Re: POC: Cleaning up orphaned files using undo logs

From: Dilip Kumar <dilipbalaut(at)gmail(dot)com>
To: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, Dmitry Dolgov <9erthalion6(at)gmail(dot)com>, Kuntal Ghosh <kuntalghosh(dot)2007(at)gmail(dot)com>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: POC: Cleaning up orphaned files using undo logs
Date: 2019-07-09 10:28:22
Message-ID: CAFiTN-uM5CUS2xvDYcrv4xfUrd3hcGWym4PtrrEQvX0yd5cH2A@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Sat, Jul 6, 2019 at 8:26 PM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
>
> On Thu, Jul 4, 2019 at 5:24 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >
> PFA, the latest version of the undo interface and undo processing patches.
>
> Summary of the changes in the patch set
>
> 1. Undo Interface
> - Rebased over latest undo storage code
> - Implemented undo page compression (don't store the common fields in
> all the records instead we get from the first complete record of the
> page).
> - As per Robert's comment, UnpackedUndoRecord is divided in two parts,
> a) All fields which are set by the caller.
> b) Pointer to structures which are set internally.
> - Epoch and the Transaction id are unified as full transaction id
> - Fixed handling of dbid during recovery (TODO in PrepareUndoInsert)
>
> Pending:
> - Move loop in UndoFetchRecord to outside and test performance with
> keeping pin vs pin+lock across undo records. This will be done after
> testing performance over the zheap code.
> - I need to investigate whether Discard checking can be unified in
> master and HotStandby in UndoFetchRecord function.
>
> 2. Undo Processing
> - Defect fix in multi-log rollback for subtransaction.
> - Assorted defect fixes.
>
> Others
> - Fixup for undo log code to handle full transaction id in
> UndoLogSlot for discard and other bug fixes in undo log.
> - Fixup for Orphan file cleanup to pass dbid in PrepareUndoInsert
>
PFA, updated patch version which includes
- One defect fix in undo interface related to undo page compression
for handling persistence level
- Implemented pending TODO optimization in undo page compression.
- One defect fix in undo processing related to the prepared transaction

--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com

Attachment Content-Type Size
undo_20190709.tar.gz application/x-gzip 90.7 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Thomas Munro 2019-07-09 11:03:18 Re: Introduce timeout capability for ConditionVariableSleep
Previous Message Antonin Houska 2019-07-09 10:15:37 Re: Attempt to consolidate reading of XLOG page