Undo logs

From: Thomas Munro <thomas(dot)munro(at)enterprisedb(dot)com>
To: Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Undo logs
Date: 2018-05-24 22:22:11
Message-ID: CAEepm=2EqROYJ_xYz4v5kfr4b0qw_Lq_6Pe8RTEC8rx3upWsSQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox
Lists: pgsql-hackers

Hello hackers,

As announced elsewhere[1][2][3], at EnterpriseDB we are working on a
proposal to add in-place updates with undo logs to PostgreSQL. The
goal is to improve performance and resource usage by recycling space

The lowest level piece of this work is a physical undo log manager,
which I've personally been working on. Later patches will build on
top, adding record-oriented access and then the main "zheap" access
manager and related infrastructure. My colleagues will write about

The README files[4][5] explain in more detail, but here is a
bullet-point description of what the attached patch set gives you:

1. Efficient appending of new undo data from many concurrent
backends. Like logs.
2. Efficient discarding of old undo data that isn't needed anymore.
Like queues.
3. Efficient buffered random reading of undo data. Like relations.

A test module is provided that can be used to exercise the undo log
code paths without needing any of the later zheap patches.

This is work in progress. A few aspects are under active development
and liable to change, as indicated by comments, and there are no doubt
bugs and room for improvement. The code is also available at
github.com/EnterpriseDB/zheap (these patches are from the
undo-log-storage branch, see also the master branch which has the full
zheap feature). We'd be grateful for any questions, feedback or

[1] https://amitkapila16.blogspot.com/2018/03/zheap-storage-engine-to-provide-better.html
[2] https://rhaas.blogspot.com/2018/01/do-or-undo-there-is-no-vacuum.html
[3] https://www.pgcon.org/2018/schedule/events/1190.en.html
[4] https://github.com/EnterpriseDB/zheap/tree/undo-log-storage/src/backend/access/undo
[5] https://github.com/EnterpriseDB/zheap/tree/undo-log-storage/src/backend/storage/smgr

Thomas Munro

Attachment Content-Type Size
0001-Add-undo-log-manager-v1.patch application/octet-stream 108.8 KB
0002-Provide-access-to-undo-log-data-via-the-buffer-ma-v1.patch application/octet-stream 37.5 KB
0003-Add-developer-documentation-for-the-undo-log-stor-v1.patch application/octet-stream 12.0 KB
0004-Add-tests-for-the-undo-log-manager-v1.patch application/octet-stream 41.4 KB
0005-Add-user-facing-documentation-for-undo-logs-v1.patch application/octet-stream 10.3 KB


Browse pgsql-hackers by date

  From Date Subject
Next Message Mike Palmiotto 2018-05-24 23:10:23 Re: contrib/sepgsql fails on Fedora 28
Previous Message Peter Geoghegan 2018-05-24 22:21:10 Re: Compiler warnings with --enable-dtrace