Database Lab 0.2.0: fast cloning of large PostgreSQL databases to build non-production environments

From: Nikolay Samokhvalov <samokhvalov(at)gmail(dot)com>
To: pgsql-announce(at)postgresql(dot)org
Subject: Database Lab 0.2.0: fast cloning of large PostgreSQL databases to build non-production environments
Date: 2020-02-06 22:31:39
Message-ID: CANNMO+LtNdi3E-g+dHZYcy_6_XhgO7DeM63B8WSfAO9axaaMAA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-announce

Database Lab 0.2.0, a tool for fast cloning of large PostgreSQL databases
to build
non-production environments, released:
https://gitlab.com/postgres-ai/database-lab

Database Lab speeds up software development cycles via enabling "thin
provisioning"
of temporary full-size Postgres databases. It is helpful in various cases:

1) Various testing, including using full-size databases in CI tests.

2) Verification of DB schema changes (database migrations), detecting
potentially
dangerous changes, before they reach production.

3) SQL performance troubleshooting: a database clone provided by Database
Lab
has execution plans identical to the original database, therefore "EXPLAIN
(BUFFERS,
ANALYZE") will show the same plan structure and the same amount of data
involved.
Verifying various SQL optimization ideas becomes simple (see also: Joe bot,
working
on top of Database Lab https://gitlab.com/postgres-ai/joe/).

Characteristics and features:
- Open Source, writing in Go.
- All components are running in Docker containers. Custom Postgres
containers are
supported (extensions, additional tools, or even modified PostgreSQL
binaries).
- Works well both on-premise and in clouds.
- Thin provisioning in seconds thanks to copy-on-write (CoW) provided by
ZFS filesystem
and special methodology of preparing PostgreSQL database snapshots.
- Any types of original source databases are supported: on-premise, clouds
(including
managed clouds like RDS – in this case, only dump/restore should be used
for initial
thick cloning), any OS, FS. ZFS is supposed to be used only on Database Lab
servers.
- Unlimited size of databases (Postgres database size is unlimited, ZFS
volume can be
up to 2128 bytes, or 256 trillion yobibytes).
- Supported PostgreSQL versions: 9.6, 10, 11, and 12.
- Thin cloning takes only a few seconds, regardless of the database size.
- REST API.
- Client CLI.
- Automated deletion of clones after the specified amount of minutes of
inactivity (configurable).
- Protection from deletion, to disable automated and accidental deletions.
- Continuously updated original copy of data is supported (based on
"restore_command").
- Multiple snapshots to allow provisioning of various versions of the
database.

Database Lab is developed and maintained by Postgres.ai team
https://postgres.ai.

Links:
- Database Lab repository on GitLab
https://gitlab.com/postgres-ai/database-lab
- What Is Database Lab? Comparison to Other Methods
https://postgres.ai/docs/database-lab/what_is_database_lab
- Tutorial https://postgres.ai/docs/database-lab/1_tutorial'
- Community Slack https://database-lab-team-slack-invite.herokuapp.com/
- Changelog and release notes
https://gitlab.com/postgres-ai/database-lab/-/blob/master/CHANGELOG.md

Browse pgsql-announce by date

  From Date Subject
Next Message David Fetter 2020-02-09 18:42:42 == PostgreSQL Weekly News - February 9, 2020 ==
Previous Message Akshay Joshi 2020-02-06 13:48:11 pgAdmin 4 v4.18 released