| From: | Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> |
|---|---|
| To: | Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com> |
| Cc: | PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, Álvaro Herrera <alvherre(at)kurilemu(dot)de>, Hou, Zhijie/侯 志杰 <houzj(dot)fnst(at)fujitsu(dot)com>, shveta malik <shveta(dot)malik(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> |
| Subject: | Re: DDL deparse |
| Date: | 2026-08-10 20:32:35 |
| Message-ID: | CAD21AoDtueBCmmyd2b7rU=Jd_T1_rX5QAHnRcfpxm+SKnCk6hQ@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
On Wed, Jun 24, 2026 at 10:09 AM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
>
> The patch was just based on the current HEAD and submitted just for
> anyone interested in this feature to check the behavior. I don't think
> it's ready for the review. We can refer the pre-rebased patches[1]
> that are split to multiple patches.
>
I've updated the DDL deparse patch for CREATE TABLE and ALTER TABLE.
There are three major changes from the previous DDL deparse code;
1. The new design explicitly mentions that DDL deparse should preserve
user intent (e.g., a clause user didn't write is not emitted) while
allowing to normalize and schema-qualify the original DDL. That makes
things what we should follow in future changes.
2. Support the latest CREATE/ALTER TABLE syntaxes and refactored the
deparse code while fixing a lot of bugs. I've refactored the DDL
deparse code and introduced some helper functions to make JSON blob
construction easy. Also, the previous patch could not deparse some
CREATE/ALTER TABLE syntax properly. For instance,
- "create table t (a int, unique (a) using index tablespace ts
deferrable);" produces "CREATE TABLE public.t (a pg_catalog.int4
STORAGE PLAIN, CONSTRAINT t_a_key UNIQUE (a) DEFERRABLE USING INDEX
TABLESPACE ts)", which is not executable syntax.
- "create table t (a int, unique (a) with (fillfactor = 90));"
produces "CREATE TABLE public.t2 (a pg_catalog.int4 STORAGE PLAIN,
CONSTRAINT t2_a_key UNIQUE (a))", which lacks the fillfactor setting.
Also, I found out that we need to differently treat the USING clause
in ALTER TABLE .. ALTER TYPE command from other claude especially when
DROP COLUMN subcommand is also executed in the same ALTER TABLE
command. The following ALTER TABLE cannot be deparsed properly:
create table test (a int, b int);
alter table test drop column a, alter column b type numeric using (a +
b)::numeric;
old design: ALTER TABLE public.test DROP COLUMN a, ALTER COLUMN b SET
DATA TYPE pg_catalog."numeric" USING (("?dropped?column?" +
b))::numeric
new design: ALTER TABLE public.test DROP COLUMN a, ALTER COLUMN b SET
DATA TYPE pg_catalog."numeric" USING ((a + b))::numeri
I've confirmed these are fixed and verified that DDL depasse can
properly deparse all CREATE/ALTER TABLE commands written in the
regression tests by using the new regression tests mentioned below.
3. Regression tests now require only one regression test run.
Previously it required running regression tests twice in order to
prove that deparsed DDLs result in the same effect. A new
001_deparse_regress.pl test reliably detects the problem if newly
added clause/syntaxes miss DDL deparse support.
In the new tests, we prepare an event trigger function, and in the
process utility hook function we execute the CREATE TABLE command in a
subtransaction. In the event trigger function we perform DDL deparse
and save the generated JSON blob in TopTransactionContext. Then,
rollback the subtransaction and re-execute the deparsed DDL again.
I've added more tests for CREATE/ALTER TABLE deparse to test_ddl_deparse.
FYI test_ddl_deparse can be used for interactive tests during the
development. It can be installed via shared_preload_libraries or LOAD
command, and we can set test_ddl_deparse.print_deparsed_ddl to
'json|text|both' to see how the CREATE TABLE DDL is deparsed:
=# create extension test_ddl_deparse;
CREATE EXTENSION
=# set test_ddl_deparse.print_deparsed_ddl to 'both';
SET
=# create table test_tbl (id serial, name text);
NOTICE: deparsed JSON: {"tag": "CREATE TABLE", "command": {"fmt":
"CREATE%{persistence}s TABLE%{if_not_exists}s
%{identity}D%{of_type}s%{partition_of}s%{table_elements}s%{inherits}s%{partition_bound}s%{partition_by}s%{access_method}s%{with_clause}s%{on_commit}s%{tablespace}s",
"of_type": null, "identity": {"objname" : "test_tbl", "schemaname":
"public"}, "inherits": null, "on_commit": null, "tablespace": null,
"persistence": null, "with_clause": null, "partition_by": null,
"partition_of": null, "access_method": null, "if_not_exists": null,
"table_elements": {"fmt": " (%{elements:, }s)", "elements": [{"fmt":
"%{name}I %{coltype}T%{storage}s%{compression}s%{collation}s%{not_null}s%{default}s%{identity_column}s%{generated_column}s",
"name": "id", "type": "column", "coltype": {"typmod": "", "typarray":
false, "typename": "serial", "schemaname": ""}, "default": null,
"storage": null, "not_null": null, "collation": null, "compression":
null, "identity_column": null, "generated_column": null}, {"fmt":
"%{name}I %{coltype}T%{storage}s%{compression}s%{collation}s%{not_null}s%{default}s%{identity_column}s%{generated_column}s",
"name": "name", "type": "column", "coltype": {"typmod": "",
"typarray": false, "typename": "text", "schemaname": "pg_catalog"},
"default": null, "storage": null, "not_null": null, "collation": null,
"compression": null, "identity_column": null, "generated_column":
null}]}, "partition_bound": null}}
NOTICE: deparsed DDL: CREATE TABLE public.test_tbl (id serial, name
pg_catalog.text)
CREATE TABLE
And setting test_ddl_deparse.execute_deparsed_ddl to on does the
round-trip test.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
| Attachment | Content-Type | Size |
|---|---|---|
| v1-0004-Update-the-DDL-deparse-regression-tests-for-ALTER.patch | text/x-patch | 70.6 KB |
| v1-0001-Support-DDL-deparse-for-CREATE-TABLE-command.patch | text/x-patch | 101.8 KB |
| v1-0003-Deparse-ALTER-TABLE.patch | text/x-patch | 78.5 KB |
| v1-0002-Add-regression-tests-for-DDL-deparse.patch | text/x-patch | 81.1 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Chao Li | 2026-08-10 21:24:52 | Re: Fix detection of truncated zstd-compressed backups |
| Previous Message | Masahiko Sawada | 2026-08-10 20:29:29 | Re: Per-table resync for logical replication subscriptions |