Query jumbling ignores XML names, ROLLUP/CUBE, VARIADIC and more

From: Staroverov Ilja <i(dot)staroverov(at)ftdata(dot)ru>
To: "pgsql-bugs(at)lists(dot)postgresql(dot)org" <pgsql-bugs(at)lists(dot)postgresql(dot)org>
Cc: "michael(at)paquier(dot)xyz" <michael(at)paquier(dot)xyz>
Subject: Query jumbling ignores XML names, ROLLUP/CUBE, VARIADIC and more
Date: 2026-10-26 10:45:40
Message-ID: B388C5B3-83C2-44B3-B883-BE72A9E5A2B5@localhost.localdomain
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

Hi,

While working on the SQL/JSON patterns in pg_stat_statements [1], I
looked at the other fields marked query_jumble_ignore, and some of them
change the results of the queries. So queries that differ in more than
their constants get the same query ID and are merged into a single
entry, although the documentation says that queries are considered the
same if they are semantically equivalent except for the values of
literal constants. For example, on master as of 022a80fe6c:

SELECT XMLELEMENT(NAME foo, 'x');
SELECT XMLELEMENT(NAME bar, 'x');
SELECT calls, query FROM pg_stat_statements
WHERE query LIKE '%XMLELEMENT%';
calls | query
-------+---------------------------------
2 | SELECT XMLELEMENT(NAME foo, $1)

although the first query returns <foo>x</foo> and the second one
<bar>x</bar>. The same happens with:

- 0002: the names in XMLATTRIBUTES, XMLFOREST and XMLPI, and DOCUMENT
or CONTENT in XMLPARSE and XMLSERIALIZE;
- 0003: the column types, defaults, NOT NULL and FOR ORDINALITY of
XMLTABLE, the number of its namespaces, and the names of its columns
without PATH;
- 0004: GROUP BY ROLLUP (a, b), CUBE (a, b) and GROUPING SETS (a, b);
- 0005: the SEARCH and CYCLE clauses of recursive CTEs;
- 0006: the column definition lists of functions in FROM, like
json_to_record(...) AS t(a int) and AS t(a text);
- 0007: VARIADIC with functions taking VARIADIC "any", like
format('%s', VARIADIC ARRAY['a', 'b']), which returns "a", and the
same without VARIADIC, which returns "{a,b}";
- 0008: the fields of a composite column assigned by INSERT or UPDATE,
like UPDATE t SET c.x = 1 and UPDATE t SET c.y = 1;
- 0009: the composite type that a ROW() constructor is cast to;
- 0010: the operators of row comparisons, like < and ~<~ for text;
- 0011: OVERRIDING in INSERT and MERGE.

None of these has ever been part of the query ID, so this is not a
regression, and since fixing it changes query IDs, the attached patches
are for master only. As Michael suggested in [1], 0001 adds the tests,
showing the current behavior, and each of 0002-0011 fixes one of these
cases, so that its diff of the expected output shows the entries being
split. The SQL/XML tests go into a new file, xml.sql, with an alternate
output for builds without libxml, as in the main regression test
suite.

OVERRIDING was mentioned by Julien Rouhaud in 2020 [2], and Alvaro
Herrera thought in 2022 that it doesn't matter, as the column list is
part of the query ID [3]. But with OVERRIDING USER VALUE, the values
given for an identity column are ignored:

CREATE TABLE t (a int GENERATED BY DEFAULT AS IDENTITY, b int);
INSERT INTO t VALUES (100, 1);
INSERT INTO t OVERRIDING USER VALUE VALUES (100, 1);
TABLE t;
a | b
-----+---
100 | 1
1 | 1

and both INSERTs are in the same entry. Since Query.override is set
for all queries, 0011 changes all query IDs. It comes last, so it can
be left out without touching the others.

As in the JSON_TABLE patch in [1], column names and namespace prefixes
are left out, for example the names given by SET and USING in SEARCH
and CYCLE, except where they change the results: XMLTABLE uses the name
of a column without PATH as its path, and json_to_record() and similar
functions use the names in the column definition list as the keys to
look up.

0003 also splits JSON_TABLE entries with different column types, as
JSON_TABLE sets TableFunc.coltypes too. It overlaps with the JSON_TABLE
patch in [1], so whichever goes in second needs a rebase, which I can
do.

Two other kinds of merged queries are not addressed here, as they would
need a different approach. The tests show the ones about SQL/XML.

First, some keywords are turned into Consts by the parser, and the
values of Consts are not part of the query ID. This is the case of
STRIP or PRESERVE WHITESPACE in XMLPARSE, and of VERSION NO VALUE and
the STANDALONE option of XMLROOT, whose Consts have no location, so the
text of the entry shows the first query. It is also the case of the
field of EXTRACT and of the normal form of IS NORMALIZED and
NORMALIZE(), whose Consts have the location of the keyword, so the
entry shows it as a parameter symbol, as reported in [4].

Second, type modifiers of casts are not part of the query ID either:
CAST(x AS varchar(10)) and CAST(x AS varchar(20)) are merged, as the
type modifier is only kept in a Const argument of the length coercion
function, or in Const.consttypmod for a literal. The same goes for
XMLSERIALIZE(... AS varchar(10)).

[1] https://postgr.es/m/457969A9-5D24-422F-B09E-F388053DE028@localhost.localdomain
[2] https://postgr.es/m/CAOBaU_bt3-CwkAQLu7kUp-Yy7dyjhTntNG7Q30CssJHjaPYKEQ@mail.gmail.com
[3] https://postgr.es/m/20220926114629.fmdh4sblhpcrlnoo@alvherre.pgsql
[4] https://postgr.es/m/2a0a943877fe79eaf79f1ad752765823d95dce7e.camel@postgrespro.ru

--
Ilya Staroverov

Attachment Content-Type Size
v1-0001-pg_stat_statements-Add-tests-for-more-query-patte.patch application/octet-stream 56.1 KB
v1-0002-Include-SQL-XML-names-and-DOCUMENT-or-CONTENT-in-.patch application/octet-stream 6.1 KB
v1-0003-Include-XMLTABLE-column-definitions-in-query-jumb.patch application/octet-stream 16.4 KB
v1-0004-Include-the-kind-of-grouping-sets-in-query-jumbli.patch application/octet-stream 3.2 KB
v1-0005-Include-SEARCH-and-CYCLE-clauses-in-query-jumblin.patch application/octet-stream 5.0 KB
v1-0006-Include-column-definition-lists-in-query-jumbling.patch application/octet-stream 3.6 KB
v1-0007-Include-VARIADIC-in-query-jumbling.patch application/octet-stream 3.3 KB
v1-0008-Include-the-fields-assigned-by-FieldStore-in-quer.patch application/octet-stream 2.7 KB
v1-0009-Include-the-type-of-row-constructors-in-query-jum.patch application/octet-stream 2.6 KB
v1-0010-Include-the-operators-of-row-comparisons-in-query.patch application/octet-stream 2.6 KB
v1-0011-Include-OVERRIDING-in-query-jumbling.patch application/octet-stream 4.1 KB

Browse pgsql-bugs by date

  From Date Subject
Next Message Nitin Motiani 2026-10-26 11:21:07 Re: BUG #19761: btree_gist: "cannot subtract infinite dates" when inserting 'infinity' into a GiST index on date
Previous Message Dilip Kumar 2026-10-26 10:38:46 Re: BUG #19779: Backend SIGSEGV in UPDATE with FOR UPDATE CTE joined to UNION ALL subquery