| From: | 민영재 <dudwo990413(at)gmail(dot)com> |
|---|---|
| To: | Dmitry Dolgov <9erthalion6(at)gmail(dot)com> |
| Cc: | pgsql-bugs(at)lists(dot)postgresql(dot)org, theshallow27(at)gmail(dot)com |
| Subject: | Re: BUG #19736: `JSON_OBJECT ... ABSENT ON NULL` accepts invalid keys when the value is NULL |
| Date: | 2026-11-09 07:45:42 |
| Message-ID: | CAP=tSTAKkF6=NfGHRmPB7XyWescAxxqBAxa4YwFRk_tQg_0q3Q@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-bugs pgsql-hackers |
Hi,
CFbot reported that v2 needs a rebase after b58a9a97fe reverted the
json_check_mutability() changes. V3 is rebased onto master at 553262aff40d.
It updates the insertion context for the shared key validation helper
without restoring the reverted function; there are no behavior changes from
v2.
I also verified that the attached patch applies with git am to a clean
checkout of the stated master commit.
Thanks,
Yeongjae Min
On Sat, 17 Oct 2026 23:53:49 +0900, 민영재 <dudwo990413(at)gmail(dot)com> wrote:
Hi Dmitry, Thanks for the suggestions. V2 is attached. I found the same
issue in json_object_agg_strict() and jsonb_object_agg_strict() and
extended the fix to those workers, also covering JSON_OBJECTAGG with ABSENT
ON NULL. V2 shares the category checks through json_validate_key_category()
and clarifies that key restrictions apply even to omitted pairs. For
JSONTYPE_JSONB, json_categorize_type(..., false, ...) maps jsonb to
JSONTYPE_JSON, so v1 already covered it in the json path. The shared helper
explicitly handles both categories. All 239 core regression tests pass on
macOS/arm64 with assertions enabled, on master at 7787aec8ab00, including
the added aggregate tests. I also verified that omitted keys still avoid
output conversion without uniqueness checking, while WITH UNIQUE KEYS
retains its behavior. Thanks, Yeongjae Min On Fri, 16 Oct 2026 22:41:27
+0200, Dmitry Dolgov <9erthalion6(at)gmail(dot)com> wrote: > > On Sun, Oct 11,
2026 at 07:26:02PM +0900, 민영재 wrote: > > I reproduced both examples on
master at 9e7d152488f1. > > > > In json_build_object_worker(), the ABSENT
ON NULL skip happens before > > NULL-key validation. Both json and jsonb
paths also skip the scalar-key > > checks when omitting an entry without a
uniqueness check. > > > > The attached patch moves the json NULL-key check
before the skip and > > validates the key type before omitting an entry in
either output path. > > It avoids calling output or cast functions for
omitted keys and preserves > > the existing uniqueness checks. > > > >
Regression tests cover NULL, array, composite, json and jsonb keys with > >
NULL values. The new tests fail without the fix; with the patch, make > >
check passes all 239 tests on macOS/arm64 with assertions enabled. > >
Thanks for investigating. I think it's worth to clarify the > documentation
a bit -- one can argue that we don't need to verify keys > if the pair is
going to be skipped, and only the wording happen to > insist on it.
Currently it says: > > <replaceable>key_expression</replaceable> is a
scalar expression > defining the <acronym>JSON</acronym> key, which is >
converted to the <type>text</type> type. > It cannot be
<literal>NULL</literal> nor can it > belong to a type that has a cast to
the <type>json</type> type. > > Maybe extending it to "It cannot be ...,
even if the pair is going to be > eventually omitted due to ABSENT ON
NULL". > > It also probably makes sense to verify if there are more such
examples. > E.g. looks like jsonb_object_agg_strict exhibit something
similar: > > -- jsonb_object_agg_transfn_worker tests if the key is NULL >
select jsonb_object_agg_strict(NULL, NULL); > ERROR: 22023: field name must
not be null > > -- it also checks if the key is not a composite or json, if
the > -- value is not NULL > select jsonb_object_agg_strict('{"a":
1}'::jsonb, 'value'); > ERROR: 22023: key value must be scalar, not array,
composite, or json > > -- but it doesn't do so if the value is NULL >
select jsonb_object_agg_strict('{"a": 1}'::jsonb, NULL); >
jsonb_object_agg_strict > ------------------------- > {} > (1 row) > >
Regarding the proposed patch, maybe the condition on tcategory could be >
factored out to not repeat it few times (including one in the >
datum_to_jsonb_internal). It's also not clear to me why JSONTYPE_JSONB > is
not included in json_build_object_worker, shouldn't it be included > since
it has a cast to json?
| Attachment | Content-Type | Size |
|---|---|---|
| v3-0001-Validate-JSON-object-keys-before-omitting-null-values.patch | text/x-patch | 17.4 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Devrim Gündüz | 2026-11-09 10:01:06 | Re: Errors in pgdg repo packages |
| Previous Message | Henson Choi | 2026-11-09 05:54:05 | Re: first_value()/nth_value() with IGNORE NULLS loop forever (EXCLUDE GROUP/TIES, frame past partition end, no ORDER BY) |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Rui Zhao | 2026-11-09 07:50:17 | Re: Convert ALL SubLinks to ANY SubLinks |
| Previous Message | David Rowley | 2026-11-09 07:36:17 | Re: Wrong results with initplans moved by setrefs.c |