pgsql: Add IGNORE NULLS/RESPECT NULLS option to Window functions.

From: Tatsuo Ishii <ishii(at)postgresql(dot)org>
To: pgsql-committers(at)lists(dot)postgresql(dot)org
Subject: pgsql: Add IGNORE NULLS/RESPECT NULLS option to Window functions.
Date: 2025-10-03 00:59:45
Message-ID: E1v4U9B-001BCJ-00@gemulon.postgresql.org
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-committers

Add IGNORE NULLS/RESPECT NULLS option to Window functions.

Add IGNORE NULLS/RESPECT NULLS option (null treatment clause) to lead,
lag, first_value, last_value and nth_value window functions. If
unspecified, the default is RESPECT NULLS which includes NULL values
in any result calculation. IGNORE NULLS ignores NULL values.

Built-in window functions are modified to call new API
WinCheckAndInitializeNullTreatment() to indicate whether they accept
IGNORE NULLS/RESPECT NULLS option or not (the API can be called by
user defined window functions as well). If WinGetFuncArgInPartition's
allowNullTreatment argument is true and IGNORE NULLS option is given,
WinGetFuncArgInPartition() or WinGetFuncArgInFrame() will return
evaluated function's argument expression on specified non NULL row (if
it exists) in the partition or the frame.

When IGNORE NULLS option is given, window functions need to visit and
evaluate same rows over and over again to look for non null rows. To
mitigate the issue, 2-bit not null information array is created while
executing window functions to remember whether the row has been
already evaluated to NULL or NOT NULL. If already evaluated, we could
skip the evaluation work, thus we could get better performance.

Author: Oliver Ford <ojford(at)gmail(dot)com>
Co-authored-by: Tatsuo Ishii <ishii(at)postgresql(dot)org>
Reviewed-by: Krasiyan Andreev <krasiyan(at)gmail(dot)com>
Reviewed-by: Andrew Gierth <andrew(at)tao11(dot)riddles(dot)org(dot)uk>
Reviewed-by: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Reviewed-by: David Fetter <david(at)fetter(dot)org>
Reviewed-by: Vik Fearing <vik(at)postgresfriends(dot)org>
Reviewed-by: "David G. Johnston" <david(dot)g(dot)johnston(at)gmail(dot)com>
Reviewed-by: Chao Li <lic(at)highgo(dot)com>
Discussion: https://postgr.es/m/flat/CAGMVOdsbtRwE_4+v8zjH1d9xfovDeQAGLkP_B6k69_VoFEgX-A(at)mail(dot)gmail(dot)com

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/25a30bbd4235a49c854036c84fe90f2bc5a87652

Modified Files
--------------
doc/src/sgml/func/func-window.sgml | 38 +--
doc/src/sgml/syntax.sgml | 10 +-
src/backend/catalog/sql_features.txt | 2 +-
src/backend/executor/nodeWindowAgg.c | 467 ++++++++++++++++++++++++++++++++---
src/backend/optimizer/util/clauses.c | 1 +
src/backend/parser/gram.y | 19 +-
src/backend/parser/parse_func.c | 9 +
src/backend/utils/adt/ruleutils.c | 7 +-
src/backend/utils/adt/windowfuncs.c | 10 +
src/include/nodes/parsenodes.h | 1 +
src/include/nodes/primnodes.h | 13 +
src/include/parser/kwlist.h | 2 +
src/include/windowapi.h | 6 +
src/test/regress/expected/window.out | 406 ++++++++++++++++++++++++++++++
src/test/regress/sql/window.sql | 162 ++++++++++++
15 files changed, 1092 insertions(+), 61 deletions(-)

Browse pgsql-committers by date

  From Date Subject
Previous Message Daniel Gustafsson 2025-10-02 21:05:15 pgsql: Fixes for comments in test_bitmapset