From b431131bcb8e65244ce6325f0937ace56068d008 Mon Sep 17 00:00:00 2001 From: Tatsuo Ishii Date: Tue, 20 Oct 2026 17:34:39 +0900 Subject: [PATCH v54 6/9] Row pattern recognition patch (docs). --- doc/src/sgml/advanced.sgml | 136 ++ doc/src/sgml/func/func-window.sgml | 144 ++ doc/src/sgml/perform.sgml | 92 + doc/src/sgml/ref/select.sgml | 197 ++ src/backend/executor/README.rpr | 2976 ++++++++++++++++++++++++++++ 5 files changed, 3545 insertions(+) create mode 100644 src/backend/executor/README.rpr diff --git a/doc/src/sgml/advanced.sgml b/doc/src/sgml/advanced.sgml index 3286c2cf0b2..224bac63525 100644 --- a/doc/src/sgml/advanced.sgml +++ b/doc/src/sgml/advanced.sgml @@ -552,6 +552,142 @@ WHERE pos < 3; two rows for each department). + + Row Pattern Common Syntax can be used to perform Row Pattern Recognition + in a query. The Row Pattern Common Syntax includes four sub clauses, + which must be written in this order: + AFTER MATCH SKIP, INITIAL + or SEEK, PATTERN + and DEFINE. The first two are optional, and the + complete example below uses both. + DEFINE defines row pattern variables along with an + expression. The expression must be a logical expression, which means + it must return TRUE, FALSE + or NULL. The expression may comprise column references + and non-volatile functions. Window functions, aggregate functions, + grouping operations, set-returning functions, whole-row references and + subqueries are not allowed. An example of DEFINE is + as follows. + + +DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price), + DOWN AS price < PREV(price) + + + Note that PREV returns the price + column in the previous row if it's called in a context of row pattern + recognition. Thus in the second line the row pattern variable "UP" + is TRUE when the price column in the current row is + greater than the price column in the previous row. Likewise, "DOWN" + is TRUE when the + price column in the current row is lower than + the price column in the previous row. + + + Once DEFINE exists, PATTERN can be + used. PATTERN defines a sequence of rows that satisfies + conditions defined in the DEFINE clause. For example + the following PATTERN defines a sequence of rows + starting with a row satisfying "LOWPRICE", then one or more rows satisfying + "UP" and finally one or more rows satisfying "DOWN". Pattern variables can + be followed by quantifiers: "+" means one or more matches, "*" means zero + or more matches, "?" means zero or one match, "{n}" (n > 0) means + exactly n matches, "{n,}" (n >= 0) means at least n matches, + "{,m}" (m > 0) means at most m matches, and + "{n,m}" (0 <= n <= m, 0 < m) means between n and m matches. + Patterns can be grouped using parentheses and combined using alternation + (the vertical bar "|" for OR). For example, "(UP DOWN)+" matches one or + more repetitions of UP followed by DOWN. If a sequence of rows which + satisfies the PATTERN is found, in the starting row all columns or + functions are shown in the target list. Note that aggregations only look + into the matched rows, rather than the whole frame. On the second or + subsequent rows the window functions that read the frame, such as + first_value(), are shown as NULL, while functions that + do not depend on the frame, such as row_number(), are + unaffected. Aggregates on non-starting rows return their initial value: + for example, count() returns 0 and + sum() returns NULL. Rows that do not match the + PATTERN behave the same way. + So do rows at which the PATTERN finds an empty match, which can happen + when the pattern as a whole can match zero rows (for + example A* or A?): such a match + contains no rows, so the frame is empty. + Example of a SELECT using + the DEFINE and PATTERN clause is as + follows. + + +SELECT company, tdate, price, + first_value(price) OVER w, + max(price) OVER w, + count(price) OVER w +FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + + + company | tdate | price | first_value | max | count +----------+------------+-------+-------------+-----+------- + company1 | 2023-07-01 | 100 | 100 | 200 | 4 + company1 | 2023-07-02 | 200 | | | 0 + company1 | 2023-07-03 | 150 | | | 0 + company1 | 2023-07-04 | 140 | | | 0 + company1 | 2023-07-05 | 150 | | | 0 + company1 | 2023-07-06 | 90 | 90 | 130 | 4 + company1 | 2023-07-07 | 110 | | | 0 + company1 | 2023-07-08 | 130 | | | 0 + company1 | 2023-07-09 | 120 | | | 0 + company1 | 2023-07-10 | 130 | | | 0 +(10 rows) + + + + + Row Pattern Recognition internally uses a nondeterministic finite + automaton (NFA) to match patterns. For patterns with unbounded + quantifiers (e.g., A+ or (A B)+), + the NFA may need to track many active matching contexts simultaneously, + which could potentially lead to O(n2) + complexity as the number of rows increases. + + + + Before execution, PostgreSQL automatically + optimizes patterns to simplify their structure. This includes flattening + nested sequences and alternations, merging consecutive identical variables + (e.g., A{2,3} A{1,2} becomes A{3,5}), + removing duplicate alternatives + (e.g., (A | B | A) becomes (A | B)), + and simplifying nested quantifiers + (e.g., (A*)* becomes A*). + These optimizations reduce pattern complexity and also decrease + nesting depth, making the 254-level depth limit rarely encountered. + They are applied transparently and can be observed + in EXPLAIN output. + + + + To mitigate the O(n2) complexity described + above, PostgreSQL also employs + a context absorption optimization. When a pattern starts with a greedy + unbounded element, newer matching contexts cannot produce longer matches + than older contexts. By detecting and eliminating these redundant + contexts, the matching complexity is reduced from + O(n2) to O(n) for many common patterns. + + When a query involves multiple window functions, it is possible to write out each one with a separate OVER clause, but this is diff --git a/doc/src/sgml/func/func-window.sgml b/doc/src/sgml/func/func-window.sgml index bb41387f873..823923ac1da 100644 --- a/doc/src/sgml/func/func-window.sgml +++ b/doc/src/sgml/func/func-window.sgml @@ -278,6 +278,150 @@ IGNORE NULLS nth_value. + + Row Pattern Recognition navigation functions are listed in + . These functions + can be used to describe the DEFINE clause of Row Pattern Recognition. + The names PREV, NEXT, + FIRST, and LAST are + recognized as navigation functions only in an unqualified call; a + schema-qualified call is resolved as an ordinary function instead. + + + + Row Pattern Navigation Functions + + + + + Function + + + Description + + + + + + + + + prev + + prev ( value anyelement [, offset bigint ] ) + anyelement + + + Returns value evaluated at the row that is + offset rows before the current row within + the partition; + returns NULL if the target row is outside the partition. + offset defaults to 1 if omitted. + offset must be a non-negative integer; + an offset of 0 refers to the current row itself. + offset must not be NULL. + Can only be used in a DEFINE clause. + + + + + + + next + + next ( value anyelement [, offset bigint ] ) + anyelement + + + Returns value evaluated at the row that is + offset rows after the current row within + the partition; + returns NULL if the target row is outside the partition. + offset defaults to 1 if omitted. + offset must be a non-negative integer; + an offset of 0 refers to the current row itself. + offset must not be NULL. + Can only be used in a DEFINE clause. + + + + + + + first + + first ( value anyelement [, offset bigint ] ) + anyelement + + + Returns value evaluated at the row that is + offset rows after the match start row; + returns NULL if the target row is beyond the current row. + offset defaults to 0 if omitted, referring to + the match start row itself. + offset must be a non-negative integer. + offset must not be NULL. + Can only be used in a DEFINE clause. + + + + + + + last + + last ( value anyelement [, offset bigint ] ) + anyelement + + + Returns value evaluated at the row that is + offset rows before the current row within + the match; + returns NULL if the target row is before the match start row. + offset defaults to 0 if omitted, referring to + the current row itself. + offset must be a non-negative integer. + offset must not be NULL. + Can only be used in a DEFINE clause. + + + + + +
+ + + PREV and NEXT may wrap + FIRST or LAST for compound + navigation. For example, + PREV(FIRST(val, 2), 3) fetches the value at + 3 rows before the row that is 2 rows after the match start. + The inner FIRST or LAST is + checked first, as it would be on its own: if the inner + FIRST row is beyond the current row, or the inner + LAST row is before the match start row, the result + is NULL, even when the outer offset would lead back into range. The + outer PREV or NEXT may then + reach rows outside the match, and returns NULL only if the target row + is outside the partition. + The reverse nesting (FIRST/LAST + wrapping PREV/NEXT) is not + permitted. Same-category nesting (e.g., + PREV inside PREV) is also + prohibited. + The offset argument must be a run-time constant: + it cannot reference columns or contain a navigation operation. + In a compound navigation the inner FIRST or + LAST call must be the whole first argument of + PREV or NEXT; an argument + that merely contains one, such as PREV(FIRST(val) + 1), + is rejected. + The value argument, on the other hand, must + include at least one column reference; an argument made only of + constants or parameters, such as PREV(1), is + rejected. + + The SQL standard defines a FROM FIRST or FROM LAST diff --git a/doc/src/sgml/perform.sgml b/doc/src/sgml/perform.sgml index ea8da01b779..767fb1a0170 100644 --- a/doc/src/sgml/perform.sgml +++ b/doc/src/sgml/perform.sgml @@ -702,6 +702,98 @@ FROM tenk1 t1 WHERE t1.ten = (SELECT (random() * 10)::integer); happen without the sub-SELECT construct. + + When examining query plans for Row Pattern Recognition with + EXPLAIN, the Pattern line may + carry absorption markers. They report an analysis the planner has + already made, not an opportunity left open. A number sign + # marks the comparison point of an absorption, which + is always an element carrying an unbounded quantifier, and a tilde + ~ marks an element that lies within the absorbable + region. Neither marker requires an alternation to be present. + For example, using the stock + table from , a query that looks for + repeated up-then-down movements reports its pattern as + (up~ down~)+#: + + +EXPLAIN (COSTS OFF) +SELECT company, tdate, price, count(price) OVER w +FROM stock +WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN ((UP DOWN)+) + DEFINE UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + + QUERY PLAN +-------------------------------------------------------------------&zwsp;------------------------------------ + WindowAgg + Window: w AS (PARTITION BY company ORDER BY tdate ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: (up~ down~)+# + Nav Mark Lookback: 1 + -> Sort + Sort Key: company, tdate + -> Seq Scan on stock + + + Here the tildes mark up and + down as lying within the absorbable region, while + the trailing number sign marks the repeated group as the comparison + point. + + + + The Nav Mark Lookback line reports the row-retention + bound the executor computed for the backward navigations used in + DEFINE (PREV and + LAST); a Nav Mark Lookahead line + appears when DEFINE navigates forward + (FIRST). Each reports a row count when the + navigation offsets are constant; the lookahead is measured from the + match start, so a compound navigation that reaches back past it, such + as PREV(FIRST(val, 1), 2), reports a negative one. + Otherwise Nav Mark Lookback reports + runtime for an offset that is not known until + execution, or retain all when no bound could be + computed and every row of the frame has to be kept; and + Nav Mark Lookahead reports runtime + or infinite. + Under ANALYZE, a node that was executed shows the + bound resolved for its last scan instead of runtime: + a row count, retain all or + infinite. + + + + Under ANALYZE, further lines report counters for the + matcher's internal states and contexts. Those are diagnostics for the + implementation rather than a stable interface; their names and contents + may change. + + + + Neither the AFTER MATCH SKIP nor the + INITIAL subclause is printed. Nothing is lost for + INITIAL, which is the only mode currently supported, + but AFTER MATCH SKIP still shows through: absorption + is analyzed only under SKIP PAST LAST ROW, so the + same pattern prints with the # and + ~ markers in that mode and without them in the + others. + The markers are also left out when the frame ends at + offset FOLLOWING rather + than UNBOUNDED FOLLOWING, or when a + DEFINE condition uses FIRST, or + LAST with an offset of its own, whether alone or + inside PREV or NEXT. + + diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml index 18392f7caec..e970f18349d 100644 --- a/doc/src/sgml/ref/select.sgml +++ b/doc/src/sgml/ref/select.sgml @@ -929,6 +929,7 @@ WINDOW window_name AS ( expression [, ...] ] [ ORDER BY expression [ ASC | DESC | USING operator ] [ NULLS { FIRST | LAST } ] [, ...] ] [ frame_clause ] +[ row_pattern_common_syntax ] @@ -1079,6 +1080,177 @@ EXCLUDE NO OTHERS a given peer group will be in the frame or excluded from it. + + The + optional + row_pattern_common_syntax + defines the Row Pattern Recognition condition for + this window. + row_pattern_common_syntax + includes the following subclauses. + + +[ { AFTER MATCH SKIP PAST LAST ROW | AFTER MATCH SKIP TO NEXT ROW } ] +[ INITIAL | SEEK ] +PATTERN ( pattern_variable_name [ quantifier ] [ ... ] ) +DEFINE definition_variable_name AS expression [, ...] + + AFTER MATCH SKIP PAST LAST ROW or AFTER MATCH + SKIP TO NEXT ROW controls how to proceed to the next row position + after a match is found. With AFTER MATCH SKIP PAST LAST + ROW (the default) the next row position is next to the last row + of the previous match. On the other hand, with AFTER MATCH SKIP TO + NEXT ROW the next row position is next to the first row of the + previous match. INITIAL or SEEK + specifies from which row in the frame pattern matching begins. + If INITIAL is specified, the match must start + from the first row in the frame. If SEEK is specified, + the set of matching rows does not necessarily start from the first row. The + default is INITIAL. Currently + only INITIAL is supported. DEFINE + defines definition variables along with a boolean + expression. PATTERN defines a sequence of rows that + satisfies certain conditions using variables defined + in the DEFINE clause (an empty + PATTERN() is not accepted by the syntax). Each pattern + variable can be followed by a quantifier to specify how many times it + should match: * (zero or more), + + (one or more), + ? (zero or one), + {n} + (exactly n times, n > 0), + {n,} + (at least n times, n >= 0), + {,m} + (at most m times, m > 0), or + {n,m} + (between n and m + times, 0 <= n <= m, 0 < m). + Reluctant quantifiers (e.g., *?, +?, + ??, + {n,m}?) + are supported. + The exclusion ({- and -}) is not + accepted by the syntax, and the permutation + (PERMUTE) is rejected as an unsupported feature. + PERMUTE is recognized wherever + a ( follows it, so a pattern variable of that name has + to be written "permute" in that position. + Patterns can be grouped using parentheses, and alternation (OR) can be + expressed using the vertical bar |. + For example, (A B)+ matches one or more repetitions + of the sequence A followed by B, and A | B matches + either A or B. + If a pattern variable is not defined in + the DEFINE clause, it is not automatically added + to the DEFINE clause. Instead, the executor evaluates + the variable as TRUE at execution time, behaving as if + the following definition existed. + + +variable_name AS TRUE + + + Conversely, variables defined in the DEFINE clause + but not used in the PATTERN clause are rejected + with an error. The DEFINE clause itself is not + optional: leaving it out is a syntax error, even where every pattern + variable would evaluate as TRUE. + A variable may be defined only once in the DEFINE + clause; a second definition of the same name is rejected with an + error. + + + + Row pattern recognition constrains the frame clause it is attached to. + The frame must be written in ROWS mode + as BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING, or + as BETWEEN CURRENT ROW + AND offset FOLLOWING with + a positive offset. + RANGE and GROUPS mode, any other + frame start or end, and + any frame_exclusion option other + than EXCLUDE NO OTHERS are rejected with an error. + + + + Note that the maximum number of unique pattern variables + used in the PATTERN clause is 240. + Additionally, the maximum nesting depth of pattern groups + (parentheses) is 254 levels. An alternation costs a level of its own, + so a pattern that nests alternations reaches the limit at about half + as many parentheses. + However, pattern optimizations such as flattening nested sequences + and simplifying nested quantifiers may reduce the effective depth, + so this limit is rarely reached in practice. + The pattern is also limited to 32767 elements, counted after these + optimizations: one for each occurrence of a pattern variable, two for + each group with a quantifier other than {1}, one + for each alternation plus one for each of its alternatives, and one + more for the pattern as a whole. A larger pattern is rejected with + an error. + + + + Row pattern recognition cannot appear anywhere in a common table + expression that belongs to a WITH RECURSIVE clause; + such a query is rejected with an error. The same restriction reaches + CREATE RECURSIVE VIEW, which is rewritten + to WITH RECURSIVE. + + + + A column reference in the DEFINE clause must be + written without a qualifier. The SQL standard reserves the qualifier + slot for a pattern variable, so a table name or alias is rejected + there, and a pattern variable qualifier is rejected as not supported. + The column name therefore has to resolve on its own across the + whole FROM clause, and an ambiguous name is + rejected. Where the name is not unique, rename the column in + the FROM clause, for example with a column alias on + a subquery. A column of an outer query cannot be referenced from + a DEFINE condition, qualified or not; such a + reference is rejected as not supported. + + + + The same applies to names that are not columns. A parameter or + variable of the routine whose body contains the query is readable from + a DEFINE condition, but only unqualified: the + PostgreSQL spellings that qualify one with + the routine name or a block label occupy the reserved slot and are + rejected there. To read a field of a composite parameter or record + variable, parenthesize it, as in (p).amount; that is + field selection on a value rather than a qualified name, so the slot + stays free. + + + + A PL/pgSQL function written with + #variable_conflict use_variable is an exception, as + it is everywhere else: that setting makes + PL/pgSQL resolve any name one of its + variables owns before the query does, so inside such a function + qualified spellings such as fn.threshold keep + working, and a variable whose name matches a pattern variable takes + A.price as well. This is the same shadowing the + setting applies to table columns; see + . + + + + Two planner optimizations are not applied to a window that uses row + pattern recognition. A filter on the result of a monotonic window + function, such as rn <= 3 on + a row_number() computed in a subquery, is not + pushed down into the window as a run condition, so the window is + evaluated to the end of the partition instead of stopping early. And + the frame clause is not replaced with a cheaper one, even when the + window functions using it do not depend on the frame. Query results + are not affected, only the cost of evaluating the window. + + The purpose of a WINDOW clause is to specify the behavior of window functions appearing in the query's @@ -2192,6 +2364,31 @@ SELECT 2+2; + + Row Pattern Recognition + + + PostgreSQL supports row pattern recognition + both in a WINDOW clause entry and in an + inline OVER ( ... ) specification. The SQL standard + defines more subclauses: MEASURES and + SUBSET. They are not currently supported + in PostgreSQL. Also in the standard there are + more variations in the AFTER MATCH clause. + + + + The largest difference is in DEFINE: a column + reference qualified by a pattern variable, such + as A.price, is not implemented and is rejected as an + unsupported feature, even though the standard writes + its DEFINE examples that way. An unqualified column + reference in a DEFINE expression therefore always + reads the row being tested, and other rows of the match are reachable + only through the navigation operations. + + + Nonstandard Clauses diff --git a/src/backend/executor/README.rpr b/src/backend/executor/README.rpr new file mode 100644 index 00000000000..16e158fdcf3 --- /dev/null +++ b/src/backend/executor/README.rpr @@ -0,0 +1,2976 @@ +============================================================================ + PostgreSQL Row Pattern Recognition (RPR): Flat-Array Stream NFA Guide +============================================================================ + + Readers should understand the specification of row pattern + recognition in the SQL standard [1][2]. If you do not have access + to the SQL standard, Oracle's manual or Trino's manual can be + alternatives. + + This README's scope is the entire process from PATTERN/DEFINE clause + parsing to NFA runtime execution, together with the contracts the + feature places on the rest of the planner (Chapter IV) and on the + two printers that display it (Chapter XIII). + + Related code, by the phase each file serves. A file appears once, + under the phase where its row pattern work belongs: + + Parser + - src/backend/parser/gram.y (grammar and keywords) + - src/backend/parser/parse_rpr.c (PATTERN/DEFINE analysis) + - src/include/parser/parse_rpr.h (parser entry points) + - src/include/parser/parse_node.h (DEFINE parse state) + - src/backend/parser/parse_clause.c (transformRPR() call site) + - src/backend/parser/parse_func.c (navigation name binding) + - src/backend/parser/parse_expr.c (DEFINE column restrictions) + - src/backend/parser/parse_target.c (DEFINE star expansion) + - src/backend/parser/parse_cte.c (WITH RECURSIVE rejection) + - src/backend/parser/parse_agg.c (grouping participation) + + Planner + - src/backend/optimizer/plan/rpr.c (rewrites and compilation) + - src/include/optimizer/rpr.h (types and constants) + - src/backend/optimizer/plan/createplan.c (match_start dependencies) + - src/backend/optimizer/plan/initsplan.c (DEFINE columns needed) + - src/backend/optimizer/plan/planner.c (window clause handling) + - src/backend/optimizer/plan/setrefs.c (DEFINE Vars to OUTER_VAR) + - src/backend/optimizer/plan/subselect.c (DEFINE param sets) + - src/backend/optimizer/path/allpaths.c (keeping DEFINE columns) + - src/backend/optimizer/path/costsize.c (DEFINE evaluation cost) + - src/backend/optimizer/prep/prepjointree.c (PlaceHolderVar wrapping) + - src/backend/optimizer/util/var.c (join alias expansion) + - src/backend/rewrite/rewriteManip.c (marking navigation args) + + Executor + - src/backend/executor/nodeWindowAgg.c (reduced frame and nav trim) + - src/backend/executor/execRPR.c (NFA engine) + - src/include/executor/execRPR.h (NFA public API) + - src/backend/executor/execExpr.c (navigation opcode compile) + - src/backend/executor/execExprInterp.c (navigation opcode eval) + - src/backend/jit/llvm/llvmjit_expr.c (JIT path for those opcodes) + + Node support + - src/include/nodes/parsenodes.h (parse node definitions) + - src/include/nodes/primnodes.h (RPRNavExpr) + - src/include/nodes/plannodes.h (plan node definitions) + - src/include/nodes/execnodes.h (execution state) + - src/backend/nodes/nodeFuncs.c (expression tree walking) + - src/backend/nodes/copyfuncs.c (RPRPattern copy support) + - src/backend/nodes/outfuncs.c (RPRPattern out support) + - src/backend/nodes/readfuncs.c (RPRPattern read support) + - src/backend/nodes/queryjumblefuncs.c (DEFINE clause jumble) + + Output + - src/backend/utils/adt/ruleutils.c (deparse for pg_get_viewdef) + - src/backend/commands/explain.c (EXPLAIN output) + +============================================================================ + +What is a Flat-Array Stream NFA? + + The NFA (Nondeterministic Finite Automaton) in this implementation + is not a traditional state-transition graph but a flat array of + fixed-size 16-byte elements. At runtime, it processes the row stream + in a forward-only manner, expanding epsilon transitions eagerly + without backtracking. + + - Flat-Array: Pattern compiled into a flat array, + not a graph (Chapter V) + - Stream: Rows consumed sequentially in one direction, + never revisited (Chapter XIV) + - NFA: Nondeterministic execution where multiple states + coexist within a single context (Chapter VII) + +Chapter I Row Pattern Recognition Overview +============================================================================ + +Normative reference: ISO/IEC 19075-5 (SQL Technical Report, Part 5: Row +pattern recognition in SQL). Subclause numbers cited throughout this code +base refer to that document. Where Chapters 4 (FROM clause) and 6 (WINDOW +clause) describe parallel material, this implementation cites the Chapter 6 +subclause first because it targets Feature R020. + +RPR is a feature introduced in SQL:2016 that matches regex-based +patterns against ordered row sets. + +The SQL standard defines two forms: + + Feature R010: MATCH_RECOGNIZE (FROM clause) + - Dedicated table operator + - Provides dedicated functions such as MATCH_NUMBER(), CLASSIFIER() + - Supports ONE ROW PER MATCH / ALL ROWS PER MATCH + + Feature R020: RPR in a window (WINDOW clause) + - Integrated into the existing window function framework + - No ROWS PER MATCH clause: every input row is returned once, and + only its reduced window frame depends on the match. In R010 + terms this corresponds to ONE ROW PER MATCH WITH UNMATCHED ROWS: + a match is reported on its starting row, and a row skipped inside + a match or matched by none gets an empty frame + - No MATCH_NUMBER() + +This implementation targets Feature R020. + +The basic syntax is as follows: + + SELECT ... + OVER ( + PARTITION BY ... + ORDER BY ... + ROWS BETWEEN CURRENT ROW AND [UNBOUNDED | offset] FOLLOWING + AFTER MATCH SKIP TO NEXT ROW | SKIP PAST LAST ROW + [INITIAL | SEEK] + PATTERN ( ) + DEFINE AS , ... + ) + +SEEK is defined in the standard but not implemented. The offset in +the frame's end boundary must not be zero or a negative value. + +The PATTERN clause is a regular expression over row pattern variables. +The DEFINE clause specifies boolean conditions that determine whether each +variable evaluates to true for the current row. + +Example: + + PATTERN (A+ B) + DEFINE A AS price > PREV(price), + B AS price < PREV(price) + +This pattern matches "a span where prices rise consecutively then drop." + +PERMUTE is not supported (the parser raises an error). Its syntax is in the +grammar so that the standard spelling is diagnosed rather than read as a +pattern variable followed by a group; write the alternations out instead. + +RPR is not allowed in WITH RECURSIVE or CREATE RECURSIVE VIEW (ISO/IEC +9075-2 7.17; ISO/IEC 19075-5 6.17.5). transformWithClause() +(parse_cte.c) rejects it by walking the raw parse tree with +contain_rpr_walker(), before the CTEs are analyzed. + +Chapter II Overall Processing Pipeline +============================================================================ + +RPR processing is divided into three phases. The planner handles the two +halves of a row pattern window separately: the DEFINE expressions travel +through the planner's ordinary expression machinery, and the PATTERN is +compiled into the element array the executor runs: + + +--------------------------------------------------------------+ + | 1. Parsing (Parser) | + | SQL text -> PATTERN parse tree + DEFINE expression tree | + | | + | 2. Planning (Optimizer/Planner) | + | DEFINE expressions -> pull-up, qual preprocessing, | + | grouping, input target, Var fixup (Chapter IV) | + | PATTERN parse tree -> optimization -> flat NFA elements | + | (Chapter V) | + | | + | 3. Execution (Executor) | + | Row-by-row matching via NFA simulation | + +--------------------------------------------------------------+ + +Each phase uses independent data structures, and the interfaces between +phases are well-defined: + + Parser -> Planner: WindowClause.rpPattern (RPRPatternNode tree) + WindowClause.defineClause (TargetEntry list) + + Planner -> Executor: WindowAgg.rpPattern (RPRPattern struct) + WindowAgg.defineClause (TargetEntry list) + +Chapter III Parsing Phase +============================================================================ + +III-1. Entry Point + + transformWindowDefinitions() (parse_clause.c) + +-- transformRPR() (parse_rpr.c) + +transformRPR() is invoked when RPCommonSyntax is present and performs the +following: + + (1) Frame option validation + - Only ROWS is allowed (RANGE, GROUPS are not) + - The start boundary must be CURRENT ROW + - EXCLUDE option is not allowed + - The end boundary must be UNBOUNDED FOLLOWING or offset FOLLOWING + (not CURRENT ROW); the offset's value, even a literal's, is not + checked here but at execution (XI-4) + + (2) Transcription to WindowClause + - Copies the rpPattern and rpSkipTo fields + + (3) DEFINE clause transformation (transformDefineClause) + + The two frame diagnostics need somewhere to point, and a defaulted frame + leaves only the start of the window definition. WindowDef therefore + carries frameLocation and excludeLocation, which the frame productions in + gram.y set to the ROWS/RANGE/GROUPS keyword and to the EXCLUDE keyword; + transformRPR() falls back to the window's own location when either is -1. + A frame or exclusion production added later has to set them too, or its + errors lose their cursor position. Note that the EXCLUDE check reads the + FRAMEOPTION_EXCLUSION bits, not excludeLocation: EXCLUDE NO OTHERS sets + no bit and is accepted, since it excludes nothing. + +III-2. PATTERN parse tree + +The parser transforms the PATTERN clause into an RPRPatternNode tree. +Each node has one of the following four types: + + RPR_PATTERN_VAR Variable reference. Name stored in varName field. + RPR_PATTERN_SEQ Concatenation. Children node list in children. + RPR_PATTERN_ALT Alternation (or). Branch node list in children. + RPR_PATTERN_GROUP Group (parentheses). Body node list in children. + +All nodes have min/max fields to express quantifiers: + + A -> VAR(A, min=1, max=1) + A+ -> VAR(A, min=1, max=INF) + A* -> VAR(A, min=0, max=INF) + A? -> VAR(A, min=0, max=1) + A{3,5} -> VAR(A, min=3, max=5) + +If the reluctant field is true, the quantifier is reluctant (non-greedy). + +The braced forms are range-checked in the grammar itself. Every bound must +be below RPR_QUANTITY_INF (parsenodes.h), which is also the sentinel the +unbounded spellings store in max. {n} and {,m} require a bound of at least +1, as does the upper bound of {n,m}; the lower bound of {n,m} and the bound +of {n,} may be 0; and {n,m} requires n <= m. So A{0}, A{,0} and A{0,0} are +rejected, while A{0,} is another way to write A*. That cap on the lower +bound is the only one the grammar imposes, which X-6 revisits as a cost. + +Example: PATTERN ((A+ B) | C*) + + ALT + +-- GROUP(1, 1) + | +-- SEQ + | +-- VAR(A, 1, INF) + | +-- VAR(B, 1, 1) + +-- VAR(C, 0, INF) + +Parentheses always produce a GROUP node; a GROUP(1, 1) like the one above is +unwrapped later, by Phase 1 (h). + +The lexer counts '|' as both a self character and an operator character, so +a quantifier written hard against an alternation arrives as a single Op +token rather than as two: "A*|B" yields Op "*|", not '*' then '|'. Every +reluctant spelling glues the same way ("*?|", "+?|", "??|"), as does the Op +after a braced quantifier, as in "A{2,3}?|B". + +Making '|' a self character takes a lone '|' out of Op everywhere, not only +inside PATTERN. The expression grammar therefore spells it out: a_expr and +b_expr carry prefix and infix '|' productions at the precedence of Op, and +MathOp lists '|', so all_Op and the OPERATOR(...) forms still accept it. +A new production that takes an operator through Op has to accept '|' as +well. psqlscan.l and pgc.l were not changed and still leave '|' out of +self, so they no longer match scan.l. + +row_pattern_quantifier_opt therefore accepts each glued spelling, takes the +quantifier it half spells, and sets RPRPatternNode.trailing_alt on the term +to record that an alternation operator came with it. The flag stays on the +term while the sequence keeps growing; splitRPRTrailingAlt() then splits the +finished sequence at the flagged term into ALT(left, right), where the right +branch is the whole remaining sequence. That is what keeps '|' the +lowest-precedence operator of the pattern language: "A*|B C" parses as +"A* | (B C)", identical to the spaced form. A flagged term with nothing to +its right is a dangling '|' and is rejected. + +trailing_alt is transient. splitRPRTrailingAlt() clears it on every path, +the error paths included, so a finalized tree never carries it and it +reaches neither the plan nor the query id. + +III-3. DEFINE Clause Transformation + +transformDefineClause() first validates the PATTERN variable count and +collects the names, then rejects any DEFINE variable that PATTERN does not +use. After that it processes each DEFINE variable as follows: + + (1) Checks for duplicate variable names -- this one in a pass of its + own over the whole list, before any expression is transformed + (2) Transforms the expression and coerces it to Boolean right away, + through transformWhereClause() as for WHERE and HAVING, so that the + steps below see the final expression form + (3) Wraps in a TargetEntry with the variable name set in resname + +Parse analysis puts nothing in the query targetlist for a DEFINE clause. +Getting what it reads to the WindowAgg is the planner's job, and it is done +the way havingQual's is: build_base_rel_tlists() marks the columns needed so +they reach the top of the join tree, and make_window_input_target() asks for +them again in the node's own input target (IV-6). + +After all variables are processed: + (4) Validates navigation nesting and offsets (define_walker) + +Step (4) is where the shape every later phase assumes gets established. +define_walker() requires the argument of each navigation to contain at least +one column reference, and requires each offset to be a run-time constant: an +offset may contain neither a column reference nor another navigation. The +Var-free rule is what lets the executor settle an offset once per scan (VII-6) +rather than once per row -- the offset expression is evaluated with no current +row installed and the result pinned for the whole scan, which is sound only +because no column can appear there. A Param is still allowed, and is the case +RPR_NAV_OFFSET_NEEDS_EVAL exists for. An omitted offset stays +NULL on the node and the kind's default is supplied at execution: 1 for +PREV/NEXT, 0 for FIRST/LAST, and 1 for a compound outer offset. + +Step (4) also changes the expression it walks. A PREV or NEXT whose +argument is exactly one FIRST or LAST call is merged in place into one +RPRNavExpr of a compound kind (RPR_NAV_PREV_FIRST and so on): the inner +call's argument and offset become arg and offset_arg, and the outer offset +moves to compound_offset_arg. Every other nesting is rejected: FIRST or +LAST over any navigation, PREV or NEXT over PREV or NEXT, an inner +navigation that is only part of the argument, as in PREV(v + FIRST(v)), and +a third level. A navigation inside an offset is rejected as well. Once +step (4) is done, no RPRNavExpr contains another one anywhere; the planner +relies on this (compute_matchStartDependent() asserts it), and the deparser +rebuilds the nested spelling from the compound kind. + +Variables that are used in PATTERN but not defined in DEFINE are implicitly +evaluated as TRUE (matching all rows). + +The restrictions the sections below put a DEFINE condition under are keyed on +p_rpr_define, a parse state flag transformDefineClause() holds up for the +whole of step (2), rather than on p_expr_kind. p_expr_kind names the +innermost clause, and a condition can reach two clauses that set a kind of +their own: an aggregate's FILTER, and its ORDER BY, which is also what WITHIN +GROUP writes. Keyed on the kind, every rule below would stop applying inside +those, and what they hid would be resolved and kept. A sub-select gets a +parse state of its own and does not inherit the flag, which is right: the +restrictions end at the query boundary. + +Three rules are left keyed on the kind -- the ones that reject an aggregate, +a window function and a set-returning function -- because a DEFINE condition +reaches those clauses only underneath an aggregate, and an aggregate in a +DEFINE condition is rejected at its own node. That holds as long as the +aggregate belongs to this query level, and the two ways its level can be +lifted out of the clause, an outer column reference and a sublink, are both +rejected on the way down. Relaxing either of those puts this back in play. + +III-4. Column References in a DEFINE Condition + +A name written in a DEFINE condition must be unqualified: ISO/IEC 19075-5 6.5 +reserves the qualifier slot for a row pattern variable, so nothing else may +occupy it. transformColumnRef() (parse_expr.c) enforces this wherever in a +DEFINE condition the name stands, rejecting a reference that resolves to an +outer query's column, a two-part name qualified by a FROM-clause range +variable, and every other qualified name -- including one that +p_post_columnref_hook resolves, such as a SQL function's parameter or a +PL/pgSQL variable spelled with its routine name or block label. Unqualified, +those remain readable; it is the spelling that is refused, not the value. + +A pattern variable qualifier such as A.price is decided before name +resolution, by matching the qualifier against the PATTERN variable names +that transformDefineClause() installs in the parse state for the duration of +the DEFINE transformation and clears afterwards. It has to be caught there +because a pattern variable names no range table entry, so ordinary +resolution would reject it as a qualified expression instead; it is reported +as not yet implemented. Deciding on the qualifier alone also means a +pattern variable takes that name in the qualifier slot from anything else +that could answer to it -- a FROM-clause alias, or the containing routine, +whose own parameters become unreachable by their qualified spelling when a +pattern variable is named after it. Nothing is lost that way, since a +qualified name is rejected in DEFINE whoever it names; what changes is which +of the two rules reports it. + +A reference ending in a star, such as t.* or s.t.*, is decided before name +resolution too, right after the pattern variable test: no qualifier makes a +whole-row reference legal here, so resolving it first would only choose +which rejection it gets. The remaining qualified forms are diagnosed only +after the reference has resolved, so that a misspelled column still gets the +ordinary "Perhaps you meant" hint rather than being blamed on its qualifier. + +All of these rules see only the names the ref hooks leave to the query +parser, so none of them is the last word on what a DEFINE clause accepts. +A procedural language that answers a name first keeps it, and PL/pgSQL's +p_pre_columnref_hook answers whenever the function was written with +"#variable_conflict use_variable". In such a function fn.threshold and +rec.field resolve to the PL/pgSQL datum and return before the qualified-name +rule runs, and a PL/pgSQL variable sharing a name with a pattern variable +takes A.price before the reservation above is reached. The identical text +in a function using the default resolution is rejected by both. + +This is not an oversight in the rules. use_variable redirects name +resolution wholesale -- it takes names a table column would otherwise own +too, which is why the default is to raise an ambiguity error instead -- and +a clause of one statement is not the place to carve an exception out of a +function-wide pragma. + +A lone name that resolves to a range variable is a whole-row reference with +no star to recognize it by, so that form alone is rejected only after it has +resolved. ExpandColumnRefStar() (parse_target.c) deliberately declines to +expand "something.*" in a DEFINE condition: expanding binds by RTE rather +than by name, so ROW(t.*) would slip past the checks above instead of +reaching them. It declines by reporting that it did not expand, and the +caller then hands the whole reference to transformExpr(), which is where the +star rejection comes from; NIL would not say that, being also what expanding +a relation with no columns returns. + +Where it declines is after both ref hooks have had their shot, and that is +the point of deciding there. A name a hook owns is not a reference to a +FROM-clause relation and keeps the expansion it has everywhere else. +Withholding it would not reject such a name -- none of the rules above has +anything to say about one the query parser never resolves -- it would leave +transformColumnRef() to read "rec.*" as the single whole value "rec", which +is a different condition rather than a refused one, and differs silently +wherever a row constructor is not counting its entries. The DEFINE scope +is read off the parse state, not off any exprKind, since a clause nested in +the condition carries a kind of its own. + +Parentheses turn a qualified name into field selection on a value, and that +form stays available: "(x).f" and ROW((x).*) reach a composite parameter or +record variable without occupying the qualifier slot. They are not a way +around the rules above -- a parenthesized range variable is still a +whole-row reference and is still rejected -- which is why the A_Indirection +arm of transformExpressionList() needs no DEFINE test of its own. + +III-5. Navigation Name Resolution + +Inside a DEFINE clause the names PREV, NEXT, FIRST and LAST denote row +pattern navigation rather than functions. ParseFuncOrColumn() +(parse_func.c) recognizes them before any catalog lookup, and only for an +unqualified call written in function syntax somewhere in a DEFINE condition; +column syntax and CALL are excluded. Once the name matches there is no +fallback to function resolution, so an ordinary function of one of these +names is reachable only through a schema-qualified call. That is also why +the deparser has to force-qualify such a function name (XIII-6). + +Skipping the lookup does not skip the shared checks. The recognized name is +carried through the wrong-kind-of-routine and decoration checks as if it +were an ordinary function, so DISTINCT, WITHIN GROUP, ORDER BY, FILTER, OVER +and RESPECT/IGNORE NULLS keep their usual messages, and only then is the +RPRNavExpr built. What that path lets through for a plain function -- +VARIADIC, named arguments, and any argument count other than one or two -- +is rejected with dedicated errors. Anything added to the shared path has to +stay safe to run on a name that will never reach the catalog. + +III-6. Query Jumbling + +A DEFINE clause is a list of TargetEntry whose resname carries the variable +being defined, and TargetEntry.resname is query_jumble_ignore everywhere +else in the tree. Left at that, "DEFINE A AS p > 50, B AS p < 50" and +"DEFINE B AS p > 50, A AS p < 50" would jumble alike and share one +pg_stat_statements entry, though they are different queries. +WindowClause.defineClause is therefore declared custom_query_jumble and +handled by hand in queryjumblefuncs.c: the custom function first jumbles the +list exactly as the generated code would and then appends each entry's +resname. A window with no DEFINE clause has an empty list, so the second +step contributes nothing and its query id is unchanged. + +The other RPR node fields follow the usual rule. RPRNavExpr.navno is +query_jumble_ignore because the planner assigns it (VII-6), and +RPRPatternNode.trailing_alt never survives parsing (III-2). + +Chapter IV Planning the DEFINE Clause +============================================================================ + +The compilation of Chapter V is only part of what the planner does with a +row pattern window. The rest is spread over the code that handles window +clauses in general, and it exists because DEFINE is the one window clause +field that carries an expression tree of its own. This chapter collects +those contracts. Each is a rule that a change elsewhere in the planner can +break without any RPR-specific code being touched. + +IV-1. Ownership of the DEFINE Expression Tree + +defineClause is the one field in which a window clause owns an expression +tree. partitionClause and orderClause hold only SortGroupClause references +into the query targetlist, and a frame offset may not contain Vars, so for +every other window clause field a Query-wide scan sees the expressions +through the targetlist. The DEFINE expressions are reached only through the +window clause itself. + +query_tree_walker() and query_tree_mutator() (nodeFuncs.c) therefore descend +into wc->defineClause in both branches, the QTW_EXAMINE_SORTGROUP one and +the plain one, and expression_tree_walker() descends into it from +T_WindowClause. Every Query-wide rewriter thus reaches the Vars a DEFINE +clause holds, and cannot tell a dead window's Vars from a live one's. Join +removal, for instance, deletes a relid from the whole parse tree once +nothing needs the relation, which requires that no ordinary Var of it be +left anywhere. + +Whoever decides that a window clause will not be executed is therefore +responsible for emptying defineClause at that moment, rather than expecting +later scans of the tree to skip it. That decision is made in one place, +grouping_planner() (planner.c): it empties defineClause in every window +clause that is not in activeWindows, which also covers a query whose window +functions were all folded away and so has no active window at all. The +generic walkers are not the only reader: build_base_rel_tlists() marks what +a DEFINE clause reads as needed at relation 0 (IV-6), and a column needed +by nothing that runs would keep an outer join from being removed. +grouping_planner() runs for a subquery too, when that subquery is planned, +and so before the subquery's own query_planner() builds its relation +targetlists or removes any join. Planning the outer query does not touch a +subquery's DEFINE clause (IV-6). Only defineClause is cleared, never the +WindowClause itself: winref is a one-based index into windowClause, so the +list must keep its length and its order. rpPattern stays as well. It +holds no Vars, an undefined pattern variable simply matches TRUE, and it is +what marks the clause as a row pattern window (IV-5). + +IV-2. Navigation Arguments, Subquery Pull-up and Join Aliases + +A navigation's argument is evaluated at the row the navigation lands on, not +at the row being tested (VII-3). When subquery pull-up substitutes a +subquery output expression into a DEFINE clause, a replacement that does not +depend on the row -- a constant, or anything built only from values outside +the subquery -- must not simply be folded into that argument, or PREV(x) +would collapse to a value that no longer varies with the row it is read at. + +replace_rte_variables_mutator() (rewriteManip.c) therefore handles RPRNavExpr +itself instead of leaving it to the generic mutator: it sets in_rpr_nav_arg +while it walks arg and restores the previous value before walking offset_arg +and compound_offset_arg, which are ordinary expressions read at the current +row. pullup_replace_vars_callback() (prepjointree.c) adds that flag to the +conditions that make a replacement a wrapping candidate, beside +varnullingrels and the caller's own wrap option. A replacement that is +itself a simple Var or PlaceHolderVar still escapes the wrapper, as does an +expression that contains the subquery's Vars and no non-strict constructs; +this costs nothing since such a replacement already varies with the row. +Anything else -- including the row-independent expression that would +otherwise be folded into the argument -- comes back wrapped in a +PlaceHolderVar. + +The pull-up does not reach the DEFINE clause through query_tree_mutator(). +perform_pullup_replace_vars() (prepjointree.c) names each part of the upper +query it rewrites, and among them a loop over windowClause passes every +non-empty defineClause to pullup_replace_vars(). subquery_planner() does +the same for flatten_group_exprs() (IV-4). The guarantee of IV-1 covers +only the generic walkers; a pass that rewrites the upper query field by +field has to list defineClause itself. + +Join alias expansion substitutes into a DEFINE clause as well: a Var naming +a join output column is replaced by the expression behind it. +flatten_join_alias_vars_mutator() (var.c) carries its own copy of the same +protection. It handles RPRNavExpr itself and sets in_rpr_nav_arg over arg +alone, as replace_rte_variables_mutator() does, and while that flag is set +it wraps a replacement in a PlaceHolderVar unless it is a Var or a +PlaceHolderVar of the same level. Unlike pullup_replace_vars_callback(), +it wraps a strict expression over row Vars as well. +It wraps only when it has a PlannerInfo; flatten_join_alias_for_parser() +passes none and gets the replacement unwrapped. + +IV-3. Preprocessing a DEFINE Clause as a Qual + +A DEFINE condition is a search condition in the sense WHERE is one: the +parser coerces it to boolean through transformWhereClause() (III-3), and a +row for which it yields FALSE or NULL is not a match. subquery_planner() +(planner.c) therefore hands each condition to preprocess_expression() as +EXPRKIND_QUAL, in the loop that preprocesses the window clause's frame +offsets. That is after subquery pull-up, and before the GROUP Vars of +IV-4 are expanded and before grouping_planner() empties the DEFINE clause +of a window it will not run (IV-1). + +As a qual the condition goes through every step a WHERE clause does. Join +alias Vars are flattened first (IV-2). eval_const_expressions() then folds +constants, inlines SQL functions and inserts the actual values of default +arguments. canonicalize_qual() runs with is_check false, so a NULL +constant at the top level of an AND or OR is treated as FALSE, which is +sound only because a NULL result and a FALSE one are told apart nowhere +downstream. Uplevel Vars become Params, and the result comes back as an +implicit-AND list. A TargetEntry holds one expression, so the loop turns +the list straight back into one with make_ands_explicit(); a condition that +folds to TRUE is left as a constant TRUE. + +The NFA evaluates a DEFINE at most once per row and caches the result +(VII-3), re-evaluating only the match_start dependent variables of VII-5. +The number of evaluations is not something a query can rely on, so a +volatile function is rejected. The check runs right after the +preprocessing, over the window clause's whole defineClause, and raises +"DEFINE clause cannot contain volatile functions". +It sits in the planner rather than in parse analysis to follow the +convention of not checking expression volatility while parsing, and running +after folding matters both ways. Folding can remove a volatile call -- a +dead CASE arm, or a VOLATILE SQL function whose body inlines to a constant +-- and it can bring in one parse analysis never saw: a STABLE function +whose default argument is volatile gets that argument spliced in here +(rpr_off_leak() in rpr_base.sql). The check sees only what is still there +when this subquery is planned. A window dropped with a subquery that +pull-up flattens, and a subquery the planner never plans, are not checked, +which is the same rule that lets a volatile fold away. A window that no +window function references is checked all the same: grouping_planner() +empties its DEFINE clause (IV-1) only after the check has run. In a +subquery kept by OFFSET 0 such a window's volatile DEFINE is therefore +rejected, as it is at the top level. + +A volatile expression the DEFINE clause shares with GROUP BY passes the +check, and rightly so. Parse analysis has replaced the DEFINE copy with a +GROUP Var (IV-4), and although that Var is expanded back after the check, +setrefs.c resolves the expansion against the grouping step's output: the +pattern match reads the value GROUP BY computed once per input row, and no +volatile evaluation is left in the DEFINE clause. + +Downstream the condition is an ordinary expression again. +set_upper_references() (setrefs.c) fixes its Vars to OUTER_VAR (IV-7). +ExecInitWindowAgg() (nodeWindowAgg.c) splits it back into an implicit-AND +list with make_ands_implicit() and compiles it with ExecInitQual(), which +builds no ExprState at all for a constant TRUE. nfa_eval_var_match() +(execRPR.c) evaluates it with ExecQual() in rprContext (VII-3), so a NULL +result is no match, and a missing ExprState is always a match. + +The compiled conditions go into defineClauseExprs in DEFINE order, and the +list index is the varId, because buildRPRPattern() (rpr.c) enters the +DEFINE names into varNames in that order before it scans PATTERN. Nothing +between the two checks that index. Every step that touches the list, this +preprocessing included, keeps its order today, but a reorder would +evaluate one variable's condition for another and give a wrong answer with +nothing to show for it. ExecInitWindowAgg() therefore asserts, for each +entry, that its index is below numVars and that varNames holds the entry's +resname at that index. + +IV-4. A DEFINE Clause That Takes Part in Grouping + +A DEFINE expression may reference a GROUP BY column, so parseCheckAggregates() +puts GROUP Vars into defineClause the same way it puts them into the +targetlist, and subquery_planner() expands them again with +flatten_group_exprs(). It passes the same root it passes for the targetlist, +so that the varnullingrels a grouping set attached survive onto the +replacement. + +Expanding them separately is not an option. set_upper_references() later +matches the DEFINE copy of an expression against the targetlist copy and +insists the two agree; a mismatch is not a wrong answer at runtime but a +"variable not found in subplan target list" failure at plan time. + +This is also why make_window_input_target() stops at a subexpression the +input target already computes whole. The grouping step produces only that +expression, so asking for the Vars underneath it would ask the grouping step +for columns it cannot produce. This is the one rule the havingQual handling +it is modelled on does not need: make_group_input_target() builds the input +of the grouping step, while the window input target sits above it. + +IV-5. Optimizations an RPR Window Is Excluded From + +Two prosupport-driven optimizations skip a row pattern window. Both +recognize it by a non-null rpPattern, not by a non-empty defineClause, +which grouping_planner() may have emptied (IV-1). + +find_window_run_conditions() (allpaths.c) declines to push a run condition +down. A run condition stops evaluating a monotonic window function once the +qual can no longer be satisfied; but in a row pattern window the partition is +divided into reduced frames, and each one has to be evaluated to the end of +the partition, so an early stop would cut a later reduced frame short. + +optimize_window_clauses() (planner.c) does not offer the clause to the +window functions' support functions at all. A support function is free to +propose any frameOptions it likes, and RPR requires the frame shape of +Chapter I; skipping the clause outright is what keeps one from replacing the +frame with a shape RPR cannot run. The duplicate-clause check that follows +a successful rewrite still compares rpSkipTo, defineClause and rpPattern. + +IV-6. Keeping a DEFINE-only Column Alive + +A column that nothing reads except a DEFINE expression still has to reach +the WindowAgg, and nothing in subquery pruning has to protect it for that. +A DEFINE clause reads the column of a relation in its own query's range +table, through a Var of its own level; it never reads the subquery output +entry that may carry the same column upward. build_base_rel_tlists() +(initsplan.c) marks every Var and PlaceHolderVar the clause holds as needed +at relation 0, so the column is carried to the top of the join tree, and +make_window_input_target() (planner.c) adds it to the WindowAgg's input +target. When remove_unused_subquery_outputs() (allpaths.c), planning the +outer query, replaces an output entry no outer reference reads with a NULL +constant, that changes what the subquery returns and not what its DEFINE +clause sees. The function has no row pattern code. + +When the relation a DEFINE clause reads is itself a subquery, the Var names +one of that subquery's output columns, and the mark build_base_rel_tlists() +made puts it in the relation's reltarget. remove_unused_subquery_outputs() +takes attrs_used from that reltarget, so the column is kept like any other +the upper query reads. + +A window clause that will not run gets no such mark: grouping_planner() +empties its DEFINE clause before query_planner() calls +build_base_rel_tlists() (IV-1). + +IV-7. Var Fixup and Costing + +The DEFINE expressions cross the plan-node interface inside the WindowAgg, +so set_upper_references() (setrefs.c) rewrites their Vars to OUTER_VAR +alongside the node's targetlist and qual. That is what lets the NFA +evaluate a DEFINE against the outer tuple slot (VII-3). The generic plan +walk does not reach a window clause's own expressions, so the fixup is +spelled out for defineClause by hand. + +SS_finalize_plan() has the same blind spot and the same fix. A DEFINE +expression may carry a Param -- a navigation offset settled per scan is the +case that matters (VII-6) -- and the parameter sets a plan node advertises are +collected by walking its expressions. The T_WindowAgg arm therefore runs +finalize_primnode() over defineClause beside startOffset and endOffset, so +that a DEFINE Param reaches extParam and allParam. A parameter missing from +those sets is one that does not force a rescan when it changes. + +Costing sees an RPR window only through the WindowClause. When rpPattern is +set, cost_windowagg() (costsize.c) charges each DEFINE expression's per-tuple +cost once per input tuple, once for every DEFINE variable, on top of the +window functions' own costs. That is an upper bound, since the lazy +evaluation of VII-3 skips a variable no active state tests at that row, but it +keeps an expensive DEFINE visible to the choice of plan below the WindowAgg. + + +Chapter V Compilation Phase +============================================================================ + +V-1. Entry Point + + create_windowagg_plan() (createplan.c) + +-- buildRPRPattern() NFA compilation (6 phases) + +V-2. The 6 Phases of buildRPRPattern() + + Phase 1: parse tree optimization (optimizeRPRPattern) + Phase 2: Statistics collection (scanRPRPattern) + Phase 3: Memory allocation (makeRPRPattern) + Phase 4: NFA element fill (fillRPRPattern) + Phase 5: Finalization (finalizeRPRPattern) + Phase 6: Absorbability analysis (computeAbsorbability) + +Phase 2 also rejects a pattern too large for the element array or nested too +deep. The depth test is made on entry to each node, before the children +raise the counter, so the one-byte RPRDepth can never wrap. Neither limit +depends on the input rows: a pattern that compiles once compiles every time. + +V-3. Phase 1: Parse Tree Optimization + +After copying the parser-generated parse tree, the following optimizations are +applied. + +A rewrite must keep not only the set of matchable lengths but also which +match is preferred. Several of them can move a repetition's exit decision +past a choice point inside the body, which leftmost-choice-first (7.2) makes +observable, so they are gated on the body consuming a fixed number of rows. +Alternatives of equal length count as fixed: they pick different variables, +never different rows. + +RPR_QUANTITY_INF is a sentinel meaning unbounded, not a count, and the +rewrites keep it one. A min is always finite and a max is at least 1, +so the sentinel can appear only as a max; Phase 4 asserts that on every +element it writes. A rewrite that computes a new bound therefore +declines rather than store a finite result that reached the sentinel: +(b), (c) and (g) leave the nodes unmerged when a sum or product +overflows int32 or lands on RPR_QUANTITY_INF, and (e) leaves the group +alone when one more iteration would. Declining is always safe, since +the unrewritten form matches the same rows in the same order, which is +what lets these passes treat overflow as a fallback rather than as an +error. + + (a) SEQ flattening: Unwrap nested SEQ nodes + SEQ(A, SEQ(B, C)) -> SEQ(A, B, C) + + (b) Consecutive variable merging: Merge consecutive occurrences of the + same variable into a single quantifier + A A -> A{2} + A{2,3} A{1,2} -> A{3,5} + + (c) Consecutive group merging: Merge repeated identical groups whose + body consumes a fixed number of rows + (A B)+ (A B)+ -> (A B){2,INF} + (A | B B)+ (A | B B)+ stays as-is (how the iterations split between + the two groups is a choice the merged form does not have) + + (d) Consecutive ALT merging: Merge repeated identical ALT nodes + (A | B) (A | B) (A | B) -> (A | B){3} + + (e) Prefix/suffix merging: Merge an identical sequence before or after + a group. A prefix copy is mandatory and comes before the group, so + it always merges; a suffix copy merges only when the body consumes a + fixed number of rows. + A B (A B)+ -> (A B){2,INF} + + (f) ALT flattening and deduplication + (A | (B | C)) -> (A | B | C) + (A | B | A) -> (A | B) + + (g) Quantifier multiplication: Collapse nested quantifiers when the + achievable counts are contiguous and the preferred match does not + move + (A+)+ -> A+ + (A{2,3}){5} -> A{10,15} + (A{2,})* stays as-is (count 1 unreachable; A* would be wrong) + (A{2,3}){1,2} stays as-is (counts contiguous, but the nested form + settles the first iteration first and prefers three rows where + A{2,6} takes four) + + (h) Single-child unwrap + SEQ(A) -> A, ALT(A) -> A, (A){1,1} -> A + A group whose only child is an unquantified variable hands its own + quantifier to that child instead: (A)+? -> A+? + + (i) Reluctance normalization: a fixed count leaves reluctance nothing + to decide, so it is cleared wherever min == max + A{2}? -> A{2}, (A B){2}? -> (A B){2} + This runs before and after each node's own rewrites. (b), (c), (e) + and (g) decline a reluctant node outright, and (c) to (f) compare + nodes with equal(), which treats a reluctant node as unequal to its + greedy twin. Without the normalization (A{2}?){3} would stay nested + where (A{2}){3} collapses to A{6}, and (A{2}? | A{2}) would keep both + branches. The absorbability analysis of V-7 reads the flag as well, + on a group's BEGIN and with no bound test, so a fixed-count reluctant + group becomes absorbable exactly where its greedy spelling already was. + +V-4. Phase 4: NFA Element Array Generation + +Transforms the optimized parse tree into a flat array of RPRPatternElement. +This is the core data structure used for NFA simulation at runtime. + +Phases 2 and 4 walk the same tree and must agree on its size: Phase 3 +allocates exactly what Phase 2 counted, and Phase 4 fills the array +with no bound test of its own. The count is one element per VAR, a +BEGIN and an END for each GROUP whose quantifier is not {1,1}, an ALT +plus one SEP per branch for each alternation, and one FIN for the whole +pattern; a SEQ contributes nothing. A new element kind therefore has +to be added to scanRPRPattern() and fillRPRPattern() together, and so +does any change to the depth an element is given. + +The rule is that a node's own markers sit at the depth the node was +given and its contents one level deeper: BEGIN and END at the group's +depth with the body at depth + 1, ALT and every SEP at the alternation's +depth with each branch at depth + 1. A SEQ adds no level, and a VAR +sits at the depth it was given. counts[] (VI-1) is indexed by depth, so +a group's count lives at its markers' depth and a VAR's at its own; an +alternation keeps no count, but its branches still add a level to +maxDepth. The first element after a group body that is shallower than +the body is the group's END, which is how isUnboundedStart() finds where +the group closes and where isFixedLengthChildren() stops. + +RPRPatternElement struct (16 bytes): + + Field Size Description + --------------------------------------------------------- + varId 1B Variable ID (0-0xEF) or control code (0xFB-0xFF) + depth 1B Group nesting depth + flags 1B Bit flags (see below) + reserved 1B Padding + min 4B Quantifier lower bound + max 4B Quantifier upper bound + next 2B Next element index (sequential flow) + jump 2B Branch link (ALT/SEP), own END (BEGIN), loop-back (END) + +Pattern variables occupy varId 0 to RPR_VARID_MAX (0xEF) inclusive, +giving 240 distinct variables. Any varId with the high nibble set +(0xF0-0xFF) is reserved for control elements; 0xF0-0xFA are currently +spare. + +Control codes: + + RPR_VARID_BEGIN (0xFB) Group start marker + RPR_VARID_END (0xFC) Group end marker + RPR_VARID_ALT (0xFD) Alternation start marker + RPR_VARID_SEP (0xFE) Alternation branch separator + RPR_VARID_FIN (0xFF) Pattern completion marker + +Element flags (1 byte, bitmask): + + 0x01 RPR_ELEM_RELUCTANT (VAR, BEGIN, END) + Non-greedy quantifier. Prefers shorter match: try exit-loop + first, then repeat. Set on VAR for simple (A+?), + on BEGIN+END for group ((...)+?). + + 0x02 RPR_ELEM_EMPTY_LOOP (END) + Group body can produce empty match (all children nullable). + Creates a fast-forward exit clone alongside the normal + loop-back below min. (V-6) + + 0x04 RPR_ELEM_EMPTY_PREFERRED (END) + Group body prefers the empty match over a consuming one. + Orders the fast-forward ahead of the loop-back below min, + where the group's own greed says nothing. (V-6) + + 0x08 RPR_ELEM_ABSORBABLE_BRANCH (VAR, BEGIN, END, ALT) + Element lies within an absorbable region. Used at runtime to + track whether the current NFA state is in an absorbable + context. See "V-7. Absorbability Analysis" and + "IX-2. Solution: Context Absorption" for more details about + absorption. + + 0x10 RPR_ELEM_ABSORBABLE (VAR, END) + Absorption comparison point. Where to compare consecutive + iterations for absorption. + - Simple unbounded VAR (A+): set on the VAR itself + - Unbounded GROUP ((A B)+): set on the END element only + + Accessor macros: + RPRElemIsReluctant(e) (e)->flags & 0x01 + RPRElemCanEmptyLoop(e) (e)->flags & 0x02 + RPRElemIsEmptyPreferred(e) (e)->flags & 0x04 + RPRElemIsAbsorbableBranch(e) (e)->flags & 0x08 + RPRElemIsAbsorbable(e) (e)->flags & 0x10 + +Roles of next and jump: + + - next: The next element to move to "after consuming" the current element. + For VAR, the next position after a successful match. + For BEGIN/END, the next position inside/outside the group. + For ALT, the first branch's content; for SEP, the next branch's + content (post-ALT on the last SEP). + + - jump: A second link, whose target depends on the element kind. + In ALT, the first branch's terminating SEP. + In SEP, the branch link to the next branch's SEP (-1 on the last). + In BEGIN, the group's own END. A group with min=0 is skipped + through that END's next, without arriving at the END. + In END, a loop-back to the start of the group body. + +The examples below build up in order: a GROUP, then an ALT (which introduces +the SEP branch-separator markers), then the two combined. + +Example: PATTERN ((A B)+) -- GROUP + + idx varId depth min max next jump Description + -------------------------------------------------------------- + 0 BEGIN 0 1 INF 1 3 Group start + 1 A(0) 1 1 1 2 -1 A + 2 B(1) 1 1 1 3 -1 B + 3 END 0 1 INF 4 1 Group end + 4 FIN 0 1 1 -1 -1 Pattern completion + + - idx 0: BEGIN. next(=1) enters the group body. + jump(=3) is the group's END. When min=0 the skip path leaves + through END.next (=4, FIN) without arriving at the END. + - idx 3: END. next(=4) exits the group. + jump(=1) loops back to the start of the group body. + +Example: PATTERN (A+ B | C) -- ALT + + Parse tree: ALT(SEQ(VAR(A,1,INF), VAR(B,1,1)), VAR(C,1,1)) + + Compilation result: + + idx varId depth min max next jump Description + ------------------------------------------------------------ + 0 ALT 0 1 1 1 3 Alternation start + 1 A(0) 1 1 INF 2 -1 Branch 1: A+ + 2 B(1) 1 1 1 6 -1 Branch 1: B -> FIN + 3 SEP 0 1 1 4 5 Branch 1 terminator + 4 C(2) 1 1 1 6 -1 Branch 2: C -> FIN + 5 SEP 0 1 1 6 -1 Branch 2 terminator (last) + 6 FIN 0 1 1 -1 -1 Pattern completion + + - idx 0: ALT marker. next(=1) enters branch 1's content, jump(=3) is + branch 1's terminating SEP + - idx 1: Variable A. next(=2) is B + - idx 2: Variable B. next(=6) is FIN (branch tail redirected past the ALT) + - idx 3: SEP. next(=4) enters branch 2's content, jump(=5) links to the + next branch's SEP + - idx 4: Variable C. next(=6) is FIN + - idx 5: SEP. jump(=-1) marks the last branch; next(=6) is post-ALT + - idx 6: FIN marker. Match completion signal + + Each branch is bounded by a trailing SEP, including the last, so a branch's + extent is fixed by its own SEP. The branch link runs from ALT through the + SEP chain, never through the branch content. That keeps it clear of a + branch that ends with a quantified group, whose BEGIN.jump is the group's + own END. + +Example: PATTERN ((B C)* | A) -- GROUP + ALT combined + + idx varId depth min max next jump Description + -------------------------------------------------------------- + 0 ALT 0 1 1 1 5 Alternation start + 1 BEGIN 1 0 INF 2 4 Branch 1: (B C)* -- jump is + 2 B(1) 2 1 1 3 -1 its own END; the skip + 3 C(2) 2 1 1 4 -1 leaves via END.next + 4 END 1 0 INF 8 2 Branch 1 tail -> post-ALT + 5 SEP 0 1 1 6 7 Branch 1 terminator + 6 A(0) 1 1 1 8 -1 Branch 2: A -> post-ALT + 7 SEP 0 1 1 8 -1 Branch 2 terminator (last) + 8 FIN 0 1 1 -1 -1 Pattern completion + + A branch-terminal optional group's skip path: + + The skip taken when an optional group matches zero times leaves through + END.next, not through "the element after END". Here the element after END + is the branch's own SEP terminator (idx 5). A SEP is a compile-time + boundary, never a runtime state; landing there would (via the default VAR + switch arm) mis-treat the SEP's varId as an always-true variable and consume + a spurious row, or fall through SEP.next into the next branch. END.next is + already redirected past the alternation with the rest of the branch tail + (idx 4's next is 8), so the zero-match skip ends the branch -- an empty + match, exactly as it would outside an alternation -- with no redirect of its + own. The same holds in the last branch, where the element after END is the + final SEP. + + BEGIN.jump names the group's END so that entering the group can mark that + END visited in one step (see X-6). + +V-5. Reluctant Flag (RPR_ELEM_RELUCTANT) + +The reluctant flag is set during Phase 4 (fillRPRPattern) from the parse +tree node's reluctant field. Phase 1 (i) has already cleared that field +wherever min == max, so A{2}? and (A B){2}? reach here as the plain +A{2} and (A B){2}. The flag reverses the priority of quantifier +expansion at runtime: + + Greedy (default): try loop-back first, then exit (prefer longer match) + Reluctant: try exit first, then loop-back (prefer shorter match) + +The flag is set on all elements that carry the quantifier: + + Simple VAR (A+?): RPR_ELEM_RELUCTANT on the VAR element + Group ((...)+?): RPR_ELEM_RELUCTANT on BEGIN and END elements + +At runtime (nfa_advance), the flag controls Depth-First Search +(DFS) exploration order: + + VAR with quantifier: (a VAR has no jump; looping stays on the element) + Greedy: primary path = stay (loop), second path = next (exit) + Reluctant: primary path = next (exit), second path = stay (loop) + + END element: + Greedy: primary path = jump (loop-back), second path = next (exit) + Reluctant: primary path = next (exit), second path = jump (loop-back) + + BEGIN with min=0: + Greedy: primary path = next (enter group), second path = END.next (skip) + Reluctant: primary path = END.next (skip), second path = next (enter group) + +The absorption optimization requires greedy quantifiers. Reluctant +quantifiers are excluded from absorbability analysis (see V-7). + +V-6. Empty Match Flags (RPR_ELEM_EMPTY_LOOP, RPR_ELEM_EMPTY_PREFERRED) + +Both flags are set during Phase 4 (fillRPRPatternGroup) on the END element +and describe the group's body, not the group itself. fillRPRPattern* +returns them for every node and the END inherits the body's pair: a +concatenation carries a bit only when every child does (AND), an +alternation is nullable when any branch is (OR) but takes its preference +from the first branch alone, since leftmost-choice-first makes that the +preferred one. + +RPR_ELEM_EMPTY_LOOP -- the body is nullable, i.e. every path through it +can match zero rows: + + (A? B?)+ Both children nullable -> body nullable -> END gets flag + (A | B*)+ B* is nullable, making the ALT nullable -> END gets flag + +Both examples keep their BEGIN/END pair through Phase 1. A single nullable +child, as in (A?)*, is multiplied away by (g) into A*, and an unquantified +group is unwrapped by (h), so neither leaves an END to carry the flag. + +It marks the END for the cycle detection of X-6, which only ever tests a +nullable END, and it lets nfa_advance_end offer a fast-forward exit beside +the loop-back below min, treating the remaining required iterations as +empty matches. + +RPR_ELEM_EMPTY_PREFERRED -- the body's preferred derivation is the empty +one, which is what orders those two paths. A bare variable consumes a +row; only a reluctant quantifier that may take zero repetitions prefers to +skip it. The group's own greed cannot decide this: in ((A? B?){2}) min +equals max, so the group has no choice of iteration count left to be +greedy or reluctant about, while the body still prefers to consume rows. +The flag is what distinguishes ((A? B?){2}) from ((A?? B??){2}). +(See X-4(c) for detailed runtime behavior.) + +V-7. Absorbability Analysis (RPR_ELEM_ABSORBABLE) + +Context absorption is an optimization technique that reduces O(n^2) to O(n). +(Runtime behavior is described in Chapter IX.) + +This phase determines whether the pattern has a structure suitable for the +absorption optimization and sets flags on the relevant elements: + + RPR_ELEM_ABSORBABLE Absorption comparison point + RPR_ELEM_ABSORBABLE_BRANCH Element within an absorbable region + +Eligibility conditions: + + (1) SKIP PAST LAST ROW (not NEXT ROW) + (2) Frame end is UNBOUNDED FOLLOWING + (3) No DEFINE variable depends on match_start (see IX-3(c)) + +Structural conditions (isUnboundedStart + computeAbsorbabilityRecursive): + + Case 1: Simple VAR+ (e.g., A+) + -> ABSORBABLE | ABSORBABLE_BRANCH set on the VAR + Case 2: GROUP+ with fixed-length children (min == max, recursively) + e.g., (A B)+, (A B{2})+, ((A (B C){2}){2})+ -> + ABSORBABLE_BRANCH on all elements within the group, + ABSORBABLE | ABSORBABLE_BRANCH on END + + Why this is safe: when every child has min == max, the group + is semantically equivalent to unrolling its body into {1,1} + elements. E.g., (A B{2})+ behaves like (A B B)+. Each + iteration consumes a fixed number of rows, so an earlier + context's count always dominates a later one's (monotonicity). + + Case 3: GROUP+ whose body starts with VAR+ (e.g., (A+ B)+) + -> Recurses from BEGIN into the body, applying Case 1. + ABSORBABLE | ABSORBABLE_BRANCH set on A, and + ABSORBABLE_BRANCH on the enclosing BEGIN. + B and END get no flags -> absorption stops once past A. + Case 4: ALT (e.g., A+ | B+) + -> Each branch, reached through the SEP chain, is judged on + its own by the cases above. The ALT gets + ABSORBABLE_BRANCH when any branch qualifies; SEPs get no + flags. e.g., A+ | C D (A+ only), ((A+ B) | C) D (A+). + (See B-7) + +The walk starts at element 0 and descends only through a greedy BEGIN +or an ALT at the start of a scope, so only the first element of each +scope can become a comparison point: A B+ gets no flags. Case 2 +rejects any ALT inside the group, even one whose branches have equal +length, so (A | B)+ gets no flags although V-3 counts such a body as +fixed. The walk then tries the ALT's branches, so (A+ | B)+ still +flags A+ by Case 1. + +A reluctant group disqualifies its whole subtree. computeAbsorbability- +Recursive() returns at such a BEGIN, so (A+ B)+? gets no flags at all +even though A+ itself is greedy: the group prefers its shorter +alternative, which is the opposite of the ordering absorption relies on. + +Absorbability is determined per-element, not per-pattern. +Absorption comparison is performed only when a state resides at an +element with the RPR_ELEM_ABSORBABLE flag. Once a state leaves the +flagged region, absorption is permanently disabled for that state. + +Through this mechanism, the runtime guarantees monotonicity: +"a context that started earlier always subsumes a context that +started later." + +Chapter VI NFA Runtime Data Structures +============================================================================ + +VI-1. RPRNFAState -- NFA State + +A single NFA state represents "how far the pattern has progressed." + + Field Description + ----------------------------------------------------------- + elemIdx Index of the current pattern element + counts[] Repetition count per group depth + isAbsorbable Whether the state is in an absorbable region + next Next state in the linked list + +The size of the counts array is rpPattern->maxDepth (= maximum group +nesting depth + 1), allocated as a flexible array member at the end of +the struct. + +Example: In PATTERN ((A B)+ C), a state waiting for B in the 3rd iteration + + Element array: [0:BEGIN(d0) 1:A(d1) 2:B(d1) 3:END(d0) 4:C(d0) 5:FIN] + + elemIdx = 2 (B, depth 1) + counts[0] = 2 (depth 0: depth of END. Group completed 2 iterations) + counts[1] = 0 (depth 1: shared by A and B. A zeroed its own slot when it + exited, per the count-clear policy, so a state parked on B + always enters with zero) + + Counts are indexed by depth, not by elemIdx. counts[0] is incremented when + passing through END(depth 0), and the group repetition count is preserved + even when the state is at B(depth 1). + +Definition of two states being "equal": + + Two states are equal if they have the same elemIdx and the same counts + up to the depth of that element. + nfa_states_equal() compares counts[0..elem->depth] using memcmp. + Only counts at or below the depth of the current element are meaningful. + +Counts saturate rather than wrap. Every increment goes through +RPRCountIncrement(), which stops at RPR_COUNT_INF; that value is +RPR_QUANTITY_INF (PG_INT32_MAX), so a saturated count compares as +unbounded in RPRElemCanLoop() and RPRElemWithinMax() exactly as an +unbounded max does. Two consequences follow: a saturated count no +longer distinguishes the states that reached it, so nfa_states_equal() +may collapse them, and the count-dominance test of IX-3 reads two +saturated counts as dominating each other. Both are harmless -- past +RPR_COUNT_INF iterations a quantifier can neither run out nor be +exceeded -- but a bare count++ added anywhere in the engine would +overflow int32 instead of settling there. + +That is the far end. Below it the same comparison is finer than the +future it stands for: where max is RPR_QUANTITY_INF, no decision in the +engine reads a count above min, so states differing only there behave +alike from that point on and this memcmp still keeps them apart. A +branching unbounded pattern that never reaches FIN pays for it, and the +XXX at nfa_states_equal() carries the cost and what a clamp would have +to preserve. + +VI-2. RPRNFAContext -- Matching Context + +A single context represents "a matching attempt started from a specific +start row." + + Field Description + --------------------------------------------------------------------- + states Linked list of active NFA states + matchStartRow Row number where matching started + matchEndRow Last row of the match; below matchStartRow for + an empty match, -1 before any match + lastProcessedRow Last row processed + matchedState State that reached FIN, or NULL; its being + non-NULL is what records a match (the state + itself is never read) + matchUpdated Whether the advance now running already recorded + a match (at most one per advance) + hasAbsorbableState Whether this context can absorb other contexts + allStatesAbsorbable Whether this context can be absorbed + next, prev Doubly-linked list + +Since the NFA is nondeterministic, multiple states can coexist +simultaneously within a single context. + +Example: In PATTERN ((A | B) C), if the first row matches both A and B, +two states coexist within the context: + + Element array: [0:ALT 1:A 2:SEP 3:B 4:SEP 5:C 6:FIN] + + State 1: elemIdx=5 (waiting for C, via branch A) + State 2: elemIdx=5 (waiting for C, via branch B) + +In this case, since the (elemIdx, counts) of the two states are equal, +nfa_append_state_unique() retains only State 1 (branch A), which was +added first. +Because DFS processes the first branch of ALT first, the state via A +is registered first, and the state via B is discarded as a duplicate. +This is the preferment guarantee. + +VI-3. RPR Fields of WindowAggState + + nfaContext / nfaContextTail Doubly-linked list of active contexts + nfaContextFree Reuse pool for contexts + nfaStateFree Reuse pool for states + nfaVarMatched Lazy per-row tri-state cache, varMatched[varId] + nfaVisitedEnds Nullable ENDs this DFS reached (cycle check) + nfaVisitedMinWord Lowest bitmapword index touched since reset + nfaVisitedMaxWord Highest bitmapword index touched since reset + nfaStateSize Precomputed size of RPRNFAState + defineMatchStartDependent DEFINE vars needing per-context evaluation + (match_start_dependent) + nfaLastProcessedRow Last row processed by NFA (-1 = none) + + EXPLAIN ANALYZE instrumentation counters are omitted here; see + execnodes.h for the full list. + + The active context list is ordered by matchStartRow, strictly ascending + from head to tail, with at most one context per start row: + ExecRPRStartContext() only appends at the tail and asserts that the new + start row is past the tail's. The driver loop appends the context for + currentPos + 1 after each row, and the on-demand path in + update_reduced_frame creates one only where none exists. Other code + relies on the head holding the smallest matchStartRow: the lookup in + update_reduced_frame (VII-1), the tuplestore mark (VII-6), the + nav_match_start the head context uses without invalidation (VII-5), and + the tail-to-head walk of absorption (IX-5). + +Memory management: + + States and contexts are managed through their own free lists. + Instead of palloc, they are obtained from the reuse pool, and + returned to the pool upon deallocation. + This reduces the overhead of frequent allocation/deallocation. + +Chapter VII NFA Execution: 3-Phase Model +============================================================================ + +VII-1. Entry Point and Overall Flow + +ExecWindowAgg() drives the match once for every row of the scan by +calling ensure_reduced_frame(), so the match tracks the row scan rather +than frame access. That function is idempotent: it calls +update_frameheadpos() and update_reduced_frame() only when +get_reduced_frame_status() reports the row as RF_NOT_DETERMINED. +row_is_in_reduced_frame(), which the frame access paths call to classify +a row, goes through ensure_reduced_frame() first and then reads the +recorded result. + +Flow of update_reduced_frame(): + + (1) Find or create a context for the target row + (2) Enter the row processing loop + (3) After the loop ends, record the match result + +Pseudocode of the row processing loop: + + targetCtx = NULL + if nfaContext != NULL: -- head is the oldest context + if nfaContext->matchStartRow > pos: -- pos already skipped past + return + if nfaContext->matchStartRow == pos: + targetCtx = nfaContext + if targetCtx == NULL: + if pos <= nfaLastProcessedRow: -- already unmatched or skipped + return + targetCtx = ExecRPRStartContext(pos) + + for currentPos = startPos; targetCtx->states != NULL; currentPos++: + if not rpr_prepare_row(currentPos): -- row does not exist + ExecRPRFinalizeAllContexts() -- finalize all contexts + ExecRPRCleanupDeadContexts() -- clean up after finalization + break + + ExecRPRProcessRow(currentPos) -- 3-phase processing + ExecRPRStartContext(currentPos + 1) -- pre-create next start point + ExecRPRCleanupDeadContexts() -- remove dead contexts + +Key point: Processing a single row may require processing multiple rows +ahead. Due to the nature of window functions, determining the frame for +row N requires looking at rows beyond N. + +Partition end: when rpr_prepare_row() reports no row at the scan +position, ExecRPRFinalizeAllContexts() stops every context that is still +running. It calls nfa_match() with a NULL varMatched, which fails every +VAR, then nfa_advance() to drain the epsilon transitions that failure +leaves behind. The point is uniformity: every context comes out with +states == NULL, so the cleanup that follows classifies them all by one +rule. Genuine FIN reaches have all been recorded in flight, so only +three shapes survive to this call: pure pursuit (matchedState == NULL), +which the forced mismatch turns into a failure; an empty-match candidate +whose VAR states are still chasing a longer match (matchedState != NULL, +matchEndRow < matchStartRow); and a recorded match whose states are +still looping for a longer one. + +ExecRPRCleanupDeadContexts() then frees every context left with no +states, with two exceptions. The context the caller passes as +excludeCtx is never freed: the caller still owns it and reads its result +from it. A context holding a recorded match is left to the SKIP logic, +and the test for that is matchedState, not matchEndRow -- an empty match +ends one row before matchStartRow (VII-2), so a row-length test would +take it for a failure and count it as pruned or mismatched. + +VII-2. Context Creation: ExecRPRStartContext() + +Creates a new context and performs the initial advance. + + (1) Allocate context via nfa_context_make() + (2) Set matchStartRow = pos + (3) Create initial state: elemIdx=0 (first pattern element), + counts=all zero + (4) Call nfa_advance() with currentPos = pos - 1 (no row consumed + yet) + +The initial advance expands epsilon transitions at the beginning of +the pattern. For example, the initial advance for PATTERN ((A | B) C): + + Start: elemIdx=0 (ALT) + -> Expand ALT branches + -> elemIdx=1 (A) -- VAR, so add state; stop here + -> elemIdx=3 (B) -- VAR, so add state; stop here + (element 2 is branch 1's SEP terminator) + + Result: Two states in the context {waiting for A, waiting for B} + +Reaching FIN during the initial advance is recorded like any other +match. Since currentPos is pos - 1, the record has matchEndRow < +matchStartRow: an empty match (RF_EMPTY_MATCH) if matchedState is set, +otherwise unmatched (RF_UNMATCHED). + +The quantifier flavors reach it differently. Reluctant min=0 (A*?, A??): +the skip path reaches FIN first and early termination prunes the enter +paths, so the empty match is final. Greedy (A*): the enter path adds its +VAR states before the skip path records FIN, so those states survive and +may match a longer span on a later row. + +VII-3. Row Preparation: rpr_prepare_row() + +Prepares the DEFINE evaluation context for the current row. DEFINE +predicates are NOT evaluated here; each variable is evaluated lazily the +first time the NFA consumes it (Phase 1 / nfa_eval_var_match), so a +variable that no active state tests at this row is never evaluated. + + fetch current row into temp_slot_1 -- return false if out of partition + set ecxt_outertuple = current row + invalidate nav_slot_pos + reset nfaVarMatched[] to RPR_VAR_UNEVALUATED + +nfaVarMatched is a tri-state array (RPRVarMatch): RPR_VAR_UNEVALUATED, +RPR_VAR_TRUE, or RPR_VAR_FALSE. nfa_eval_var_match() evaluates a +variable's DEFINE with ExecQual() on first consumption and caches the +result, so a NULL result is cached as RPR_VAR_FALSE (IV-3). The caller +(advance_reduced_frame_nfa) holds winstate->currentpos at the scan +position for the whole row (restored after the loop) because the deferred +navigation opcodes read currentpos. + +DEFINE evaluation runs in rprContext, a third ExprContext that +ExecInitWindowAgg() creates only for a window carrying a DEFINE clause. +It has to stay distinct from tmpcontext and from the output context +ps_ExprContext: nfa_eval_var_match() resets it before every predicate +evaluation, and a shared context would free the input or output tuple's +memory underneath its owner. A predicate leaves nothing behind that +outlives it -- the verdict is a by-value bool cached in nfaVarMatched -- +so no caller has to arrange the reset on its behalf. +ExecAssignExprContext() overwrites ps_ExprContext on each call, so the +DEFINE context is built between the two standard ones and the last call +is the one that establishes the output context. + +To support row navigation operators (PREV, NEXT, FIRST, LAST), a 1-slot +model is used: only ecxt_outertuple is set to the current +row. Navigation is handled by EEOP_RPR_NAV_SET/RESTORE opcodes +emitted during DEFINE expression compilation: + + NAV_SET: save ecxt_outertuple, swap in target row via nav_slot + (evaluate): argument expression reads from swapped slot + NAV_RESTORE: restore original ecxt_outertuple + +The pair is not reentrant. SET keeps the slot it swaps out in a single +field, winstate->nav_saved_outertuple, and fetches the target row into +the single nav_slot, so a navigation evaluated inside another one's +argument would overwrite both, and the outer RESTORE would put back +nav_slot rather than the current row. The executor relies on +define_walker() (III-3) never letting one RPRNavExpr reach the argument +of another: PREV or NEXT over FIRST or LAST is flattened into one +compound RPRNavExpr, and every other nesting in an argument, such as +FIRST(PREV(x)), PREV(PREV(x)) or PREV(x + FIRST(y)), is rejected. + +Between SET and the argument the compiler plants an EEOP_JUMP_IF_NULL +step whose target is the RESTORE step, so a navigation to a row that +does not exist skips the argument expression entirely instead of +evaluating it against the wrong row. That makes the SET step's resnull +a contract rather than a by-product: it has to be written on every path +-- true when the target row is out of range, a definitive false when the +row exists -- because resnull may still carry a stale value from an +earlier evaluation of the same expression. + +Compound navigation (PREV(FIRST()), NEXT(FIRST()), PREV(LAST()), +NEXT(LAST())) is flattened by the parser into a single RPRNavExpr with +a compound kind (RPR_NAV_PREV_FIRST, etc.). The executor computes the +target position in two steps: first the inner reference point +(match_start + N or currentpos - N) with match-range validation, then +the outer adjustment (+/- M) with partition-range validation. If +either step is out of range, the result is NULL. + +The offsets are optional in the source text, and the default differs by +kind: PREV and NEXT default to 1, FIRST and LAST to 0, and every compound +kind defaults to inner 0 with outer 1, so PREV(FIRST(x)) reads the row +before the match start. RPRNavExpr leaves a missing offset as a NULL +offset_arg or compound_offset_arg rather than materializing a Const; +resolve_nav_offsets() supplies the kind's default once per scan (VII-6), +and RPRNavKind (primnodes.h) is what records which default applies. + +When the computed target is the current row -- LAST(expr), PREV(expr, 0) +and NEXT(expr, 0) all resolve there -- ExecEvalRPRNavSet() skips the +tuplestore fetch and the swap altogether and just reports the row as +present. It records nav_saved_outertuple first all the same, so that +EEOP_RPR_NAV_RESTORE stays a harmless no-op: RESTORE recognizes the +elision by finding ecxt_outertuple unchanged and returns immediately, +which is correct because the argument read the current row's slot rather +than nav_slot. + +nav_slot caches the last fetched position (nav_slot_pos) to avoid +redundant tuplestore lookups when multiple navigation calls target +the same row. + +That single slot is also why a navigation result cannot simply be left +where the argument produced it. A pass-by-reference result points into +nav_slot's tuple memory, and the next navigation in the same expression +frees that tuple when it re-fetches the slot for another position. +EEOP_RPR_NAV_RESTORE therefore copies a non-null pass-by-ref result into +ecxt_per_tuple_memory, which survives until the next ResetExprContext, +before it returns. Rather than look the type up on every evaluation, +ExecInitExprRec() reads the type length and by-value flag once from the +RPRNavExpr's resulttype and stores them in the shared RPRNavState as it +emits the RESTORE step; every compilation of the same navigation writes +the same pair, so repeating it is harmless. + +The nfaVarMatched entries are filled lazily during Phase 1 (Match) as +variables are consumed. + +VII-4. Slot Swap Consumers: Interpreter and JIT + +The swap happens in the middle of an already-compiled expression, so +everything that expression cached about the outer tuple goes stale at +that point and has to be refreshed: + + - Deformed columns. The expression's FETCHSOME step ran once, against + the original slot, so ExecEvalRPRNavSet() calls slot_getallattrs() + on the target slot before installing it. A narrower deform there + would leave the argument reading unset tts_values entries. + + - The caller's cached slot pointer. ExecEvalRPRNavSet() and + ExecEvalRPRNavRestore() update econtext->ecxt_outertuple only, so + every caller must reload its own copy afterwards. The interpreter + (execExprInterp.c) reassigns its local outerslot in both opcode + arms. + + - The JIT's entry-block loads. llvm_compile_expr() (llvmjit_expr.c) + normally loads tts_values and tts_isnull once in the entry block and + reuses them for every EEOP_OUTER_VAR. For an RPR window it first + scans the steps for EEOP_RPR_NAV_SET or EEOP_RPR_NAV_RESTORE, and + when it finds one, each EEOP_OUTER_VAR reloads the slot pointer from + econtext instead of using the cached entry-block values. + +The SET step reaches the tuplestore through ExecRPRNavGetSlot() +(nodeWindowAgg.h), the one executor entry point expression evaluation uses: +it bounds-checks the position against the partition and returns NULL for a +row outside it, which is what that step reports as a null result. RESTORE +fetches nothing. It puts back the slot SET swapped out, and copies a +pass-by-reference result into per-tuple memory so that a later fetch of +nav_slot cannot pull the tuple out from under it. + +Both evaluators run out of line, reached through build_EvalXFunc(), +which is why ExecEvalRPRNavSet and ExecEvalRPRNavRestore appear in +referenced_functions in llvmjit_types.c. + +DEFINE expressions are otherwise ordinary ExprStates, so a row pattern +window is JIT-compiled like any other; the RPR-specific part of +expression compilation is ExecInitExprRec()'s T_RPRNavExpr arm +(execExpr.c). + +VII-5. Per-Context Invalidation (match_start_dependent variables) + +DEFINE variables that depend on match_start -- those containing FIRST or a +compound PREV_FIRST/NEXT_FIRST, or a LAST that carries an offset of its own, +whether plain or inside a compound PREV_LAST/NEXT_LAST -- are identified at +plan time via defineMatchStartDependent. For the head context, +advance_reduced_frame_nfa sets nav_match_start to its matchStartRow before +matching, so lazy evaluation uses the correct +FIRST/LAST base position. + +When processing a context whose matchStartRow differs, +nfa_invalidate_dependent_vars() resets only the dependent variables to +RPR_VAR_UNEVALUATED so they are re-evaluated lazily against this context's +matchStartRow, installs nav_match_start to that value, and invalidates the +nav_slot cache. match_start-independent variables keep their cached value +across contexts (they do not read nav_match_start). + +nav_match_start is left installed and NOT restored: FIRST/LAST read it at +evaluation time, which happens later during nfa_match(); the next +context's invalidation or the next row's setup overwrites it. + +Summary of evaluation strategy by navigation content (a variable is +evaluated once per row and cached, except dependent ones which are +re-evaluated once per differing context): + + Navigation content evaluation + ------------------------------------------------------- + No navigation cached (once per row) + PREV/NEXT only cached (once per row) + LAST (no offset) cached (once per row) + LAST (with offset) per-context + FIRST (any) per-context + Compound (inner FIRST) per-context + Compound (inner LAST, no off.) cached (once per row) + Compound (inner LAST, w/off.) per-context + +VII-6. Tuplestore Mark and Trim (nodeWindowAgg.c) + +Navigation functions require access to past rows via the tuplestore. +To allow tuplestore_trim() to free rows that are no longer reachable, +the executor computes two offsets per scan (see resolve_nav_offsets; +build_define_offsets resolves the constant ones at init for EXPLAIN): + + navMaxOffset (Nav Mark Lookback): + Maximum backward reach from currentpos. Contributed by PREV, + LAST (any offset, including the default 0), and compound + PREV_LAST/NEXT_LAST. + Mark position: currentpos - navMaxOffset. + + navFirstOffset (Nav Mark Lookahead): + Minimum forward reach from match_start. Contributed by FIRST + and compound PREV_FIRST/NEXT_FIRST. Can be negative when + compound PREV_FIRST looks before match_start. + Mark position: oldest_context->matchStartRow + navFirstOffset. + +The actual mark is set to: min(lookback_mark, lookahead_mark). +This ensures all rows reachable by any navigation function are retained. + +The mark advance_nav_mark() moves belongs to nav_winobj, a WindowObject +that exists only for RPR navigation, with its own tuplestore read and +mark pointer pair allocated in prepare_tuplestore() and reset in +begin_partition(). Holding it apart from agg_winobj and from the +per-window-function objects is what lets the DEFINE clause's lookups +trim the tuplestore without moving anyone else's fetches. The mark only +ever advances, and every RPR window gets a nav_winobj whether or not its +DEFINE navigates at all. + +Apart from everyone else, but not from itself: the pair is one pair, and +both rpr_prepare_row() and ExecRPRNavGetSlot() reach the tuplestore +through it. The frontier row and the navigation target are not the same +row, so the two drag the single seekpos back and forth. In memory that +costs a pointer move; spilled it costs a re-read, and the XXX at +prepare_tuplestore() carries the measurement. + +Each RPRNavExpr carries a navno, its index into that list of offsets. +The parser leaves it -1; compute_define_metadata() +(optimizer/plan/createplan.c) numbers the navigations in walk order +while it classifies match_start dependency, and build_define_offsets() +fills rprNavOffsets by walking the same expressions in the same order, +so entry i is the navigation with navno i. The offsets live in +executor state rather than on the RPRNavExpr, the plan tree being +read-only, so the expression compiler looks the entry up by navno when +it emits the EEOP_RPR_NAV_SET/RESTORE pair. The two walks have to stay +in step: execExpr.c raises "RPRNavExpr navno %d out of range" instead +of indexing past the list, and rejects an entry that does not point +back at the same RPRNavExpr. + +What the compiler takes from the entry is its RPRNavState, the state +VII-3 calls shared. build_nav_offsets() creates one per navigation with +both offsets null, and the SET and RESTORE steps compiled for that +navigation both point at it, so build_define_offsets() has to run before +the ExecInitQual() loop over defineClause in ExecInitWindowAgg(). +resolve_one_nav() pins the inner and outer offsets, defaults included, +into it: at init for a navigation whose offsets are all Const +(for EXPLAIN), and again on every scan through resolve_nav_offsets(). +ExecEvalRPRNavSet() reads them from there, asserting they are set, +rather than evaluating the offset per row, so a rescan changes the +offsets the compiled DEFINE expressions use without recompiling them. + +When offsets contain non-constant expressions (Param), the executor sets +navMaxOffsetKind/navFirstOffsetKind to RPR_NAV_OFFSET_NEEDS_EVAL. A constant +offset is resolved at init, as is a bind parameter the planner folded to a +Const for a custom plan; under a generic plan that parameter stays a Param and +resolves per scan, as a PARAM_EXEC offset does. Either way every navigation is +settled again per scan by resolve_nav_offsets(), which is where a null or +negative offset is rejected. On a backward-reach overflow, navMaxOffsetKind is +set to RPR_NAV_OFFSET_RETAIN_ALL, disabling trim for that dimension. The +forward reach has no such sentinel: an overflowing FIRST reach clamps to +PG_INT64_MAX, which bounds nothing and which EXPLAIN prints as "infinite". An +offset that resolves negative is rejected at execution, so that navigation can +never run and is left out of both reaches; it behaves exactly as if it were not +in the DEFINE. Each dimension is reported only when some navigation feeds it +(hasMaxNav, hasFirstNav), so an empty one prints nothing rather than a reach of +zero. + +VII-7. ExecRPRProcessRow(): 3-Phase Processing + +NFA processing for a single row is divided into three phases: + + +--------------------------------------------+ + | Phase 1: MATCH (convergence) | + | Compare the current row against each VAR | + | state. Remove states that fail to match. | + | | + | Phase 2: ABSORB (absorption) | + | Merge duplicate contexts to prevent | + | state explosion. | + | | + | Phase 3: ADVANCE (expansion) | + | Expand epsilon transitions to prepare | + | for the next row. | + +--------------------------------------------+ + +This ordering is important: + + - Match executes first to "consume the current row." + - Absorb executes immediately after Match, when states have been updated. + - Advance executes last to prepare "states waiting for the next row." + +Chapter VIII Phase 1: Match +============================================================================ + +nfa_match() iterates through each state in the context. Every state it +sees is a VAR: the advance phase parks only VAR states, and a fresh context +is advanced before its first match. This is asserted, not checked at +runtime. For each state: + + (1) Compare against the current row using nfa_eval_var_match() + (2) Match success: increment repetition count, retain state + (3) Match failure: remove state + +Match determination (nfa_eval_var_match): + + If varId is within the range of defineClauseExprs: + Use the value of varMatched[varId], first evaluating the DEFINE + predicate if the entry is still RPR_VAR_UNEVALUATED (VII-3) + + If varId exceeds the range (variable not defined in DEFINE): + Unconditionally true (matches all rows) + +Immediate advance to the comparison point: + + For a VAR inside an absorbable region -- one carrying + RPR_ELEM_ABSORBABLE_BRANCH without being the comparison point itself -- + that has no iteration left (count >= max) and whose next element is END, + the Match phase advances through the END chain immediately. + This is necessary for accurate state comparison in Phase 2 (Absorb). + + Example: In PATTERN ((A B)+), A stays where it is when it matches, since + its next element is B; when B matches, the state advances through END so + that the group count is complete for the absorption comparison with + other contexts. + +Chapter IX Phase 2: Absorb (Context Absorption) +============================================================================ + +Absorption is the runtime optimization that collapses contexts which +have converged on identical future behavior. Two contexts are +treated as equivalent when one's bookkeeping (elemIdx and per-depth +iteration counts) is dominated by another's; the younger one is then +discarded. The optimization is safe because pattern matching is +monotonic -- an earlier context's reachable matches always contain a +later context's. This is what reduces the naive O(N^2) state count +to O(N). + +IX-1. Problem + +In the current implementation, a new context is started for each row +processed. Applying PATTERN (A+) to 10 rows produces 10 contexts, +each of which tracks state independently. + +If there are N rows, the total number of states becomes O(N^2): + + Context 1 (started at row 1): can match A up to N times + Context 2 (started at row 2): can match A up to N-1 times + ... + Context N (started at row N): can match A 1 time + +IX-2. Solution: Context Absorption + +Key observation: a context started earlier contains all matches +of a later-started context (monotonicity principle). + +If Context 1 started at row 1 and matched A 5 times, the state +where Context 2 (started at row 2) matched A 4 times is already +contained within Context 1. + +Therefore Context 2 can be "absorbed" into Context 1. + +Worked example for PATTERN (A+) over 3 rows (each matches A): + + After row 1: + Ctx_1 (started row 1): state at A with counts[0] = 1 + + After row 2: + Ctx_1: state at A with counts[0] = 2 + Ctx_2 (started row 2): state at A with counts[0] = 1 + -> Same elemIdx; Ctx_1.count (2) dominates Ctx_2.count (1). + -> Ctx_2 absorbed. + + After row 3: + Ctx_1: state at A with counts[0] = 3 + Ctx_3 (started row 3): state at A with counts[0] = 1 + -> Ctx_1.count (3) dominates Ctx_3.count (1). + -> Ctx_3 absorbed. + +Total active contexts stays at O(1) instead of growing with N. + +The monotonicity argument covers only a context's future (in-progress) +matches, not a match it has already recorded (matchedState) -- e.g., one +on a non-absorbable branch, which an absorbing context cannot reproduce. +Absorption therefore excludes any context holding a recorded match (see +nfa_update_absorption_flags()). This costs no efficiency: SKIP PAST LAST +ROW still prunes such redundant contexts once the covering match is +recorded (nfa_prune_skipped_contexts(), called from ExecRPRProcessRow()). + +IX-3. Absorption Conditions + +Planner-time prerequisites (all must hold for absorption to be enabled): + + (a) SKIP PAST LAST ROW. SKIP TO NEXT ROW creates overlapping + contexts that cannot be safely absorbed. + (b) Unbounded frame (ROWS BETWEEN CURRENT ROW AND UNBOUNDED + FOLLOWING). Limited frames apply differently to each context, + breaking the monotonicity principle. + (c) No match_start_dependent navigation in DEFINE. + + Mechanism: each context has a different matchStartRow, so FIRST + resolves to a different row for each context at the same + currentpos. An earlier context's DEFINE result no longer + subsumes a later one's, making count-dominance comparison + invalid. Rather than comparing matchStartRow at runtime (which + would complicate the absorb path), any match_start dependency + disables absorption entirely. + + Navigation content match_start dep. absorption + ------------------------------------------------------------ + No navigation none safe + PREV/NEXT only none safe + LAST (no offset) none safe + LAST (with offset) boundary check unsafe + FIRST (any) direct unsafe + Compound (inner FIRST) direct unsafe + Compound (inner LAST, no off.) none safe + Compound (inner LAST, w/off.) boundary check unsafe + + The "match_start dep." column classifies how the navigation ties a + DEFINE result to the context's matchStartRow: + + none Independent of matchStartRow. The result depends + only on currentpos (or a fixed offset from it), so + every context evaluates it identically. + direct Computed from matchStartRow itself -- FIRST counts + forward from match start -- so the resolved row, + and thus the result, differs per context. + boundary check The resolved row is currentpos-relative (LAST with + a backward offset, or a compound whose inner LAST + carries an offset), but its in-range test is taken + against the match range [matchStartRow, currentpos]. + The range bound differs per context, so the result + can too. + + Only "none" is safe for absorption; "direct" and "boundary check" + both make an earlier context's result stop subsuming a later one's + (see (c) above). + +Runtime conditions (evaluated per context pair): + + (1) The pattern is marked as isAbsorbable (see V-7) + (2) allStatesAbsorbable of the target context is true + (3) An earlier context "covers" all states of the target + +Cover condition (nfa_states_covered) -- "count-dominance": + + Every state of the later context must sit on an element carrying + RPR_ELEM_ABSORBABLE, the comparison point; a state anywhere else + makes the two contexts incomparable and nothing is covered. Such a + state is covered when the earlier context holds an absorbable state + with the same elemIdx whose count at that element's depth is greater + than or equal -- a same-elemIdx state that has already left the + absorbable region does not cover. The earlier context's iteration + count thus dominates the later one's; this is the count-dominance + comparison referenced in IX-3(c). + +IX-4. Dual-Flag Design + +Two boolean flags make the absorption decision efficient: + + hasAbsorbableState (monotonic: only true->false transition possible) + "Does this context have the ability to absorb other contexts?" + true if at least one absorbable state exists. + Transitions to false when states are removed leaving no absorbable + states. + Once false, it never becomes true again. + + allStatesAbsorbable (dynamic until a match is recorded) + "Can this context be absorbed?" + true if all states are in an absorbable region and no match is + recorded. + Becomes false when a non-absorbable state is added; reverts to true + when it is removed. Recording a match also sets it false and that + does not revert, since absorbing would free the match. + +IX-5. Absorption Order + +nfa_absorb_contexts() traverses from tail (newest) to head (oldest). + + for ctx = tail to head: + if ctx.allStatesAbsorbable: + for older = ctx.prev to head: + if older.hasAbsorbableState: + if nfa_states_covered(older, ctx): + free(ctx) -- absorbed + break + +Since inspection starts from the newest context, the most recently started +(= having the shortest match) context is absorbed first. + +Chapter X Phase 3: Advance (Epsilon Transition Expansion) +============================================================================ + +X-1. Overview + +nfa_advance() expands epsilon transitions from each state after Match, +generating "new states waiting for the next row." + +An epsilon transition is a transition that moves without consuming a row: + + - ALT: branch to each alternative + - BEGIN: enter group (or skip if min=0) + - END: loop-back within group (or exit when condition is met) + - FIN: record match completion + - VAR loop/exit: repeat/exit according to the quantifier + +Expansion stops upon reaching a VAR element, and the state is added. +This is because VAR is the element that "will consume the next row." + +X-2. Processing Order: DFS and Preferment + +advance processes states in lexicographic order, +performing Depth-First Search (DFS) on each state. + +This DFS order is what guarantees the SQL standard's "preferment": + + The branch that appears first in the PATTERN text takes precedence. + +Example: PATTERN (A | B) C + + The first branch A of the ALT takes precedence over the second branch B. + When both A and B can match, the match via A is selected. + +nfa_append_state_unique() prevents duplicate addition of the same state, +so the state added first (= from the preferred branch) is retained. + +X-3. Routing Function: nfa_route_to_elem() + +Most inter-element transitions in the advance phase go through +nfa_route_to_elem(), but three callers reach nfa_advance_state() +directly: nfa_advance() (each state it pops), nfa_advance_alt() (its +per-branch clone), and nfa_route_to_elem()'s own skip path. The result +matches either way -- for a bypassed VAR, nfa_advance_var()'s count-zero +handling produces what park-and-skip would -- so "a VAR not yet matched" +is handled in two places. + +nfa_route_to_elem() branches on the type of the next element: + + If the next element is VAR: + (1) Add the state to the context (nfa_append_state_unique) + (2) If the VAR has min=0, also add a skip path (recurse via next). + A reluctant VAR (A??, A*?) reverses the order: the skip path + goes first, and the waiting state is dropped if it reaches FIN + -> Expansion stops here (VAR is the element that "will consume the + next row") + + If the next element is non-VAR (ALT, BEGIN, END, FIN): + -> Recursively call nfa_advance_state() to continue expansion + +With this structure, advance recursively follows epsilon transitions +until reaching a VAR, consistently stopping only at VAR elements. + +X-4. Per-Element advance Behavior + +(a) ALT (nfa_advance_alt) + + Upon encountering an ALT element, branches are expanded in preference order + via the SEP branch-separator chain, stopping at the first branch that + records a match -- a later branch's FIN would replace the preferred one. + ALT.next is the first branch's content and ALT.jump is that branch's + terminating SEP; each SEP.jump links to the next branch's SEP (-1 on the + last) and each SEP.next is the next branch's content. + + ALT.next -> branch 1 content; ALT.jump -> SEP1 -> SEP2 -> ... (jump chain) + SEP_i.next -> branch (i+1) content + + nfa_advance_state() is recursively called at each branch's content. + +(b) BEGIN (nfa_advance_begin) + + Handles group entry. + jump names the group's own END. The skip path goes to END.next without + arriving at the END; END.next is the element after END, or the post-ALT + element when the group ends an alternation branch (V-4). + + BEGIN does not reset the count at its depth; it only asserts the slot + is already zero. Under the count-clear policy the previous occupant + of a depth slot clears it on the way out, so entry finds it clean. + See X-4(c) and VI-1. + + Greedy (default): + (1) Enter the group body (move via next) + (2) If min=0, also add a group skip path (move to END.next) + + Reluctant: + Order reversed -- skip path first, group entry second. + If the skip path reaches FIN, the group entry path is not generated + (shortest match preferred). + +(c) END (nfa_advance_end) + + Handles group termination. This is the core of the repetition logic. + + Let count be the count at the current depth: + + count < min: + Loop-back (move via jump, repeat the group body) + + If the RPR_ELEM_EMPTY_LOOP flag is set: + In addition to loop-back, also add a fast-forward exit path. This + is because the body may produce an empty match, causing count to + never reach min. fast-forward resets counts[depth] to 0 and exits + via next (treating the remaining required iterations as empty + matches). + + The body decides which of the two comes first, not the group's own + greed: the fast-forward is explored first exactly when + RPR_ELEM_EMPTY_PREFERRED is set (V-6). If it then reaches FIN, + the loop-back is dropped, as in the min <= count < max arm below. + + min <= count < max: + Greedy: loop-back first, exit second + Reluctant: exit first, loop-back second + If the exit path reaches FIN, loop-back is omitted. + + count >= max: + Unconditional exit (move via next) + + On exit: reset counts[depth] = 0, and if the next element is an outer END, + increment the count at the outer depth. + +(d) VAR (nfa_advance_var) + + Handles repeat/exit for a VAR element with a quantifier. + + Let count be the count at the current depth: + + count < min: + Unconditional loop (stay at the same elemIdx, wait for the next row) + + min <= count < max: + Greedy: loop first, exit (next) second + Reluctant: exit first, loop second + If the exit path reaches FIN, loop is omitted. + + count >= max: + Unconditional exit (move via next) + + On exit: reset counts[depth] = 0, and if the next element is an outer END, + increment the count at the outer depth, as in (c). + + Every exit and skip in the advance phase goes through + nfa_state_exit_to(), which clears the exited depth slot and increments + the count of an END it lands on: the exits of (c) and (d), the + fast-forward of (c), the min=0 VAR skip of X-3 and the group skip of + (b). A skip that lands on an outer END therefore counts one iteration + of that group, and the group's min check and the below-min + fall-through of X-6 read that count. The inline END chain of + nfa_match() (Chapter VIII) repeats the clear and increment by hand, so + the two must change together. + +(e) FIN + + Match success. The current state is moved to matchedState for recording, + and matchEndRow is set to the current row. + + Upon reaching FIN, all remaining unprocessed states are removed + (early termination). By DFS order, the path that reached FIN first + has the highest preferment, so the rest are inferior paths. + This is the core mechanism that guarantees preferment. + + The same rule holds inside one state's DFS. Wherever two + continuations are explored in order -- ALT branches, BEGIN's entry and + skip, END's loop-back and exit or fast-forward, a reluctant VAR's exit + and loop, an optional reluctant VAR's skip and entry -- the second is + explored only if the first did not record a match, in greedy as well + as reluctant order. matchUpdated stays set for the rest of the + advance, so every enclosing frame stops as well. Otherwise a later + continuation could reach FIN again in the same DFS (FIN is not marked + visited) or park a state that completes on a later row, and replace + the preferred match. nfa_add_matched_state() asserts that one advance + records at most one match, and nfa_append_state_unique() that nothing + is parked after it. + + In SKIP PAST LAST ROW mode the later contexts that started within the + match range become unreachable, but reaching FIN does not free them: + it only sets matchUpdated. ExecRPRProcessRow() reads that flag once + nfa_advance() has finished with the context and calls + nfa_prune_skipped_contexts() there. + +X-5. State Deduplication: nfa_append_state_unique() + +When adding a new state to a context, it is compared against existing +states; +if an identical state already exists, it is not added. + +Comparison criteria: elemIdx + counts[0..elem->depth] (see VI-1) + +This deduplication is the core mechanism that suppresses NFA state +explosion. +Because DFS order causes preferred-branch states to be added first, +identical states from lower-priority branches are automatically discarded. + +X-6. Cycle Detection: nfaVisitedEnds + +When a group body can produce an empty match, +looping back from END may cause an infinite loop. + +Example: PATTERN ((A? B?)+) + + A? and B? both have min=0, so the body can pass through without + matching. If the group repeats: BEGIN -> A? skip -> B? skip -> END -> + A? skip -> ... The loop-back re-enters the body directly, since + END.jump is the group's first child; BEGIN is visited only on initial + group entry. + +To prevent this: + + (1) At compile time: set the RPR_ELEM_EMPTY_LOOP flag on the END of + groups whose body is nullable. The runtime effect of this flag + is described in X-4(c): when count < min, a fast-forward exit + path is added, resolving the deadlock where count cannot + increase due to empty matches. + + (2) At runtime: initialize the nfaVisitedEnds bitmap immediately before + DFS expansion of each state within advance (once per state). During + DFS, an END carrying RPR_ELEM_EMPTY_LOOP is marked in two places, + and nothing else is: nfa_advance_state marks it on arrival, and + nfa_advance_begin marks it when the group is entered through its + BEGIN (BEGIN.jump names the END). Reaching a marked END means the + body derived an empty match for this iteration -- a DFS takes only + epsilon transitions, so no row was consumed since the last visit, or + since the group was entered. The state that reaches it is not + discarded: it leaves the group there, so that "leave the group" + keeps its rank among the alternatives (see X-4(c)). The entry mark + is what catches the first iteration: without it the first arrival + finds the END unmarked, an empty iteration at count >= min loops + back, and a derivation the standard excludes is kept -- + ((A? | B){1,2} C) over rows {B},{A,C},{C} then matched rows 1-2 + instead of rows 1-3. A loop-back needs no entry mark, since it + leaves the END it has just marked. + + Nothing else is marked, because nothing else can cycle. A revisit is + a cycle only when it carries no progress, and a state is really + (elemIdx, counts) -- the identity nfa_append_state_unique() compares. + A VAR consumes a row, and so does every derivation of a body that + cannot match empty; a loop-back into either is progress, not a cycle. + Marking them would discard legitimate re-entry and lose the match + outright: ((A | B B){1,3}){3} would then find no match at all. + + The guard is min-aware: a marked END exits only at count >= min (an + empty iteration at or above the lower bound stops the quantifier, + TR 19075-5 7.2.8). Below min the guard falls through to the normal + must-loop path instead, so the next iteration still runs and its + consuming branches park as usual -- a derivation that goes empty + first and consumes later stays reachable, and it is the preferred + one: ((A? | B){3} C) over rows {B},{A,C},{C} matches rows 1-2 by + going empty twice and taking B on the third iteration, not rows 1-3 + by taking B on the first. This terminates because every arrival at + an END increments its count, so the fall-through reaches min in + bounded steps and then exits through the guard. + + The count advances one empty iteration at a time, and each step is a + recursion, so the cost follows the lower bound -- which the grammar + caps only at RPR_QUANTITY_INF. The steps cannot be collapsed by + setting the count to the bound directly: the intermediate counts are + the enumeration order, and each one parks the body's consuming + branches at a rank of its own. ((A? | B){4} C) over rows + {B},{A},{A,C},{C} matches rows 1-3, which needs the loop-back at + count 3 specifically. + + That bound is on the count, not on the expansion. Within one + expansion only a nullable END is marked, so ALT and BEGIN may be + re-entered any number of times; where several alternations in a row + have nullable branches, the expansion enumerates paths instead of + states and costs 2^k for k such alternations. See the XXX at the + marking site in nfa_advance_state(). + + Marks are never cleared during a DFS. Clearing the body's marks on a + below-min loop-back would disarm the guard for any nested reluctant + unbounded quantifier inside the body, whose empty iterations then + recurse without bound: (A (B*?)+?){2,} on a single matching row. The + outer count does not advance while the inner one spins, so "the count + stops at min" is not on its own a termination argument. + +Chapter XI Match Result Processing +============================================================================ + +XI-1. Match Result + +RPR tracks the current match result as a single entry in WindowAggState +with two fields: rpr_match_start and rpr_match_length. When +rpr_match_start is >= 0 the entry describes the result for that position, +and rpr_match_length gives the kind: -1 for an unmatched row, 0 for an +empty match (pattern matched but consumed no rows), and >= 1 for a real +match of that many rows. When rpr_match_start is < 0, the position has +not been evaluated yet (RF_NOT_DETERMINED). + +A row's status against the current match result can be obtained by +calling get_reduced_frame_status(). + +The frame accessors in nodeWindowAgg.c -- WinGetSlotInFrame() and +ignorenulls_getfuncarginframe(), the latter serving IGNORE NULLS -- +clip every seek to the reduced frame. Each asks +row_is_in_reduced_frame() about frameheadpos and takes the length it +returns as the frame's extent: a WINDOW_SEEK_HEAD relpos at or past +that length is out of frame, as is a WINDOW_SEEK_TAIL relpos whose +backward distance reaches it, the tail case landing on +frameheadpos + relpos + length - 1. That arithmetic assumes the reduced +frame is one contiguous run beginning at frameheadpos, which holds only +because RPR rejects EXCLUDE (III-1) and pins the frame start at +CURRENT ROW; relaxing either restriction means revisiting both +accessors. The same guarantee is why a WINDOW_SEEK_TAIL access under +RPR marks frameheadpos rather than the accessed row: the frame start +never moves backwards, so the mark can never end up ahead of a row a +later fetch still needs. + +An empty match is still recorded as a success: update_reduced_frame() +passes it to ExecRPRRecordContextSuccess() with length 0, so EXPLAIN +counts it under NFA Matched. The frame accessors never see that 0, +though. row_is_in_reduced_frame() returns -1 for RF_EMPTY_MATCH exactly +as for RF_UNMATCHED, so the row's reduced frame is empty. It cannot +return 0 there: 0 is its "RPR not defined" answer, which the accessors +and the aggregation loop of XI-5 read as the unclipped full frame. +RF_FRAME_HEAD returns the match length (>= 1) and RF_SKIPPED returns -2. + +XI-2. AFTER MATCH SKIP + +Determines the starting point for the next match attempt after a successful +match: + + SKIP TO NEXT ROW: + New match attempt begins from the row after the match start row. + Overlapping matches are possible. + + SKIP PAST LAST ROW: + New match attempt begins from the row after the match end row. + Only non-overlapping matches are possible. + +The clause is optional and its absence is not neutral: the grammar +supplies SKIP PAST LAST ROW, so that is what an RPR window gets by +default, and with it the absorption prerequisite of IX-3(a). + +RPSkipTo has a third value, ST_NONE, which the RPR productions never +produce. It is what a window with no row pattern common syntax leaves in +WindowClause.rpSkipTo, so ST_NONE means "not an RPR window", never "an +RPR window that did not say". + +The result slot of XI-1 holds one match at a time, so SKIP TO NEXT ROW +needs it cleared before each row: ExecWindowAgg() calls +clear_reduced_frame() at the top of every row when rpSkipTo is +ST_NEXT_ROW. Without that, a row inside the previous match would +answer RF_SKIPPED and be reported as an interior row instead of +starting a match of its own. SKIP PAST LAST ROW keeps the slot across +rows, which is exactly what makes the interior rows of its match report +as skipped. + +XI-3. INITIAL vs SEEK + + Standard definition (ISO/IEC 19075-5 6.12): + INITIAL: "is used to look for a match whose first row is R." + SEEK: "is used to permit a search for the first match anywhere + from R through the end of the full window frame." + In either case, if there is no match, the reduced window frame is empty. + The default is INITIAL. + + Current implementation: + SEEK is not supported (the parser raises an error). + Only INITIAL is supported, searching only for matches starting at each + row position pos. + +XI-4. Bounded Frame Handling + + With RPR, the frame mode is always ROWS and the frame start must be + CURRENT ROW. The frame end must be UNBOUNDED FOLLOWING or a positive + offset (n >= 1) FOLLOWING; a CURRENT ROW end or a zero offset is + rejected, since it would reduce the frame to the single current row. + + The zero-offset rejection happens only at execution, in + calculate_frame_offsets(), the same place the ordinary frame bounds + are computed. The parser accepts any offset FOLLOWING, so a literal + 0 and an offset that is not constant until the scan -- a bind + parameter, say -- both reach that check, and it raises "frame ending + offset must be positive with row pattern recognition". + + When the frame is bounded (e.g., ROWS BETWEEN CURRENT ROW AND 5 + FOLLOWING), ExecRPRProcessRow derives the upper bound itself from + winstate->frameOptions and winstate->endOffsetValue, using a + frameOffset of -1 to mean the frame runs to the partition end. In the + match phase, a context whose frame boundary has been reached + (currentPos == matchStartRow + frameOffset + 1) is matched with a NULL + varMatched instead, which forces a mismatch and finalizes it. This + prevents matches from extending beyond the window frame. The sum is + clamped to PG_INT64_MAX on overflow. + + Note that bounded frames also disable context absorption at the + planner level (see IX-3(b)), since the frame boundary breaks the + monotonicity assumption required for correct absorption. + +XI-5. Window Aggregates over the Reduced Frame + + RPR switches off the moving-aggregate path. eval_windowaggregates() + marks every plain aggregate for restart on every row when + rpr_is_defined(), because one row's reduced frame need not overlap the + next row's at all, and overlap is the assumption an inverse transition + function rests on. The aggregation loop then reads the reduced frame + itself to find where to stop: it stops when currentpos is determined + but aggregatedupto is not, when row_is_in_reduced_frame() reports an + unmatched row, or when the base row of the aggregation turns out to be + an interior (RF_SKIPPED) row of a match. + +Chapter XII Worked Example: Full Execution Trace +============================================================================ + +XII-1. Query + + SELECT company, tdate, price, + first_value(price) OVER w AS start_price, + last_value(price) OVER w AS end_price + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + PATTERN (A+ B) + DEFINE A AS price > PREV(price), + B AS price < PREV(price) + ); + +XII-2. Data + + Row# tdate price + -------------------------- + 0 2024-01-01 100 + 1 2024-01-02 110 + 2 2024-01-03 120 + 3 2024-01-04 115 + 4 2024-01-05 130 + +XII-3. Compilation Result + + PATTERN (A+ B) -> unchanged after optimization + + idx varId depth min max next jump + ----------------------------------------- + 0 A(0) 0 1 INF 1 -1 A+ + 1 B(1) 0 1 1 2 -1 B + 2 FIN 0 1 1 -1 -1 + + DEFINE: A -> "price > PREV(price)", B -> "price < PREV(price)" + isAbsorbable = true (A+ is a simple unbounded VAR) + +XII-4. Execution Trace + +The trace lists every variable's DEFINE value together for readability. In +the lazy model each variable is evaluated only when a state consumes it +(nfa_eval_var_match); a variable no state tests at a row stays +RPR_VAR_UNEVALUATED. + +--- Row 0 (price=100) --- + + update_reduced_frame(0) called. + + Context C0 created (matchStartRow=0). + Initial advance: elemIdx=0(A) -> VAR, so state is added. + C0.states = [{elemIdx=0, counts=[0]}] + + DEFINE values, row 0: + A: price(100) > PREV(price) -> no PREV -> false + B: price(100) < PREV(price) -> no PREV -> false + varMatched = [false, false] + + ExecRPRProcessRow(0): + Phase 1 (Match): A(0) state vs varMatched[0]=false -> state removed + C0.states = [] (empty) + + Phase 2 (Absorb): skipped (no states) + Phase 3 (Advance): skipped (no states) + + Context C1 created (matchStartRow=1). + Initial advance: C1.states = [{elemIdx=0, counts=[0]}] + + C0.states is empty, so the loop terminates. + matchedState is NULL -> unmatched. + +--- Row 1 (price=110) --- + + update_reduced_frame(1) called. + + C1 is the head context and starts at 1, so it is the target. + C1.states = [{elemIdx=0, counts=[0]}] + + DEFINE values, row 1: + A: 110 > PREV(100) -> true + B: 110 < PREV(100) -> false + varMatched = [true, false] + + ExecRPRProcessRow(1): + Phase 1 (Match): A(0) match succeeds -> counts[0]++ -> counts=[1] + C1.states = [{elemIdx=0, counts=[1]}] + + Phase 3 (Advance): + State {elemIdx=0, counts=[1]}: A+ (min=1, count=1, max=INF) + count >= min, so: + Greedy -> loop first: keep {elemIdx=0, counts=[1]} + exit: reset counts[0]=0, next(=1) -> {elemIdx=1, + counts=[0]} + C1.states = [{elemIdx=0, counts=[1]}, {elemIdx=1, counts=[0]}] + +--- Row 2 (price=120) --- + + Context C2 created (matchStartRow=2). + Initial advance: C2.states = [{elemIdx=0, counts=[0]}] + + DEFINE values, row 2: + A: 120 > PREV(110) -> true + B: 120 < PREV(110) -> false + varMatched = [true, false] + + ExecRPRProcessRow(2): (each phase walks every context in turn) + Phase 1 (Match): + C1 {elemIdx=0, counts=[1]}: A matches -> counts=[2] + C1 {elemIdx=1, counts=[0]}: B does not match -> removed + C2 {elemIdx=0, counts=[0]}: A matches -> counts=[1] + C1.states = [{elemIdx=0, counts=[2]}] + C2.states = [{elemIdx=0, counts=[1]}] + + Phase 2 (Absorb): + Does C1 (started earlier) cover C2? + C1: {elemIdx=0, counts=[2]}, C2: {elemIdx=0, counts=[1]} + Same elemIdx, C1.counts >= C2.counts -> covered + C2 absorbed. -> removed. + + Phase 3 (Advance): + {elemIdx=0, counts=[2]}: Greedy -> loop + exit + Loop: {elemIdx=0, counts=[2]} + Exit: reset counts[0]=0, next(=1) -> {elemIdx=1, counts=[0]} + C1.states = [{elemIdx=0, counts=[2]}, {elemIdx=1, counts=[0]}] + + Context C3 created (matchStartRow=3). + +--- Row 3 (price=115) --- + + DEFINE values, row 3: + A: 115 > PREV(120) -> false + B: 115 < PREV(120) -> true + varMatched = [false, true] + + ExecRPRProcessRow(3): + Phase 1 (Match): + {elemIdx=0, counts=[2]}: A does not match -> removed + {elemIdx=1, counts=[0]}: B matches -> counts=[1] + C1.states = [{elemIdx=1, counts=[1]}] + + Phase 3 (Advance): + {elemIdx=1, counts=[1]}: B (min=1, max=1) + count(1) >= max(1) -> unconditional exit + Reset counts[0]=0, next = 2 (FIN) + FIN reached -> matchEndRow = 3, matchedState recorded. + Early termination: no remaining states, so completed immediately. + C1.states = [] (empty after reaching FIN) + + Context C4 created (matchStartRow=4). + + C1.states is empty and matchedState is set -> match succeeds, + spanning matchStartRow=1 through matchEndRow=3. + + rpr_match_start = 1, rpr_match_length = 3 + +--- Row 4 (price=130) --- + + update_reduced_frame(4) called. + C3 was pruned when C1 recorded its match: under SKIP PAST LAST ROW every + context that started within the match's range is freed there. + C4 is the head context and starts at 4, so it is the target. + + DEFINE values, row 4: + A: 130 > PREV(115) -> true + B: 130 < PREV(115) -> false + + ... No subsequent rows, so ExecRPRFinalizeAllContexts() is called. + Match incomplete -> unmatched. + +XII-5. Final Result + + Row 0: unmatched -> reduced frame empty (window funcs NULL, count() 0) + Row 1: match head -> frame = rows 1 through 3 + Row 2: inside match -> skipped + Row 3: inside match -> skipped + Row 4: unmatched -> reduced frame empty (window funcs NULL, count() 0) + +Chapter XIII Deparse and EXPLAIN Output +============================================================================ + +An RPR window is printed back by two independent printers, and they produce +different text on purpose. Both must round-trip: what pg_get_viewdef() +prints has to re-parse into the same query, and what EXPLAIN prints has to +describe the pattern the executor will actually run. + +XIII-1. Two Printers, Two Spellings + +pg_get_viewdef() and friends (get_rule_windowspec(), get_rule_pattern() in +ruleutils.c) walk the parse tree the parser built, so a stored view shows +the PATTERN as the user wrote it. The clause is emitted in full, each part +on its own line: AFTER MATCH SKIP, then INITIAL, then PATTERN, then DEFINE. +INITIAL is printed unconditionally, since SEEK is not implemented and the +window clause records no flag that would distinguish the two. + +EXPLAIN (show_window_def(), deparse_rpr_pattern() in explain.c) walks the +compiled element array instead, so it shows the pattern after the Phase 1 +rewrites of V-3: PATTERN (A A) is stored as written but explains as "a{2}". +EXPLAIN also parenthesizes every group and every alternation for +self-consistency, so a top-level A | B explains as "(a | b)" where +pg_get_viewdef() prints it bare. Only the pattern is shown; the DEFINE +clause and the skip mode do not appear in EXPLAIN output. + +XIII-2. Reading the Compiled Array + +Two markers may follow a quantifier in EXPLAIN's output, and they report the +flags of V-7: "#" on an element carrying RPR_ELEM_ABSORBABLE, the comparison +point, and "~" on one carrying only RPR_ELEM_ABSORBABLE_BRANCH, the region. +So "a+#" is an absorbable A+, "(a~ b~){2,}#" an absorbable (A B){2,}, and a +pattern that prints no marker at all was found unabsorbable. Neither +character can occur in an unquoted variable name, and a name containing one +is double-quoted, so a marker is never read as part of a name. + +The printer leans on two compile-time invariants. A {1,1} group never +reaches the array (V-3 (h)), so every surviving BEGIN/END pair carries a +non-trivial quantifier, which is read from the END element. A fixed count +is normalized to greedy (V-3 (i)), so a trailing "?" in EXPLAIN's output is +always reluctance and never {0,1}. + +It also relies on two properties of the SEP chain of V-4. A nested +alternation's last SEP has its next redirected past the enclosing +alternation, exactly as a branch tail does, so next is not a way to find +where an alternation ends. The last SEP is however always emitted as the +alternation's final element, so the index of the last SEP plus one is this +alternation's own post-ALT element. That is how rpr_alt_scope_end() +(explain.c) bounds an alternation and walks its branch boundaries, and a +change to the SEP chain has to keep both properties. + +XIII-3. Pattern Variable Quoting + +A pattern variable is printed by quote_pattern_variable() (ruleutils.c), +which is quote_identifier() plus one extra case: the name permute is quoted +even though PERMUTE is an unreserved keyword. A bare permute followed by +'(' inside a PATTERN would be re-read as the PERMUTE syntax the parser +rejects, so the deparsed text would no longer re-parse. The DEFINE clause +has no such hazard, because a name there is always followed by AS; +get_rule_define() therefore uses plain quote_identifier(), and one variable +can legitimately print bare in DEFINE and quoted in PATTERN. + +quote_pattern_variable() is exported from ruleutils.h precisely so that +EXPLAIN's own printer can call it. Both printers must spell a variable the +same way, so a new printer calls it rather than quote_identifier(). + +XIII-4. Reluctance On A Fixed Count + +The parse-tree printer emits nothing at all for a {1,1} node, so a reluctant +{1,1} would print as a bare '?', which re-parses as the {0,1} quantifier. +It therefore emits an explicit {1} first, and A{1}? deparses as a{1}?. + +EXPLAIN's printer has no such case and instead asserts min != max on a +reluctant element, because V-3 (i) has already cleared reluctance wherever +min == max. The parse tree ruleutils.c reads has not been through that +pass: buildRPRPattern() runs Phase 1 on a copy of the pattern and never +writes back to the WindowClause. That is why only one of the two printers +needs the guard. + +XIII-5. Settling the Column Names a DEFINE Clause References + +Inside DEFINE a column reference has no qualifier available, because the +qualifier slot names a pattern variable. get_rule_define() prints with +varprefix off for that reason, so every column it prints has to resolve, +unqualified, to the same column when the text is read back. + +set_deparse_for_query() calls mark_define_columns() (ruleutils.c) before +set_using_names() chooses any USING name and before column aliases are +assigned. It collects the level-zero Vars of every window's defineClause and +hands each to mark_define_column(), which makes the printed name unique +within the owning RTE and against the names reserved so far, stores it in +that RTE's colnames entry, and reserves it with reserve_colname() in +dpns->using_names, the list that holds the globally unique USING names. No +other RTE may then be given that name, so a same-named column elsewhere in +the query is uniquified and its RTE gets a column alias list. Without this, +another relation of the query acquiring such a column would make an existing +view's DEFINE clause ambiguous on re-parse. + +Settling these names first lets set_using_names() choose around them rather +than over them. Before it invents a name for a merged column it asks +preset_input_colname() whether an input of the join, followed down through +joinaliasvars, has settled one already, and if so adopts that name. + +mark_define_column() has to resolve the Var the way the printer will resolve +it -- through varnosyn/varattnosyn, which the parser sets on every Var it +makes, so that a Var reading a join column names the join RTE -- or the name +it settles is not the name that reaches the output. A system column, whose +name comes from the catalog, cannot be renamed and is only reserved. The +other columns that cannot be renamed, those of a relation RTE outside the +FROM clause, are not reachable from DEFINE, where the qualifier slot names a +pattern variable; nor are whole-row references, which the parser rejects, or +dropped columns. + +The guarantee rests on set_relation_column_names(), whose rules apply to +every query, with a DEFINE clause or not. For a function RTE it appends the +columns the composite result type has gained since parse time +(function_rte_late_colnames(); only for a single function without WITH +ORDINALITY or a column definition list), so a reserved name is kept off +them and the positional column alias list counts them. A table function +RTE (XMLTABLE, JSON_TABLE) prints a column alias list when it has +user-written column aliases or a column was renamed, as other non-relation +RTEs do. The columns colname_is_fixed() reports are never renamed there. + +XIII-6. Navigation Names Shadow User Functions + +Within a DEFINE clause the parser binds an unqualified prev, next, first or +last to a navigation operation before any catalog lookup (III-5), so an +unqualified call to a user function of one of those names would change meaning +across a deparse and re-parse. get_rule_define() sets inRPRDefine in the +deparse context for the duration of the clause, and generate_function_name() +force-qualifies exactly those four names while it is set, the same treatment +cube and rollup get inside GROUP BY. Only the exact lower-case spellings are +at risk: a mixed-case function name deparses quoted and cannot match the +parser's downcased comparison. + +inRPRDefine is part of the deparse context, so every routine that builds one +initializes it, and get_rule_define() saves and restores it around the clause +the way it does varprefix. + +XIII-7. DEFINE Vars Under a GROUP RTE + +When a query groups, parse analysis replaces grouped expressions with Vars of +a GROUP RTE, and the deparser expands those back with flatten_group_exprs() +before printing. The targetlist and havingQual expansion does not reach a +window's DEFINE clause, yet that clause carries GROUP Vars of its own whenever +it references a subexpression the grouping step computes (IV-4). +get_query_def() therefore runs flatten_group_exprs() over each window's +defineClause as well. Without it the deparsed DEFINE would name the grouping +step's output rather than the expression the user wrote, and the view would +not re-parse. + +Expansion alone is not enough when the grouped expression is written over a +column merged by USING. What it brings back is then the expression the parser +built for the merge -- for a FULL join, a COALESCE of the two inputs -- and +because a DEFINE clause prints without qualifiers that would come out as +COALESCE(id, id), which re-parses as one merged column nested inside another. +collapse_define_join_vars() runs right after the expansion and folds each such +expression, recognised against the join RTEs' joinaliasvars with nulling marks +ignored, back into a Var naming the merged column, which is what the user +wrote. Inputs are folded before the node that holds them, so a merge built +over another merge is seen whole once its inner merge has become a Var again. + +XIII-8. Navigation Trim in EXPLAIN + +The two navigation reaches of VII-6 are printed as Nav Mark Lookback and Nav +Mark Lookahead, read straight out of the WindowAggState. They appear for a +plain EXPLAIN as well, because executor init runs and that is where a +constant offset is settled. Each dimension prints its kind rather than +merely a number: "runtime" for RPR_NAV_OFFSET_NEEDS_EVAL, since the value is +only known per scan, "retain all" for RPR_NAV_OFFSET_RETAIN_ALL, and the +offset itself for RPR_NAV_OFFSET_FIXED. Only the backward dimension can +reach the retain-all state; a forward reach that overflows clamps to +PG_INT64_MAX, which prints as "infinite", and EXPLAIN asserts that the +lookahead is never retain-all. + +XIII-9. NFA Counters Under EXPLAIN ANALYZE + +With ANALYZE, once the tuplestore exists, show_windowagg_info() (explain.c) +adds the matcher's counters from the WindowAggState via show_rpr_nfa_stats(). +States: peak live, total created, and merged, the new states +nfa_append_state_unique() discards as duplicates (X-5). Contexts: peak live +and total created, plus the outcome counters, which the code that frees a +context updates; ExecRPRFreeContext() itself records no outcome. +update_reduced_frame() counts the context whose result it registers as +matched, with length matchEndRow - matchStartRow + 1 (0 for an empty match), +or else as failed. ExecRPRCleanupDeadContexts() counts a context that died +without a recorded match as failed, unless it never processed its start row, +as with one created for a row beyond the partition. Absorption counts its +context as absorbed (IX), and SKIP PAST LAST ROW counts the contexts it +frees because they start inside a recorded match as skipped (XI-2). +ExecRPRRecordContextFailure() reports a failure of length 1 as pruned and any +other as mismatched, so the pruned counter is not SKIP pruning. Every length +but a match's is lastProcessedRow - matchStartRow + 1. Matched, mismatched, +absorbed and skipped carry min/max/avg lengths, shown only when the count is +nonzero. The text format folds these into the NFA States, NFA Contexts and +NFA lines and prints the absorbed/skipped line only when either is nonzero; +the other formats emit every count as its own property. + +Chapter XIV Summary of Key Design Decisions +============================================================================ + +XIV-1. Flat Array vs Tree-Based NFA + + The compiled pattern is stored as a flat array of fixed-size 16-byte + RPRPatternElement structs rather than as a tree. + + The array is contiguous and cache-friendly, elements reference each + other by 2-byte index instead of by pointer, and the whole array can + be copied with a single memcpy when the plan node is copied. + +XIV-2. Forward-only Execution vs Backtracking + + The NFA is simulated forward-only, tracking a set of live states, + rather than by backtracking. + + Backtracking would take exponential time in the worst case, whereas + forward-only NFA simulation is polynomial. Forward-only also fits the + window pipeline, which delivers sorted rows sequentially: it needs no + re-fetching of earlier rows, and each row's DEFINE conditions (SQL + expressions such as PREV or NEXT navigation, with high re-evaluation cost) + are evaluated once per row and cached; only match_start-dependent variables + are re-evaluated per context (VII-5). DFS order yields preferment + naturally, with greedy or reluctant behavior per quantifier obtained by + reversing that order. + +XIV-3. Per-Context Management + + A separate match context is maintained for each start row. + + This supports overlapping matches under SKIP TO NEXT ROW, determines + each row's frame independently, and lets the absorption optimization + eliminate redundant contexts in O(n). + +XIV-4. Memory Pool Management + + NFA states are managed through a custom free list, and both RPRNFAState + and RPRNFAContext are allocated in a partition-lifespan memory context + that is freed in release_partition. + + NFA states are created and destroyed in large numbers per row, so the + free list avoids palloc/pfree overhead. Their size varies (the + counts[] array), but maxDepth is fixed within a single query, so all + states have the same size. + +XIV-5. Execution Optimization Summary + + The following optimizations make the NFA simulation practical. + + -- Compile-time -- + + (1) Parse Tree Optimization (V-3) + + Simplifies the parse tree before converting the pattern to an NFA. + Reduces the number of NFA elements through consecutive variable + merging (A A -> A{2}), SEQ flattening, quantifier multiplication, + and other transformations. + + Significance: Reducing the element count directly shrinks the state + space, decreasing the cost of all subsequent runtime phases (match, + absorb, advance). + + -- Runtime: advance phase -- + + (2) Group Skip (X-4(b)) + + At the BEGIN of a group with min=0, skips the entire group through + END.next (BEGIN.jump names the END). Moves directly to the first + element outside the group without exploring the group body. Greedy + enters then skips; Reluctant skips then enters. + + Significance: For optional groups (min=0), immediately generates + a skip path without exploring the body, avoiding unnecessary DFS + expansion. + + (3) State Deduplication (X-5) + + During advance, DFS may generate states with the same (elemIdx, counts) + combination through multiple paths. Additionally, for group absorption, + nfa_match performs inline advance from bounded VARs (count >= max) within + the absorbable region (ABSORBABLE_BRANCH) through END chains to reach the + comparison point (ABSORBABLE END). This process can also produce + duplicate states reaching the same END. nfa_append_state_unique() blocks + duplicate addition during advance. The inline advance adds nothing -- it + moves states in place -- so the duplicates it leaves on an END are + collapsed when the next advance re-adds their successors. + + Significance: Prevents exponential growth of the state count in + ALT branches and quantifier expansion. Since DFS order causes the + preferred branch's state to be registered first, identical states + from lower-priority branches are automatically discarded, thereby + also guaranteeing preferment. + + (4) Cycle Detection and Fast-Forward (X-6, X-4(c)) + + When a nullable group body (e.g., A?) repeats empty matches, + the END -> first-child loop-back can continue indefinitely. + + Two mechanisms resolve this: + - A visited bitmap (nfaVisitedEnds) marks a nullable END whose body + has already derived an empty iteration. On a second arrival the + state leaves the group there once count >= min; below min it falls + through to the must-loop path, whose per-arrival count increment + reaches min in bounded steps (termination) + - At an END with the RPR_ELEM_EMPTY_LOOP flag set, when count < min, + the remaining required iterations are treated as empty matches and a + fast-forward exit path out of the group is added (correctness) + + Significance: Cycle detection guarantees termination, and + fast-forward guarantees that the min condition is satisfied. + Without these, patterns containing nullable groups would fall + into infinite loops or fail to match. + + (5) Match Pruning (X-4(e)) + + When a state reaches FIN during advance, all remaining unprocessed + states of that context are removed. Because of DFS order, the path + that reaches FIN first has the highest preferment, so the remaining + paths are inferior. + + Significance: Once the best match is determined, exploration of + inferior paths is immediately terminated. This mechanism achieves + both preferment guarantees and performance optimization. + + -- Runtime: inter-context -- + + (6) Early Termination (SKIP PAST LAST ROW) + + In SKIP PAST LAST ROW mode, when a match is found, subsequent + contexts whose start rows fall within the match range are pruned + immediately without further processing. + In SKIP TO NEXT ROW mode, overlapping contexts are preserved + because each row requires its own independent match. + + Significance: Prunes subsequent contexts whose start rows overlap + with a prior match range, avoiding unnecessary processing. + + (7) Context Absorption (Chapter IX) + + If an independent context is created for each row, O(n^2) states + accumulate. By exploiting the monotonicity that an earlier-started + context subsumes the states of a later-started context, redundant + contexts are eliminated early. + + Absorbability is determined per-element; comparison is performed + only at elements with the RPR_ELEM_ABSORBABLE flag (see V-7). + + Significance: Keeps the number of active contexts at a constant + level, achieving O(n^2) -> O(n) time complexity. Without this, + performance degrades sharply on long partitions. + +Appendix A. Data Structure Relationship Diagram +============================================================================ + + Parser Layer + -------- + RPCommonSyntax + |--- rpSkipTo: RPSkipTo + |--- rpDefs: List* (ResTarget) + +--- rpPattern: RPRPatternNode* (tree) + |--- nodeType: VAR | SEQ | ALT | GROUP + |--- min, max: quantifier + |--- reluctant: bool + |--- varName: variable name (VAR only) + +--- children: List* (SEQ/ALT/GROUP only) + + Planner Layer + ---------- + WindowAgg (plan node) + |--- rpSkipTo: RPSkipTo + |--- defineClause: List + |--- defineMatchStartDependent: Bitmapset* (see VII-5) + +--- rpPattern: RPRPattern* + |--- numVars: int + |--- varNames: char** + |--- maxDepth: RPRDepth + |--- isAbsorbable: bool + |--- numElements: int + +--- elements: RPRPatternElement[] (flat array) + |--- varId (1B) + |--- depth (1B) + |--- flags (1B) + |--- reserved (1B) + |--- min, max (4B + 4B) + +--- next, jump (2B + 2B) + + Executor Layer + ---------- + WindowAggState + |--- rpSkipTo: RPSkipTo (AFTER MATCH SKIP mode) + |--- rpPattern: RPRPattern* (copied from plan) + |--- defineClauseExprs: List (DEFINE order, index == varId) + |--- nfaVarMatched: RPRVarMatch[] (per-row tri-state cache, lazy) + |--- defineMatchStartDependent: Bitmapset* (match_start_dependent + | DEFINE vars; see VII-5) + |--- nfaVisitedEnds: bitmapword* (cycle detection) + |--- nfaVisitedMinWord / nfaVisitedMaxWord: int16 + | (touched-word range for fast reset) + |--- nfaLastProcessedRow: int64 (-1 = none) + |--- nfaStateSize: Size (pre-calculated RPRNFAState allocation size) + |--- nfaContext <-> nfaContextTail (doubly-linked list) + | +--- RPRNFAContext + | |--- states: RPRNFAState* (linked list) + | | |--- elemIdx + | | |--- counts[] + | | +--- isAbsorbable + | |--- matchStartRow, matchEndRow + | |--- lastProcessedRow + | |--- matchedState (the FIN-arriving state, moved here) + | |--- hasAbsorbableState + | +--- allStatesAbsorbable + |--- nfaContextFree (recycling pool) + +--- nfaStateFree (recycling pool) + +Appendix B. NFA Element Array Examples +============================================================================ + +B-1. PATTERN (A B C) + + idx varId depth min max next jump + ------------------------------------------ + 0 A 0 1 1 1 -1 + 1 B 0 1 1 2 -1 + 2 C 0 1 1 3 -1 + 3 FIN 0 1 1 -1 -1 + +B-2. PATTERN (A+ B*) + + idx varId depth min max next jump flags + ------------------------------------------------------------------------ + 0 A 0 1 INF 1 -1 ABSORBABLE | ABSORBABLE_BRANCH + 1 B 0 0 INF 2 -1 + 2 FIN 0 1 1 -1 -1 + + Only A+ is the absorption point (Case 1). Once past A, + absorption is permanently disabled for that state. + +B-3. PATTERN (A | B | C) + + idx varId depth min max next jump + ---------------------------------------- + 0 ALT 0 1 1 1 2 next -> branch 1, jump -> SEP1 + 1 A 1 1 1 7 -1 branch 1 -> post-ALT + 2 SEP 0 1 1 3 4 branch 1 term.; next B, jump SEP2 + 3 B 1 1 1 7 -1 branch 2 -> post-ALT + 4 SEP 0 1 1 5 6 branch 2 term.; next C, jump SEP3 + 5 C 1 1 1 7 -1 branch 3 -> post-ALT + 6 SEP 0 1 1 7 -1 branch 3 terminator (last) + 7 FIN 0 1 1 -1 -1 + + Each branch is terminated by a SEP. ALT.jump enters the SEP chain, each + SEP.jump links to the next branch's SEP (-1 on the last), and each SEP.next + enters the next branch's content; the branch tails are redirected to FIN. + +B-4. PATTERN ((A B)+ C) + + idx varId depth min max next jump flags + -------------------------------------------------------------------------- + 0 BEGIN 0 1 INF 1 3 ABSORBABLE_BRANCH + 1 A 1 1 1 2 -1 ABSORBABLE_BRANCH + 2 B 1 1 1 3 -1 ABSORBABLE_BRANCH + 3 END 0 1 INF 4 1 ABSORBABLE | ABSORBABLE_BRANCH + 4 C 0 1 1 5 -1 + 5 FIN 0 1 1 -1 -1 + + Case 2: GROUP+ with {1,1} body VARs. A, B are branches; + END is the absorption point. Compare with B-6 (Case 3). + +B-5. PATTERN ((A | B)+? C) + + idx varId depth min max next jump flags + ------------------------------------------------------------------- + 0 BEGIN 0 1 INF 1 6 RELUCTANT, group start + 1 ALT 1 1 1 2 3 next -> branch 1, jump -> SEP1 + 2 A 2 1 1 6 -1 branch 1 -> END + 3 SEP 1 1 1 4 5 branch 1 term.; jump -> SEP2 + 4 B 2 1 1 6 -1 branch 2 -> END + 5 SEP 1 1 1 6 -1 branch 2 terminator (last) + 6 END 0 1 INF 7 1 RELUCTANT, group end + 7 C 0 1 1 8 -1 + 8 FIN 0 1 1 -1 -1 + + The ALT lives inside a group, so its branch tails are redirected to the + post-ALT element (here the group's END at idx 6), not out of the group. + +B-6. PATTERN ((A+ B)+ C) -- Absorbability flag example + + idx varId depth min max next jump flags + --------------------------------------------------------------------------- + 0 BEGIN 0 1 INF 1 3 ABSORBABLE_BRANCH, group start + 1 A 1 1 INF 2 -1 ABSORBABLE | ABSORBABLE_BRANCH + 2 B 1 1 1 3 -1 + 3 END 0 1 INF 4 1 group end + 4 C 0 1 1 5 -1 + 5 FIN 0 1 1 -1 -1 + + Recurses from BEGIN into the body -> A matches Case 1 (simple VAR+). + A gets ABSORBABLE | ABSORBABLE_BRANCH, BEGIN gets ABSORBABLE_BRANCH. + B and END get no flags -> absorption stops once the state advances to B. + (See V-7 Case 3) + +B-7. PATTERN ((A+ B | C*)+ D) -- Per-branch absorption in ALT + + idx varId depth min max next jump flags + --------------------------------------------------------------------------- + 0 BEGIN 0 1 INF 1 7 ABSORBABLE_BRANCH + 1 ALT 1 1 1 2 4 ABSORBABLE_BRANCH; jump -> SEP1 + 2 A 2 1 INF 3 -1 ABSORBABLE | ABSORBABLE_BRANCH + 3 B 2 1 1 7 -1 branch 1 -> END + 4 SEP 1 1 1 5 6 branch 1 term.; jump -> SEP2 + 5 C 2 0 INF 7 -1 ABSORBABLE | ABSORBABLE_BRANCH + 6 SEP 1 1 1 7 -1 branch 2 terminator (last) + 7 END 0 1 INF 8 1 EMPTY_LOOP + 8 D 0 1 1 9 -1 + 9 FIN 0 1 1 -1 -1 + + ALT branches are checked independently for absorbability. + Branch 1: A+ matches Case 1 -> A gets ABSORBABLE. B has no flag. + Branch 2: C* matches Case 1 -> C gets ABSORBABLE. + Both A and C get ABSORBABLE_BRANCH as part of their respective branch + paths. + END has EMPTY_LOOP: branch 2 (C*) is nullable, making the group body + nullable. + BEGIN and ALT get ABSORBABLE_BRANCH (on the path to absorbable elements). + The SEP branch-separator markers carry no flags: + computeAbsorbabilityRecursive walks the SEP chain but marks only + branch content. + + +References: + +[1] ISO/IEC 19075-5 Information technology - Guidance for the use of + database language SQL - Part 5: Row pattern recognition + +[2] ISO/IEC 9075-2 Information technology - Database languages - SQL - + Part 2: Foundation (SQL/Foundation) + +============================================================================ + End of document +============================================================================ -- 2.43.0