From 88dd8d17aba131d6619f709bdc19a00589444261 Mon Sep 17 00:00:00 2001 From: Dave Cramer Date: Tue, 8 Sep 2026 22:50:41 +0000 Subject: [PATCH v5 2/3] Add a fetch direction to Execute under _pq_.cursor The Bind half of _pq_.cursor lets a client create a scrollable, holdable portal, but nothing could then scroll it. The only way to read rows back was a plain Execute message, which always fetches forward from wherever the portal happens to be, so the SCROLL option bought the client nothing that it could reach over the wire. Anything more than forward reading meant falling back to DECLARE CURSOR and FETCH as text queries. Extend the Execute message, when _pq_.cursor has been negotiated, with an Int32 of fetch flags and an Int64 count. The low three bits of the flags select a direction with the same meaning as the identically named direction of the SQL FETCH command, and PQ_FETCH_MOVE asks for the repositioning without the rows, as MOVE does. The count is interpreted as FETCH would interpret it, with PQ_FETCH_ALL standing for all remaining rows; the server-internal FETCH_ALL is LONG_MAX and therefore not portable enough to put on the wire. The two fields are mandatory once the extension is active rather than an optional trailer. Two extensions that each append an optional trailer to the same message cannot be told apart by length, so sniffing is not a foundation to build further extensions on. To keep that from changing behavior, PQ_FETCH_DEFAULT (all bits zero) asks for no fetch behavior at all: the row-count field of the Execute message governs, exactly as it does without the extension, and the portal can still report PortalSuspended. An all-zero trailer is thus byte-for-byte equivalent to today's Execute, which matters for drivers that rely on the row-count field. A direction and a nonzero row count together are contradictory, so the server rejects that combination instead of guessing. Only a portal that actually holds a query can be fetched from, so the server checks the portal strategy before calling PortalRunFetch, and reports the query's own command tag with the fetched row count, or MOVE when only the position moved. libpq support for the new fields, and the tests that exercise them, follow in a separate commit. --- doc/src/sgml/protocol.sgml | 77 +++++++++++++-- src/backend/tcop/postgres.c | 184 +++++++++++++++++++++++++++++++++-- src/include/libpq/protocol.h | 29 ++++++ 3 files changed, 275 insertions(+), 15 deletions(-) diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml index afaca4aaa11..4ee69d51fe3 100644 --- a/doc/src/sgml/protocol.sgml +++ b/doc/src/sgml/protocol.sgml @@ -337,11 +337,16 @@ _pq_.cursor true - PostgreSQL 19 and later - Enables cursor options in the Bind message. - When set to true, the Bind message includes - a mandatory Int32 extension field after the result-column format - codes. The following flag values are defined: + PostgreSQL 20 and later + + + Enables cursor options in the Bind message and fetching in the Execute + message. + + + When set to true, the Bind message includes a + mandatory Int32 extension field after the result-column format codes. + The following flag values are defined: 0x0001 — SCROLL 0x0002 — NO SCROLL @@ -354,6 +359,29 @@ accepted but never necessary. A value of 0 requests no cursor options at all, and creates exactly the portal that a Bind message without this extension would create. + + + The Execute message likewise includes a mandatory Int32 field of fetch + flags followed by a mandatory Int64 row count. The low three bits of + the flags select a fetch direction: + + 0x0000 — no fetch direction + 0x0001 — FORWARD + 0x0002 — BACKWARD + 0x0003 — ABSOLUTE + 0x0004 — RELATIVE + + and 0x0008 additionally requests that the rows be + discarded rather than returned, as + MOVE does. All + other bits are reserved and must be zero. With a direction, the portal + is fetched from just as the + FETCH command of + that direction would fetch from a cursor, and the maximum row count of + the Execute message must be zero. With no direction, both the flags + and the row count must be zero, and the maximum row count governs as + usual. + @@ -1158,6 +1186,15 @@ SELCT 1/0; ReadyForQuery or RowDescription to be issued. + + If the _pq_.cursor protocol option is negotiated, + Execute includes a mandatory fetch direction and row count, which allow a + portal created with the SCROLL cursor option to be fetched from in any + direction, as the FETCH + command does. Such an Execute always runs to completion and so never + sends PortalSuspended. + + If Execute terminates before completing the execution of a portal (due to reaching a nonzero result-row count), it will send a @@ -5185,7 +5222,35 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;" Maximum number of rows to return, if portal contains a query that returns rows (ignored otherwise). Zero - denotes no limit. + denotes no limit. This field must be zero if a + fetch direction other than 0x0000 is given + below. + + + + + + Int32 (present when _pq_.cursor is negotiated) + + + Fetch flags. See the _pq_.cursor entry in + for defined values. + A value of 0 means that no fetch behavior is requested, and the + maximum row count above governs, exactly as it does without the + extension. + + + + + + Int64 (present when _pq_.cursor is negotiated) + + + Number of rows to fetch, interpreted as the count of the + FETCH command + of the same direction would be. The largest positive value + (0x7FFFFFFFFFFFFFFF) means ALL. + This field must be zero if the fetch flags are 0. diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index 524fce52dd8..cf43acd54b3 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -2057,7 +2057,7 @@ exec_bind_message(StringInfo input_message) * CURSOR_OPT_* constants in parsenodes.h, so we map between the two * representations here. */ - if (MyProcPort->protocol_cursor_enabled) + if (MyProcPort != NULL && MyProcPort->protocol_cursor_enabled) { int bind_ext_flags; @@ -2199,19 +2199,83 @@ exec_bind_message(StringInfo input_message) debug_query_string = NULL; } +/* + * fetch_count_wire_to_long + * + * Map the Int64 fetch count of the Execute message's _pq_.cursor field onto a + * platform C long, which is what PortalRunFetch expects. PQ_FETCH_ALL is the + * reserved token meaning "all remaining rows", which must map to FETCH_ALL + * (LONG_MAX) rather than being arrived at by truncation. + */ +static long +fetch_count_wire_to_long(int64 count) +{ + if (count == PQ_FETCH_ALL) + return FETCH_ALL; /* == LONG_MAX */ + + /* + * The wire count is a full 64-bit integer, but "long" is only 32 bits + * where SIZEOF_LONG < 8 (LLP64 Windows, ILP32 platforms). There, a value + * that does not fit would be silently truncated by the cast below, so + * reject it instead. Where "long" is 64 bits this test is always false, + * so compile it out rather than emit a tautological comparison. + */ +#if SIZEOF_LONG < 8 + if (count > LONG_MAX || count < LONG_MIN) + ereport(ERROR, + (errcode(ERRCODE_NUMERIC_VALUE_OUT_OF_RANGE), + errmsg("fetch count out of range for this platform"))); +#endif + return (long) count; +} + +/* + * fetch_direction_wire_to_enum + * + * Map the wire fetch direction of the Execute message's _pq_.cursor field onto + * the server's FetchDirection enum. PQ_FETCH_DEFAULT has no equivalent and is + * not accepted here; callers deal with it before getting this far. + */ +static FetchDirection +fetch_direction_wire_to_enum(int wire_direction) +{ + switch (wire_direction) + { + case PQ_FETCH_FORWARD: + return FETCH_FORWARD; + case PQ_FETCH_BACKWARD: + return FETCH_BACKWARD; + case PQ_FETCH_ABSOLUTE: + return FETCH_ABSOLUTE; + case PQ_FETCH_RELATIVE: + return FETCH_RELATIVE; + } + + ereport(ERROR, + (errcode(ERRCODE_PROTOCOL_VIOLATION), + errmsg("invalid fetch direction in Execute message: %d", + wire_direction))); + return FETCH_FORWARD; /* keep compiler quiet */ +} + /* * exec_execute_message * * Process an "Execute" message for a portal + * + * fetch_flags and fetch_count carry the Execute message's _pq_.cursor field, + * or PQ_FETCH_DEFAULT and 0 when the extension is not in use. */ static void -exec_execute_message(const char *portal_name, long max_rows) +exec_execute_message(const char *portal_name, long max_rows, + int fetch_flags, int64 fetch_count) { CommandDest dest; DestReceiver *receiver; Portal portal; bool completed; QueryCompletion qc; + int fetch_direction = fetch_flags & PQ_FETCH_DIRECTION_MASK; const char *sourceText; const char *prepStmtName; ParamListInfo portalParams; @@ -2374,12 +2438,63 @@ exec_execute_message(const char *portal_name, long max_rows) if (max_rows <= 0) max_rows = FETCH_ALL; - completed = PortalRun(portal, - max_rows, - true, /* always top level */ - receiver, - receiver, - &qc); + if (fetch_direction == PQ_FETCH_DEFAULT) + { + completed = PortalRun(portal, + max_rows, + true, /* always top level */ + receiver, + receiver, + &qc); + } + else + { + FetchDirection direction = fetch_direction_wire_to_enum(fetch_direction); + long count = fetch_count_wire_to_long(fetch_count); + bool is_move = (fetch_flags & PQ_FETCH_MOVE) != 0; + uint64 nprocessed; + + /* + * Only a portal holding a single row-returning query can be fetched + * from; PortalRunFetch would reject anything else with a less helpful + * message. + */ + if (portal->strategy != PORTAL_ONE_SELECT && + portal->strategy != PORTAL_ONE_RETURNING && + portal->strategy != PORTAL_ONE_MOD_WITH && + portal->strategy != PORTAL_UTIL_SELECT) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot fetch from a portal of this type"))); + + /* + * A MOVE only repositions the portal, so throw the rows away rather + * than sending them. The real receiver is still destroyed below, + * exactly as in the non-MOVE case. + */ + nprocessed = PortalRunFetch(portal, direction, count, + is_move ? None_Receiver : receiver); + + /* + * Report the query's own command tag, so that a scrolling Execute + * looks like an ordinary one to a client that only reads the tag. A + * MOVE returns no rows, though, so it has to say so. + */ + InitializeQueryCompletion(&qc); + if (is_move) + SetQueryCompletion(&qc, CMDTAG_MOVE, nprocessed); + else if (portal->qc.commandTag != CMDTAG_UNKNOWN) + { + CopyQueryCompletion(&qc, &portal->qc); + qc.nprocessed = nprocessed; + } + + /* + * The client said exactly how many rows it wanted, so there is never + * anything left over to report with PortalSuspended. + */ + completed = true; + } receiver->rDestroy(receiver); @@ -5057,6 +5172,8 @@ PostgresMain(const char *dbname, const char *username) { const char *portal_name; int max_rows; + int fetch_flags = PQ_FETCH_DEFAULT; + int64 fetch_count = 0; forbidden_in_wal_sender(firstchar); @@ -5065,9 +5182,58 @@ PostgresMain(const char *dbname, const char *username) portal_name = pq_getmsgstring(&input_message); max_rows = pq_getmsgint(&input_message, 4); + + /* + * Read the fetch fields of the _pq_.cursor extension. + * Like the extension's Bind field, they are mandatory + * once the extension is active, so that the server never + * has to guess from the message length whether they are + * there. A client with nothing to say sends + * PQ_FETCH_DEFAULT and a zero count. + */ + if (MyProcPort != NULL && MyProcPort->protocol_cursor_enabled) + { + fetch_flags = pq_getmsgint(&input_message, 4); + fetch_count = pq_getmsgint64(&input_message); + + if (fetch_flags & ~PQ_FETCH_VALID_FLAGS) + ereport(ERROR, + (errcode(ERRCODE_PROTOCOL_VIOLATION), + errmsg("unrecognized fetch flags in Execute message: 0x%x", + fetch_flags))); + + if ((fetch_flags & PQ_FETCH_DIRECTION_MASK) == PQ_FETCH_DEFAULT) + { + /* + * Without a direction there is nothing for the + * other fields to mean, so insist that they are + * empty rather than silently ignoring them. + */ + if (fetch_flags != PQ_FETCH_DEFAULT) + ereport(ERROR, + (errcode(ERRCODE_PROTOCOL_VIOLATION), + errmsg("fetch flags in Execute message require a fetch direction"))); + if (fetch_count != 0) + ereport(ERROR, + (errcode(ERRCODE_PROTOCOL_VIOLATION), + errmsg("fetch count in Execute message requires a fetch direction"))); + } + else if (max_rows != 0) + { + /* + * The fetch count says how many rows are wanted, + * so the row-count field has no say. Reject a + * conflicting one instead of picking a winner. + */ + ereport(ERROR, + (errcode(ERRCODE_PROTOCOL_VIOLATION), + errmsg("Execute message cannot specify both a maximum row count and a fetch direction"))); + } + } pq_getmsgend(&input_message); - exec_execute_message(portal_name, max_rows); + exec_execute_message(portal_name, max_rows, + fetch_flags, fetch_count); /* exec_execute_message does valgrind_report_error_query */ } diff --git a/src/include/libpq/protocol.h b/src/include/libpq/protocol.h index 0a2271ef8a2..90afb03a7ad 100644 --- a/src/include/libpq/protocol.h +++ b/src/include/libpq/protocol.h @@ -122,4 +122,33 @@ PQ_BIND_CURSOR_NO_SCROLL | \ PQ_BIND_CURSOR_HOLD) +/* + * Execute message extension flags for _pq_.cursor. + * + * The low three bits select a fetch direction, with the same meaning as the + * identically named direction of the SQL FETCH command. PQ_FETCH_DEFAULT + * asks for no fetch behavior at all: the row-count field of the Execute + * message governs, exactly as it does without the extension. + * + * These values are part of the wire protocol and must not change. They are + * deliberately independent of the server's FetchDirection enum (parsenodes.h) + * so that it can be reordered or extended freely. + */ +#define PQ_FETCH_DEFAULT 0x0000 /* honor the row-count field */ +#define PQ_FETCH_FORWARD 0x0001 /* FETCH FORWARD count */ +#define PQ_FETCH_BACKWARD 0x0002 /* FETCH BACKWARD count */ +#define PQ_FETCH_ABSOLUTE 0x0003 /* FETCH ABSOLUTE count */ +#define PQ_FETCH_RELATIVE 0x0004 /* FETCH RELATIVE count */ +#define PQ_FETCH_DIRECTION_MASK 0x0007 +#define PQ_FETCH_MOVE 0x0008 /* reposition only, no rows */ +#define PQ_FETCH_VALID_FLAGS (PQ_FETCH_DIRECTION_MASK | \ + PQ_FETCH_MOVE) + +/* + * Portable spelling of "all remaining rows" for the fetch count. The + * server-internal FETCH_ALL is LONG_MAX, which is platform dependent, so the + * wire protocol reserves the largest Int64 instead. + */ +#define PQ_FETCH_ALL INT64_MAX + #endif /* PROTOCOL_H */