pgCluu 3.2, a Perl program to audit PostgreSQL performance, released.
PGroonga 2.3.2 a full text search platform for all languages, released.
https://archives.postgresql.org/pgsql-jobs/2021-10/
Planet PostgreSQL: https://planet.postgresql.org/
PostgreSQL Weekly News is brought to you this week by David Fetter
Submit news and announcements by Sunday at 3:00pm PST8PDT to david@fetter.org.
Michaël Paquier pushed:
Fix snapshot builds during promotion of hot standby node with 2PC. Some specific logic is done at the end of recovery when involving 2PC transactions: 1) Call RecoverPreparedTransactions(), to recover the state of 2PC transactions into memory (re-acquire locks, etc.). 2) ShutdownRecoveryTransactionEnvironment(), to move back to normal operations, mainly cleaning up recovery locks and KnownAssignedXids (including any 2PC transaction tracked previously). 3) Switch XLogCtl->SharedRecoveryState to RECOVERY_STATE_DONE, which is the tipping point for any process calling RecoveryInProgress() to check if the cluster is still in recovery or not. Any snapshot taken between steps 2) and 3) would be empty, causing any transaction relying on a snapshot at this point to potentially corrupt data as there could still be some 2PC transactions to track, with RecentXmin moving backwards on successive calls to GetSnapshotData() in the same transaction. As SharedRecoveryState is the point to take into account to know if it is safe to discard KnownAssignedXids, this commit moves step 2) after step 3), so as we can never finish with empty snapshots. This exists since the introduction of hot standby, so backpatch all the way down. The window with incorrect snapshots is extremely small, but I have seen it when running 023_pitr_prepared_xact.pl, as did buildfarm member fairywren. Thomas Munro also found it independently. Special thanks to Andres Freund for taking the time to analyze this issue. Reported-by: Thomas Munro, Michael Paquier Analyzed-by: Andres Freund Discussion: https://postgr.es/m/20210422203603.fdnh3fu2mmfp2iov@alap3.anarazel.de Backpatch-through: 9.6 https://git.postgresql.org/pg/commitdiff/8a4237908c0fe73dd41d4d7c7a6314f17dfd7a6f
Fix warning in TAP test of pg_verifybackup. Oversight in a3fcbcd. Reported-by: Thomas Munro Discussion: https://postgr.es/m/CA+hUKGKnajZEwe91OTjro9kQLCMGGFHh2vvFn8tgHgbyn4bF9w@mail.gmail.com Backpatch-through: 13 https://git.postgresql.org/pg/commitdiff/ec2133a447318ac6d78887e91940d69e6d92a435
Refactor per-destination file rotation in logging collector. stderr and csvlog have been using duplicated code when it came to the rotation of their file by size, age or if forced by a user request (pg_ctl logrotate or the SQL function pg_rotate_logfile). The main difference between both is that stderr requires its file to always be opened, so as it is possible to have a redirection route if the logging collector is not ready yet to do its work if alternate destinations are enabled. Also, if csvlog gets disabled, we need to close properly its meta-data stored in the logging collector (last file name for current_logfiles and fd currently open for business). Except for those points, the code is the same in terms of error handling and if a file should be created or just continued. This change makes the code simpler overall, and it will help in the introduction of more file-based log destinations. This refactoring is similar to the work done in 5b0b699. Most of the duplication originates from fd801f4. Some of the TAP tests of pg_ctl check the case of a forced log rotation, but this is somewhat limited as there is no coverage for log_rotation_age or log_rotation_size (these may not be worth the extra resources to run either), and no coverage for reload of log_destination with different combinations of stderr and csvlog. I have tested all those cases separately for this refactoring. Author: Michael Paquier Discussion: https://postgr.es/m/CAH7T-aqswBM6JWe4pDehi1uOiufqe06DJWaU5=X7dDLyqUExHg@mail.gmail.com https://git.postgresql.org/pg/commitdiff/5c6e33f071537d9831db57471a06d39a175b535a
Fix compilation warning in syslogger.c. Oversight in 5c6e33f. Author: Nathan Bossart Discussion: https://postgr.es/m/DD8AD4CE-63B7-44BE-A3D2-14A4E4B19C26@amazon.com https://git.postgresql.org/pg/commitdiff/05c4248ad1bf0c2721ce9445f6908da9ece36ff8
Refactor fallback to stderr for csvlog to handle better WIN32 service case. send_message_to_server_log() would force a redirection of a log entry to stderr in some cases for csvlog, like the syslogger not being available yet. If this happens, csvlog would fall back to stderr to log some information rather than nothing. The code was organized so as stderr is done before csvlog, with csvlog checking that stderr did not happen yet with a reversed condition. With this code organization, it could be possible to lose some messages if running Postgres as a service on WIN32, as there is no usable stderr, and the handling of the StringInfoData holding the message for stderr was rather confusing because of that. This commit moves the csvlog handling to be before stderr, as as we are able to track down if it is necessary to log something to stderr. The reduces the handling of stderr to be in a single code path, adding a fallback to event logs for a WIN32 service. This also simplifies the way we handle the StringInfoData for stderr, making easier the integration of new file-based log destinations. I got to play with services and event logs on Windows while checking this change. Reviewed-by: Chris Bandy Discussion: https://postgr.es/m/YV0vwBovEKf1WXkl@paquier.xyz https://git.postgresql.org/pg/commitdiff/8b76f89c37973082b3d64f5a27937efcca9d65f6
Daniel Gustafsson pushed:
Replace occurrences of InvalidXid with InvalidTransactionId. While Xid is a known shortening of TransactionId, InvalidXid is not defined in the code. Fix comments which mistakenly were using the shorter version. Author: Bharath Rupireddy bharath.rupireddyforpostgres@gmail.com Discussion: https://postgr.es/m/CALj2ACUQzdigML868nV4cojfELPkEzNLNOk7b91Pho4JB90fng@mail.gmail.com https://git.postgresql.org/pg/commitdiff/941921b875c7710e2b070c02c7819f2510808fdd
Provide error hint if TAP tests are not enabled. The error message for trying to run the TAP tests in a tree not configured with --enable-tap-tests is quite terse, and could be made more helpful to new developers onboarding to postgres. This adds a small hint on how to get the tests running in such cases. Author: Kevin Burke kevin@burke.dev Discussion: https://postgr.es/m/CAKcy5ejKVYwUXguQcd6i9KHDm7cM7FzjQ+aayaPveoa_woyQpQ@mail.gmail.com https://git.postgresql.org/pg/commitdiff/b5cb4db91327c2cef66207bde9cbcb592b91f93c
Provide error hint on exit() check when building libpq. Commit dc227eb82 introduced a restriction on libpq that no functions which invoke exit() are allowed to be called. This was further refined and fixed in e45b0dfa1f and 2f7bae2f92 and 792259591. While this is well documented in the Makefile, the error message emitted when the check failed was terse, without hints for new developers without prior context. This adds an error hint to assist new developers onboarding to postgres. Author: Rachel Heaton rheaton@vmware.com Co-authored-by: Jacob Champion pchampion@vmware.com Discussion: https://postgr.es/m/CADJcwiVL20955HCNzDqz9BEDr6A77pz6-nac5sbZVvhAEMijLg@mail.gmail.com https://git.postgresql.org/pg/commitdiff/e9bc0441f1446f6614fa6712841acec91890e089
Fix duplicate words in comments. Remove accidentally duplicated words in code comments. Author: Dagfinn Ilmari Mannsåker ilmari@ilmari.org Discussion: https://postgr.es/m/87bl45t0co.fsf@wibble.ilmari.org https://git.postgresql.org/pg/commitdiff/7111e332c57ddb562d0ce26a4e08761a0baafb65
Fix check for trapping exit() calls in libpq. Commit e9bc0441f added an errorhint on the exit() check for libpq, but accidentally changed the nm commandline to use -a instead of -A. These options are similar enough to hide it in testing, but -a can also show debugger symbols which isn't what we want. Fix by reverting the check back to using -A again. Reported-by: Anton Voloshin a.voloshin@postgrespro.ru Discussion: https://postgr.es/m/bd2c8409-d6b3-5de9-ba0f-40c1381f630f@postgrespro.ru https://git.postgresql.org/pg/commitdiff/de744e9efbc55288572d1e81168c74ea85a4b90a
Peter Eisentraut pushed:
Update Unicode map text files. A couple of newer ones are available. There are no functional differences, but let's get them in anyway, so that there is no surprise diff next time someone wants to do some actual work in this area. https://git.postgresql.org/pg/commitdiff/ce27c8953e8e48c69c690c0e5795cde40ed59fd2
Make Unicode makefile parallel-safe. Fix the rules so that each rule is parallel safe, using the same trickery that we use elsewhere in the tree for rules that produce more than one output file. Refactor the whole makefile so that there is less repetition. Discussion: https://www.postgresql.org/message-id/18e34084-aab1-1b4c-edd1-c4f9fb04f714%40enterprisedb.com https://git.postgresql.org/pg/commitdiff/e752727195798c324e769cfebf9dc4baa1c6bb0c
Fix loop variable signedness. https://git.postgresql.org/pg/commitdiff/ba216d3b54ac334729c505ec8a725db3826290a2
Improve order in file. Move support functions for new PublicationTable node to more sensible locations in the files. https://git.postgresql.org/pg/commitdiff/d942887039a608c91084a942fe10571c6f6be35a
Tom Lane pushed:
Doc: fix minor issues in GiST support function documentation. gist.sgml and xindex.sgml hadn't been fully updated for the addition of a sortsupport support function (commit 16fa9b2b3). xindex.sgml also missed that the compress and decompress support functions are optional, an apparently far older oversight. In passing, fix gratuitous inconsistencies in wording and capitalization. Noted by E. Rogov. Back-patch to v14; the residual issues before that aren't significant enough to bother with. Discussion: https://postgr.es/m/163335322905.12519.5711557029494638051@wrigleys.postgresql.org https://git.postgresql.org/pg/commitdiff/36d1a848a86afd2855215af2a112b9bde999354a
Update our mapping of Windows time zone names some more. Per discussion, let's just follow CLDR's default zone mappings faithfully. There are two changes here that are clear improvements: * Mapping "Greenwich Standard Time" to Atlantic/Reykjavik is actually a better fit than using London, because Iceland hasn't observed DST since 1968, so this is more nearly what people might expect. * Since the "Samoa" zone is specified to be UTC+13:00, we must map it to Pacific/Apia not Pacific/Samoa; the latter refers to American Samoa which is now on the other side of the date line. The rest of these changes look like they're choosing the most populous IANA zone as representative. Whatever the details, we're just going to say "if you don't like this mapping, complain to CLDR". Discussion: https://postgr.es/m/3266414.1633045628@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/c1aa3b3c0d2125cb04df8ed0387448d8aeb9519c
Doc: improve description of UNION/INTERSECT/EXCEPT syntax. queries.sgml failed to mention the rather important point that INTERSECT binds more tightly than UNION or EXCEPT. I thought it could also use more discussion of the role of parentheses in these constructs. Per gripe from Christopher Painter-Wakefield. Discussion: https://postgr.es/m/163338891727.12510.3939775743980651160@wrigleys.postgresql.org https://git.postgresql.org/pg/commitdiff/f3fec23dbdead113700fb1b401b681fa24f1e4f4
Doc: improve timezone/README's recipe for tracking Windows zones. We should now cite CLDR as primary reference for the zone name mapping. Discussion: https://postgr.es/m/3266414.1633045628@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/db692b0c84908b4ef5ea4c15fa2d742582ad2cf9
Fix null-pointer crash in postgres_fdw's conversion_error_callback. Commit c7b7311f6 adjusted conversion_error_callback to always use information from the query's rangetable, to avoid doing catalog lookups in an already-failed transaction. However, as a result of the utterly inadequate documentation for make_tuple_from_result_row, I failed to realize that fsstate could be NULL in some contexts. That led to a crash if we got a conversion error in such a context. Fix by falling back to the previous coding when fsstate is NULL. Improve the commentary, too. Per report from Andrey Borodin. Back-patch to 9.6, like the previous patch. Discussion: https://postgr.es/m/08916396-55E4-4D68-AB3A-BD6066F9E5C0@yandex-team.ru https://git.postgresql.org/pg/commitdiff/3071bbfe44f36019710190a9273ad2bd4a947878
plperl: update ppport.h to Perl 5.34.0. Also apply the changes suggested by running perl ppport.h --compat-version=5.8.0 And remove some no-longer-required NEED_foo declarations. Dagfinn Ilmari Mannsåker Discussion: https://postgr.es/m/87y278s6iq.fsf@wibble.ilmari.org https://git.postgresql.org/pg/commitdiff/05798c9f7f08908bdd06c82d934da67535b72005
Adjust configure to insist on Perl version >= 5.8.3. Previously it only checked for version >= 5.8.0, although the documentation has said that the minimum version is 5.8.3 since commit dea6ba939. Per the discussion leading up to that commit, I (tgl) left it that way intentionally because you could, at the time, do some bare-bones stuff with 5.8.0. But we aren't actually testing against anything older than 5.8.3, so who knows if that's still true. It's pretty unlikely that anyone would care anyway, so let's just make configure's version check match the docs. Dagfinn Ilmari Mannsåker Discussion: https://postgr.es/m/87y278s6iq.fsf@wibble.ilmari.org Discussion: https://postgr.es/m/16894.1501392088@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/92e6a98c3636948e7ece9a3260f9d89dd60da278
Update test/perl/README to insist on Perl version >= 5.8.3, too. Oversight in previous commit, noted by Daniel Gustafsson. Discussion: https://postgr.es/m/87y278s6iq.fsf@wibble.ilmari.org https://git.postgresql.org/pg/commitdiff/93fb39eca643a33dd6e3c8818fc7899aa67a8103
Doc: update our claims about the minimum recommended AIX version. We currently have buildfarm members testing back to AIX 7.1, but not before, and older AIX versions are long out of support from IBM. So say that 7.1 is the oldest supported version. Discussion: https://postgr.es/m/87y278s6iq.fsf@wibble.ilmari.org https://git.postgresql.org/pg/commitdiff/08e2daf06c71881415ebd19105a8fe53f6eb2f8f
Doc: improve documentation for ^@ starts-with operator. This operator wasn't formally documented anywhere. To give it a natural home, relabel the functions-string-other table as "Other String Functions and Operators", which is more parallel to the functions-string-sql table anyway. While here, add cross-references to the pattern match and text search sections. It seems moderately likely that people would come to this section looking for those (but I don't want to actually list them in these tables). Discussion: https://postgr.es/m/CADT4RqB13KQHOJqqQ+WXmYtJrukS2UiFdtfTvT-XA3qYLyB6Cw@mail.gmail.com https://git.postgresql.org/pg/commitdiff/2ae5d72f004f599c351ee31e8da5fb3e40303760
Andres Freund pushed:
windows: Define WIN32_LEAN_AND_MEAN to make compilation faster. windows.h includes a lot of other headers, slowing down compilation significantly. WIN32_LEAN_AND_MEAN reduces that a bit. It'd be better to remove the include of windows.h (as well as indirect inclusions of it) from such a central place, but until then... Discussion: https://postgr.es/m/20210921193035.pqzay43vpyv7in43@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/8162464a25e5314e753c580389f76a9b7f69445b
Fix TestLib::slurp_file() with offset on windows. 3c5b0685b921 used setFilePointer() to set the position of the filehandle, but passed the wrong filehandle, always leaving the position at 0. Instead of just fixing that, remove use of setFilePointer(), we have a perl fd at this point, so we can just use perl's seek(). Additionally, the perl filehandle wasn't closed, just the windows filehandle. Reviewed-By: Andrew Dunstan andrew@dunslane.net Author: Andres Freund andres@anarazel.de Discussion: https://postgr.es/m/20211003173038.64mmhgxctfqn7wl6@alap3.anarazel.de Backpatch: 9.6-, like 3c5b0685b921 https://git.postgresql.org/pg/commitdiff/2f74db1236fe83e6665e5b0ddad4454c69495614
Bruce Momjian pushed:
Fujii Masao pushed:
psql: Improve tab-completion for LOCK TABLE. This commit makes psql support the tab-completion for ONLY and NOWAIT keywords of LOCK TABLE command. Author: Koyu Tanigawa Reviewed-by: Shinya Kato, Fujii Masao Discussion: https://postgr.es/m/a322684daa36319e6ebc60b541000a3a@oss.nttdata.com https://git.postgresql.org/pg/commitdiff/0b0d277c35533baecc8d1a9356f71de5f2ee0bd8
doc: Document pg_encoding_to_char() and pg_char_to_encoding(). Previously both functions were not described anywhere in the docs. But since they have been around since 7.0 and mentioned in the description for system catalog like pg_database, it's reasonable to add short descriptions for them. Author: Ian Lawrence Barwick Reviewed-by: Laurenz Albe, Fujii Masao Discussion: https://postgr.es/m/CAB8KJ=infievn4q1N4X7Vx8w4_RMPPG0pLvxhSDjy5WQOSHW9g@mail.gmail.com https://git.postgresql.org/pg/commitdiff/f6b5d05ba9a4ac7c5ebec76045c6e0afcf7c9eec
Make recovery report error message when invalid page header is found. Commit 0668719801 changed XLogPageRead() so that it validated the page header, if invalid page header was found reset the error message and retried reading the page, to fix the scenario where streaming standby got stuck at a continuation record. This change hid the error message about invalid page header, which would make it harder for users to investigate what the actual issue was found in WAL. To fix the issue, this commit makes XLogPageRead() report the error message when invalid page header is found. When not in standby mode, an invalid page header should cause recovery to end, not retry reading the page, so XLogPageRead() doesn't need to validate the page header for the retry. Instead, ReadPageInternal() should be responsible for the validation in that case. Therefore this commit changes XLogPageRead() so that if not in standby mode it doesn't validate the page header for the retry. Reported-by: Yugo Nagata Author: Yugo Nagata, Kyotaro Horiguchi Reviewed-by: Ranier Vilela, Fujii Masao Discussion: https://postgr.es/m/20210718045505.32f463ed6c227111038d8ae4@sraoss.co.jp https://git.postgresql.org/pg/commitdiff/68601985e699adeb267636fd19d3d6113554bd1f
Amit Kapila pushed:
Robert Haas pushed:
Flexible options for BASE_BACKUP. Previously, BASE_BACKUP used an entirely hard-coded syntax, but that's hard to extend. Instead, adopt the same kind of syntax we've used for SQL commands such as VACUUM, ANALYZE, COPY, and EXPLAIN, where it's not necessary for all of the option names to be parser keywords. In the new syntax, most of the options now take an optional Boolean argument. To match our practice in other in places, the options which the old syntax called NOWAIT and NOVERIFY_CHECKSUMS options are in the new syntax called WAIT and VERIFY_CHECKUMS, and the default value is false. In the new syntax, the FAST option has been replaced by a CHECKSUM option whose value may be 'fast' or 'spread'. This commit does not remove support for the old syntax. It just adds the new one as an additional option, and makes pg_basebackup prefer the new syntax when the server is new enough to support it. Patch by me, reviewed and tested by Fabien Coelho, Sergei Kornilov, Fujii Masao, and Tushar Ahuja. Discussion: http://postgr.es/m/CA+TgmobAczXDRO_Gr2euo_TxgzaH1JxbNxvFx=HYvBinefNH8Q@mail.gmail.com Discussion: http://postgr.es/m/CA+TgmoZGwR=ZVWFeecncubEyPdwghnvfkkdBe9BLccLSiqdf9Q@mail.gmail.com https://git.postgresql.org/pg/commitdiff/0ba281cb4bf9f5f65529dfa4c8282abb734dd454
Flexible options for CREATE_REPLICATION_SLOT. Like BASE_BACKUP, CREATE_REPLICATION_SLOT has historically used a hard-coded syntax. To improve future extensibility, adopt a flexible options syntax here, too. In the new syntax, instead of three mutually exclusive options EXPORT_SNAPSHOT, USE_SNAPSHOT, and NOEXPORT_SNAPSHOT, there is now a single SNAPSHOT option with three possible values: 'export', 'use', and 'nothing'. This commit does not remove support for the old syntax. It just adds the new one as an additional option, makes pg_receivewal, pg_recvlogical, and walreceiver processes use it. Patch by me, reviewed by Fabien Coelho, Sergei Kornilov, and Fujii Masao. Discussion: http://postgr.es/m/CA+TgmobAczXDRO_Gr2euo_TxgzaH1JxbNxvFx=HYvBinefNH8Q@mail.gmail.com Discussion: http://postgr.es/m/CA+TgmoZGwR=ZVWFeecncubEyPdwghnvfkkdBe9BLccLSiqdf9Q@mail.gmail.com https://git.postgresql.org/pg/commitdiff/0266e98c6b865246c3031bbf55cb15f330134e30
Dean Rasheed pushed:
Etsuro Fujita pushed:
Add missing word to comment in joinrels.c. Author: Amit Langote Backpatch-through: 13 Discussion: https://postgr.es/m/CA%2BHiwqGQNbtamQ_9DU3osR1XiWR4wxWFZurPmN6zgbdSZDeWmw%40mail.gmail.com https://git.postgresql.org/pg/commitdiff/700c73312841bd1a89263f238556ce8d8d916258
postgres_fdw: Fix comments in connection.c. Commit 27e1f1456 missed updating some comments. Reviewed-by: Bharath Rupireddy Backpatch-through: 14 Discussion: https://postgr.es/m/CAPmGK15Q2Nm6U%2Ba_GwskrWFEVBZ9_3VKOvRrprGufpx91M_3Sw%40mail.gmail.com https://git.postgresql.org/pg/commitdiff/972c7c6567fbb02a59b94ede80b17805de1bc03c