pgSCV, a Prometheus-compatible monitoring agent and metrics exporter for PostgreSQL, released.
Pgpool-II 4.2.3, 4.1.7, 4.0.14, 3.7.19 and 3.6.26, a connection pooler and statement replication system for PostgreSQL, released.
sqlite_fdw 1.2.0 released.
Crunchy PostgreSQL Operator 4.7.0, a system for deploying and managing open source PostgreSQL clusters on Kubernetes, released.
pgAdmin4 5.3, a web- and native GUI control center for PostgreSQL, released.
InfluxDB fdw 1.0.0 released https://github.com/pgspider/influxdb_fdw
griddb_fdw 2.0 released.
https://archives.postgresql.org/pgsql-jobs/2021-05/
Planet PostgreSQL: https://planet.postgresql.org/
PostgreSQL Weekly News is brought to you this week by David Fetter
Submit news and announcements by Sunday at 3:00pm PST8PDT to david@fetter.org.
David Rowley pushed:
Add missing NULL check when building Result Cache paths. Code added in 9e215378d to disable building of Result Cache paths when not all join conditions are part of the parameterization of a unique join failed to first check if the inner path's param_info was set before checking the param_info's ppi_clauses. Add a check for NULL values here and just bail on trying to build the path if param_info is NULL. lateral_vars are not considered when deciding if the join is unique, so we're not missing out on doing the optimization when there are lateral_vars and no param_info. Reported-by: Coverity, via Tom Lane Discussion: https://postgr.es/m/457998.1621779290@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/99c5852e20a0987eca1c38ba0c09329d4076b6a0
Fix setrefs.c code for Result Cache nodes. Result Cache, added in 9eacee2e6 neglected to properly adjust the plan references in setrefs.c. This could lead to the following error during EXPLAIN: ERROR: cannot decompile join alias var in plan tree Fix that. Bug: 17030 Reported-by: Hans Buschmann Discussion: https://postgr.es/m/17030-5844aecae42fe223@postgresql.org https://git.postgresql.org/pg/commitdiff/cba5c70b956810c61b3778f7041f92fbb8065acb
Tom Lane pushed:
Doc: move some catalogs.sgml entries to the right place. pg_statistic_ext_data.stxdexpr was listed under the wrong catalog, as was pg_stats_ext.exprs. Also there was a bogus entry for pg_statistic_ext_data.stxexprs. Apparently a merge failure in commit a4d75c86b. Guillaume Lelarge and Tom Lane Discussion: https://postgr.es/m/CAECtzeUHw+w64eUFVeV_2FJviAw6oZ0wNLkmU843ZH4hAQfiWg@mail.gmail.com https://git.postgresql.org/pg/commitdiff/713a431c781fbfe1a22fae4991836077f0f4c513
Fix use of uninitialized variable in inline_function(). Commit e717a9a18 introduced a code path that bypassed the call of get_expr_result_type, which is not good because we need its rettupdesc result to pass to check_sql_fn_retval. We'd failed to notice right away because the code path in which check_sql_fn_retval uses that argument is fairly hard to reach in this context. It's not impossible though, and in any case inline_function would have no business assuming that check_sql_fn_retval doesn't need that value. To fix, move get_expr_result_type out of the if-block, which in turn requires moving the construction of the dummy FuncExpr out of it. Per report from Ranier Vilela. (I'm bemused by the lack of any compiler complaints...) Discussion: https://postgr.es/m/CAEudQAqBqQpQ3HruWAGU_7WaMJ7tntpk0T8k_dVtNB46DqdBgw@mail.gmail.com https://git.postgresql.org/pg/commitdiff/e30e3fdea873e4e9517c490232ea1d3bcef6c643
Rethink definition of pg_attribute.attcompression. Redefine '\0' (InvalidCompressionMethod) as meaning "if we need to compress, use the current setting of default_toast_compression". This allows '\0' to be a suitable default choice regardless of datatype, greatly simplifying code paths that initialize tupledescs and the like. It seems like a more user-friendly approach as well, because now the default compression choice doesn't migrate into table definitions, meaning that changing default_toast_compression is usually sufficient to flip an installation's behavior; one needn't tediously issue per-column ALTER SET COMPRESSION commands. Along the way, fix a few minor bugs and documentation issues with the per-column-compression feature. Adopt more robust APIs for SetIndexStorageProperties and GetAttributeCompression. Bump catversion because typical contents of attcompression will now be different. We could get away without doing that, but it seems better to ensure v14 installations all agree on this. (We already forced initdb for beta2, anyway.) Discussion: https://postgr.es/m/626613.1621787110@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/e6241d8e030fbd2746b3ea3f44e728224298f35b
Reduce the range of OIDs reserved for genbki.pl. Commit ab596105b increased FirstBootstrapObjectId from 12000 to 13000, but we've had some push-back about that. It's worrisome to reduce the daylight between there and FirstNormalObjectId, because the number of OIDs consumed during initdb for collation objects is hard to predict. We can improve the situation by abandoning the assumption that these OIDs must be globally unique. It should be sufficient for them to be unique per-catalog. (Any code that's unhappy about that is broken anyway, since no more than per-catalog uniqueness can be guaranteed once the OID counter wraps around.) With that change, the largest OID assigned during genbki.pl (starting from a base of 10000) is a bit under
This allows reverting FirstBootstrapObjectId to 12000 with reasonable confidence that that will be sufficient for many years to come. We are not, at this time, abandoning the expectation that hand-assigned OIDs (below 10000) are globally unique. Someday that'll likely be necessary, but the need seems years away still. This is late for v14, but it seems worth doing it now so that downstream software doesn't have to deal with the consequences of a change in FirstBootstrapObjectId. In any case, we already bought into forcing an initdb for beta2, so another catversion bump won't hurt. Discussion: https://postgr.es/m/1665197.1622065382@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/a4390abecf0f5152cff864e82b67e5f6c8489698
Doc: improve libpq service-file docs, avoid overspecifying pathnames. Clarify libpq.sgml's description of service file locations and semantics. Avoid use of backtick'ed pg_config calls to describe paths; that doesn't work on Windows, and even on Unix it's an idiom that not all readers may be instantly familiar with. Don't overspecify the locations of include files, instead writing only as much as you'd use in #include directives. The previous text in these places was incorrect for some installations, depending on where "postgresql" is in the install path. Our convention for referencing the user's home directory seems to be "~", so change the one place that spelled it "$HOME". install-windows.sgml follows the platform convention of spelling file paths with "\", so change the one place that used "/". Haiying Tang and Tom Lane Discussion: https://postgr.es/m/162149020918.26174.7150424047314144297@wrigleys.postgresql.org https://git.postgresql.org/pg/commitdiff/ba356a397de565c014384aa01a945aab7d50928c
Peter Geoghegan pushed:
Consider triggering VACUUM failsafe during scan. The wraparound failsafe mechanism added by commit 1e55e7d1 handled the one-pass strategy case (i.e. the "table has no indexes" case) by adding a dedicated failsafe check. This made up for the fact that the usual one-pass checks inside lazy_vacuum_all_indexes() cannot ever be reached during a one-pass strategy VACUUM. This approach failed to account for two-pass VACUUMs that opt out of index vacuuming up-front. The INDEX_CLEANUP off case in the only case that works like that. Fix this by performing a failsafe check every 4GB during the first scan of the heap, regardless of the details of the VACUUM. This eliminates the special case, and will make the failsafe trigger more reliably. Author: Peter Geoghegan pg@bowt.ie Reported-By: Andres Freund andres@anarazel.de Reviewed-By: Masahiko Sawada sawada.mshk@gmail.com Discussion: https://postgr.es/m/20210424002921.pb3t7h6frupdqnkp@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/c242baa4a831ac2e7dcaec85feb410aefa3a996e
Fix VACUUM VERBOSE's LP_DEAD item pages output. Oversight in commit 5100010e. https://git.postgresql.org/pg/commitdiff/9afdea982420f9672b88e5c17d1ee8eec64105fc
Michaël Paquier pushed:
Disallow SSL renegotiation. SSL renegotiation is already disabled as of
48d23c72, however this does not prevent the server to comply with a client
willing to use renegotiation. In the last couple of years, renegotiation had
its set of security issues and flaws (like the recent CVE-2021-3449), and it
could be possible to crash the backend with a client attempting renegotiation.
This commit takes one extra step by disabling renegotiation in the backend in
the same way as SSL compression (f9264d15) or tickets (97d3a0b0). OpenSSL
1.1.0h has added an option named SSL_OP_NO_RENEGOTIATION able to achieve that.
In older versions there is an option called SSL3_FLAGS_NO_RENEGOTIATE_CIPHERS
that was undocumented, and could be set within the SSL object created when the
TLS connection opens, but I have decided not to use it, as it feels trickier
to rely on, and it is not official. Note that this option is not usable in
OpenSSL < 1.1.0h as the internal contents of the *SSL object are hidden to
applications. SSL renegotiation concerns protocols up to TLSv1.2. Per
original report from Robert Haas, with a patch based on a suggestion by Andres
Freund. Author: Michael Paquier Reviewed-by: Daniel Gustafsson Discussion:
https://postgr.es/m/YKZBXx7RhU74FlTE@paquier.xyz
Backpatch-through: 9.6
https://git.postgresql.org/pg/commitdiff/01e6f1a842f406170e5f717305e4a6cf0e84b3ee
Fix memory leak when de-toasting compressed values in VACUUM FULL/CLUSTER. VACUUM FULL and CLUSTER can be used to enforce the use of the existing compression method of a toastable column if a value currently stored is compressed with a method that does not match the column's defined method. The code in charge of decompressing and recompressing toast values at rewrite left around the detoasted values, causing an accumulation of memory allocated in TopTransactionContext. When processing large relations, this could cause the system to run out of memory. The detoasted values are not needed once their tuple is rewritten, and this commit ensures that the necessary cleanup happens. Issue introduced by bbe0a81d. The comments of the area are reordered a bit while on it. Reported-by: Andres Freund Analyzed-by: Andres Freund Author: Michael Paquier Reviewed-by: Dilip Kumar Discussion: https://postgr.es/m/20210521211929.pcehg6f23icwstdb@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/fb0f5f0172edf9f63c8f70ea9c1ec043b61c770e
Fix typo in heapam.c. Author: Hou Zhijie Discussion: https://postgr.es/m/OS0PR01MB571612191738540B27A8DE5894249@OS0PR01MB5716.jpnprd01.prod.outlook.com https://git.postgresql.org/pg/commitdiff/190fa5a00a8f9ecee8eef2c8e26136b772b94e19
doc: Fix description of some GUCs in docs and postgresql.conf.sample. The following parameters have been imprecise, or incorrect, about their description (PGC_POSTMASTER or PGC_SIGHUP): - autovacuum_work_mem (docs, as of 9.6~) - huge_page_size (docs, as of 14~) - max_logical_replication_workers (docs, as of 10~) - max_sync_workers_per_subscription (docs, as of 10~) - min_dynamic_shared_memory (docs, as of 14~) - recovery_init_sync_method (postgresql.conf.sample, as of 14~) - remove_temp_files_after_crash (docs, as of 14~) - restart_after_crash (docs, as of 9.6~) - ssl_min_protocol_version (docs, as of 12~) - ssl_max_protocol_version (docs, as of 12~) This commit adjusts the description of all these parameters to be more consistent with the practice used for the others. Revewed-by: Justin Pryzby Discussion: https://postgr.es/m/YK2ltuLpe+FbRXzA@paquier.xyz Backpatch-through: 9.6 https://git.postgresql.org/pg/commitdiff/2941138e60fc711bd221b3264807f36cc079dfbb
Fix MSVC scripts when building with GSSAPI/Kerberos. The deliverables of upstream Kerberos on Windows are installed with paths that do not match our MSVC scripts. First, the include folder was named "inc/" in our scripts, but the upstream MSIs use "include/". Second, the build would fail with 64-bit environments as the libraries are named differently. This commit adjusts the MSVC scripts to be compatible with the latest installations of upstream, and I have checked that the compilation was able to work with the 32-bit and 64-bit installations. Special thanks to Kondo Yuta for the help in investigating the situation in hamerkop, which had an incorrect configuration for the GSS compilation. Reported-by: Brian Ye Discussion: https://postgr.es/m/162128202219.27274.12616756784952017465@wrigleys.postgresql.org Backpatch-through: 9.6 https://git.postgresql.org/pg/commitdiff/025110663448a8c877f4b591495f2e5d187d8936
Amit Kapila pushed:
Improve docs and error messages for parallel vacuum. The error messages, docs, and one of the options were using 'parallel degree' to indicate parallelism used by vacuum command. We normally use 'parallel workers' at other places so change it for parallel vacuum accordingly. Author: Bharath Rupireddy Reviewed-by: Dilip Kumar, Amit Kapila Backpatch-through: 13 Discussion: https://postgr.es/m/CALj2ACWz=PYrrFXVsEKb9J1aiX4raA+UBe02hdRp_zqDkrWUiw@mail.gmail.com https://git.postgresql.org/pg/commitdiff/0734b0e983443882ec509ab4501c30ba9b706f5f
Doc: Update logical decoding stats information. Add the information of pg_stat_replication_slots view along with other system catalogs related to logical decoding. Author: Vignesh C Reviewed-by: Amit Kapila Discussion: https://postgr.es/m/20210319185247.ldebgpdaxsowiflw@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/0c6b92d9c6fb74255467573fde54f65139b26603
Fix assertion during streaming of multi-insert toast changes. While decoding
the multi-insert WAL we can't clean the toast untill we get the last insert of
that WAL record. Now if we stream the changes before we get the last change,
the memory for toast chunks won't be released and we expect the txn to have
streamed all changes after streaming. This restriction is mainly to ensure
the correctness of streamed transactions and it doesn't seem worth uplifting
such a restriction just to allow this case because anyway we will stream the
transaction once such an insert is complete. Previously we were using two
different flags (one for toast tuples and another for speculative inserts) to
indicate partial changes. Now instead we replaced both of them with a single
flag to indicate partial changes. Reported-by: Pavan Deolasee Author: Dilip
Kumar Reviewed-by: Pavan Deolasee, Amit Kapila Discussion:
https://postgr.es/m/CABOikdN-_858zojYN-2tNcHiVTw-nhxPwoQS4quExeweQfG1Ug@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/6f4bdf81529fdaf6744875b0be99ecb9bfb3b7e0
Peter Eisentraut pushed:
postgresql.conf.sample: Make vertical spacing consistent. https://git.postgresql.org/pg/commitdiff/8673a37c85fef00dd5b9c04197538142bec10542
Replace run-time error check with assertion. The error message was checking that the structures returned from the parser matched expectations. That's something we usually use assertions for, not a full user-facing error message. So replace that with an assertion (hidden inside lfirst_node()). Reviewed-by: Tom Lane tgl@sss.pgh.pa.us Discussion: https://www.postgresql.org/message-id/flat/452e9df8-ec89-e01b-b64a-8cc6ce830458%40enterprisedb.com https://git.postgresql.org/pg/commitdiff/388e75ad33489b77cfb9a8590a91e9287d8fb960
Add NO_INSTALL option to pgxs. Apply in libpq_pipeline test makefile, so that the test file is not installed into tmp_install. Reviewed-by: Alvaro Herrera alvherre@alvh.no-ip.org Reviewed-by: Tom Lane tgl@sss.pgh.pa.us Discussion: https://www.postgresql.org/message-id/flat/cb9d16a6-760f-cd44-28d6-b091d5fb6ca7%40enterprisedb.com https://git.postgresql.org/pg/commitdiff/6abc8c2596dbfcb24f9b4d954a1465b8015118c3
Fix vpath build in libpq_pipeline test. The path needs to be set to refer to the build directory, not the current directory, because that's actually the source directory at that point. fix for 6abc8c2596dbfcb24f9b4d954a1465b8015118c3 https://git.postgresql.org/pg/commitdiff/a717e5c771610cf8607f2423ab3ab6b5d30f44ea
Álvaro Herrera pushed:
Make detach-partition-concurrently-3 less timing-sensitive. This recently added test has shown to be too sensitive to timing when sending a cancel to a session waiting for a lock. We fix this by running a no-op query in the blocked session immediately after the cancel; this avoids the session that sent the cancel sending another query immediately before the cancel has been reported. Idea by Noah Misch. With that fix, we sometimes see that the cancel error report is shown only relative to the step that is cancelled, instead of together with the step that sends the cancel. To increase the probability that both steps are shown togeter, add a 0.1s sleep to the cancel. In normal conditions this appears sufficient to silence most failures, but we'll see that the slower buildfarm members say about it. Reported-by: Takamichi Osumi osumi.takamichi@fujitsu.com Discussion: https://postgr.es/m/OSBPR01MB4888C4ABA361C7E81094AC66ED269@OSBPR01MB4888.jpnprd01.prod.outlook.com https://git.postgresql.org/pg/commitdiff/5e0b1aeb2dfed4f1eb7ac5154c1573885a70db41
Make detach-partition-concurrently-4 less timing sensitive. Same as 5e0b1aeb2dfe, for the companion test file. This one seems lower probability (only two failures in a month of runs); I was hardly able to reproduce a failure without a patch, so the fact that I was also unable to reproduce one with it doesn't say anything. We'll have to wait for further buildfarm results to see if we need any further adjustments. Discussion: https://postgr.es/m/20210524090712.GA3771394@rfd.leadboat.com https://git.postgresql.org/pg/commitdiff/eb43bdbf5104c183412aac0fccf8e515e60d9212
Andrew Dunstan pushed:
Report configured port in MSVC built pg_config. This is a long standing omission, discovered when trying to write code that relied on it. Backpatch to all live branches. https://git.postgresql.org/pg/commitdiff/fb424ae85f6b1e32e545f13902d3ba3429be44df
fix syntax error. https://git.postgresql.org/pg/commitdiff/d69fcb9caef1ac1f38241645d4fb9f7e0ce02a70
Thomas Munro pushed:
Fabien COELHO and Aleksander Alekseev traded patches to replace the rand48 pseudo-random number generator with something more appropriate to the 64-bit era.
Greg Nancarrow and Pavel Borisov traded patches to fix a parallel worker failed assertion and coredump.
Vigneshwaran C sent in another revision of a patch to add schema level support for PUBLICATIONs.
Hou Zhijie and Amit Langote traded patches to skip partition tuple routing in the case where the partition key is constant.
Dilip Kumar, Tsutomu Yamada, and Kyotaro HORIGUCHI traded patches to fix a race condition in recovery.
Hou Zhijie sent in another revision of a patch to make it possible to parallelize INSERT ... SELECT.
Tom Lane sent in two revisions of a patch to fix an infelicity between CALL and procedures with output-only arguments.
Justin Pryzby sent in another revision of a patch to make WAL compression methods pluggable and default to lz4.
Andy Fan sent in a patch to use planner_rt_fetch instead of rt_fetch when root->simple_rte_array.
Ajin Cherian sent in five more revisions of a patch to skip empty transactions for logical replication.
Mark Dilger sent in another revision of a patch to delegate superuser tasks to new security roles.
Hou Zhijie, Bharath Rupireddy, and Tomáš Vondra traded patches to make sure postgres_fdw batching does not use too many parameters.
Bharath Rupireddy sent in a patch to provide TDE nonce size as an initdb option, add TDE nonce bytes to page pd_special structure, and adjust tests to account for configurable TDE nonce size.
Bharath Rupireddy sent in another revision of a patch to disambiguate error messages that use "non-negative."
Antonin Houska sent in a patch to narrow the cases where a concurrent UPDATE restarts heap_lock_tuple(), reducing unneeded calls to same.
Greg Sabino Mullane sent in a patch to speed up pg_checksums in cases where checksum already set by avoiding writing same when it's already set to the value expected.
Michaël Paquier sent in a patch intended to fix a bug that manifested as incorrect snapshots while promoting hot standby node when 2PC is used.
Bharath Rupireddy sent in two more revisions of a patch to check for duplicate options and error out if they're found in CREATE COLLATION.
Andrey V. Lepikhov sent in a patch to teach the optimizer to consider a partition-wise join of a non-partitioned table with each partition of a partitioned table.
Andrey V. Lepikhov sent in another revision of a patch to remove unneeded self-joins via a new equivalence class.
Peter Eisentraut sent in a patch to fix RADIUS error reporting in hba file parsing.
Vigneshwaran C and Bharath Rupireddy traded patches to improve publication error messages.
Tom Lane sent in another revision of a patch to replace pg_depend PIN entries with a fixed range check.
Kyotaro HORIGUCHI sent in two revisions of a patch to change confusing 'bracket' use to clearer wording, and add test cases for trailing garbage of (multi)range types.
Bharath Rupireddy and Hou Zhijie traded patches to make it possible to use parallel inserts in CREATE TABLE AS.
Dilip Kumar sent in a patch to fix a memory leak in decoding speculative insert with toast.
Peter Geoghegan sent in a patch to Generalize VACUUM's INDEX_CLEANUP option in a way that lets users disable the index vacuum bypassing optimization added by commit 5100010e, as well as any similar optimizations that may be added in the future.
Paul Guo sent in another revision of a patch to fsync only the affected files/directories and use copy_file_range() for file copying in pg_rewind.
Takamichi Osumi sent in another revision of a patch to log the plans of long-running queries.
Daniel Gustafsson sent in another revision of a patch to support NSS as a libpq TLS backend.
Etsuro Fujita sent in a patch to fix rescans in asynchronous appends in the PostgreSQL FDW.
Vigneshwaran C sent in a patch to add alias types regpublication and regsubscription.
Fabien COELHO sent in a patch to reduce some duplication in psql's echo code.
Laurenz Albe sent in another revision of a patch to expand the docs on PostgreSQL extension coding and background worker development a little so that key topics like allocation, interrupt handling, exit callbacks, transaction callbacks, PG_TRY()/PG_CATCH(), resource owners, transaction and snapshot state, etc are at least briefly mentioned with a few pointers to where to learn more.
Tom Lane sent in a patch to reduce memory consumption for pending inval messages.
Andreas Karlsson sent in a patch to shrink GISTSTATE.
Yura Sokolov sent in a patch to clear empty space in a page.
Tomáš Vondra sent in a patch to revert part of the COPY FREEZE improvements, namely tweaking heap_multi_insert, and removing most of 39b66a91bd (except for the heap_xlog_multi_insert bit).
Thomas Munro sent in two revisions of a patch to support direct I/O on macOS.