From 9f6afda743bf6d574b7f51a432e787f9f0f6911f Mon Sep 17 00:00:00 2001 From: Ashutosh Bapat Date: Tue, 17 Feb 2026 16:51:20 +0530 Subject: [PATCH v20260724 6/7] Resizable shared memory structures Resizable shared memory structures can be requested by specifying a new member ShmemStructOpts::maximum_size. At the startup or when the structure is created, we reserve address space worth maximum_size in the shared memory segment. It is expected that the subsystem which creates a resizable structure would initialize only the memory worth its initial size given by ShmemStructOpts::maximum_size when creating it. In an mmap'ed memory, this should allocate memory worth only the initial size. It should not allocate maximum_size worth of memory initially. As the structure is resized using ShmemResizeStruct() memory is freed or allocated in chunks of memory pages when shrinking and expanding the structure respectively. Optional ShmemStructOpts::minimum_size specification for resizable shared memory structures allows us to enforce that a resizable structure cannot be shrunk below a certain size. If minimum_size is not specified for a resizable structure, it the minimum size defaults to 0. Two additional columns are added to pg_shmem_allocations view to report minimum and maximum size respectively. The structure for which ShmemStructOpts::maximum_size is not specified or is set to 0 is considered as a fixed size structure. Existing calls to ShmemRequestStruct() requsting fixed size structures need no change. For fixed-size structures, the minimum size and maximum size are set to the initial size specified in ShmemRequestStructOpts::size. This makes maximum size and minimum size reported in pg_shmem_allocations view semantically consistent for both fixed-size and resizable structures. As a side effect, a request which specifies same minimum_size and maximum_size will be treated as a fixed size structure. Minimum, maximum and initial sizes of main shared memory area ============================================================= With the addition of resizable shared structures, the main shared memory area has three sizes to be tracked: - a. the minimum amount of shared memory that will always be needed - b. the maximum amount of shared memory that may be allocated when all the resizable structures have grown to their maximum size. It is also is the size of the address space reserved for the main shared memory area. - c. the amount of memory allocated at the server startup i.e. initial allocation. mmap needs to use the maximum size to reserve enough address space to accomodate all the shared structures. But a DBA may provision only the memory required at the startup initially and increase or descrease the memory provision as the demand changes. These three sizes are reported as GUCs shared_memory_minimum_size, shared_memory_maximum_size and shared_memory_initial_size respectively. They replace the old GUC shared_memory_size which used to report both size of the main shared memory area and amount of memory required in it. Since these two things are not the same anymore, a single GUC is not sufficient. These GUCs help DBAs to estimate memory to be provisioned at the startup and during run time as the resizable structures change their sizes. Portability =========== Resizable shared structures feature depends upon existence of function madvise() and constants MADV_REMOVE and MADV_WRITE_POPULATE. On the platforms which do not have these or the shared memory types (e.g. Sys-V) which do not provide ability to free or allocate parts of shared memory, we disable this feature. A run time GUC have_resizable_shmem indicates whether a running server supports resizable structures or not. The commit introduces a compile time flag HAVE_RESIZABLE_SHMEM which is defined if MADV_REMOVE and MADV_WRITE_POPULATE exist. We don't check existence of madvise separately, since existence of the constants implies existence of the function. HAVE_RESIZABLE_SHMEM is not defined in EXEC_BACKEND builds since that's largely used for Windows where the APIs to free and allocate memory from and to a given address space are not known to the author right now. Given that PostgreSQL is used widely on Linux (with shared memory type mmap), providing this feature on Linux benefits most of its users. Once we figure out the required Windows APIs, we will support this feature on Windows as well. Prohibiting access to the unused portion of a resizable structure ================================================================ We provide ShmemProtectStruct() to add protections on the address space reserved for a resizable shared memory structure so that the address space upto its current size is accessible whereas the part beyond that is inaccessible. Since these protections are backend specific, the subsystem using the resizable shared memory structures has to make sure to call ShmemProtectStruct() after every resize before any backend tries to access the portion of the structure between old and the new size. This can coordinated using ProcSignalBarrier mechanism in a running backend. When starting the server, Postmaster adds appropriate protections when creating these structures. In an EXEC_BACKEND case, when a backend starts, the protections are added according to the current size of the backend (through attach_fn call). However, in non-EXEC_BACKEND case, a new backend inherits stale protections from the postmaster. It needs to apply the protections according to the current sizes of these structures. Following points need more discussion. Discussion points ================= adding initial protections in non-EXEC_BACKEND case ---------------------------------------------------------------------- When a backend starts, it needs to add shmem protections it in such a way that a ProcSignalBarrier conveying the protection change is not missed. Hence we do it along side InitLocalDataChecksumState(). This has two problems 1. it delays backend startup process and 2. it loosely ties resizable shared memory structure to ProcSignalBarrier mechanim. Do we consider the complexity worth it? Is there any other way to do it without these two drawbacks? Further, we call ShmemReprotectResizableStructs() twice in the startup sequence, once immediately after the place where AttachSharedMemoryStructs() would be called in EXEC_BACKEND case and then second time after ProcSignalInit(). The first one is needed so that the processes can access the resizable structures safely. The second is needed for the reasons mentioned there. But a change in the protection during these calls will be missed by the backend and thus lead to hazardous access. Can we avoid it? If ProcSignalInit() were to be called nearer to the InitProcess(), it may make calling ShmemReprotectResizeStructs() easier. Protection in the resizing backend vs other backends ---------------------------------------------------- The patch provides an API to protect the address spaces not used because of current size of the resizable structures in every backend. Since every backend has to protect its own address space, a separate API is required. But at the same time, in some cases the resize operation itself needs to change the protection (e.g. when expanding the structure). So there's some asymmetry between the moment when the resizing backend adds protection and when other backends add it. That doesn't affect end result since adding this protection is an idempotent operation. But still there's some ugliness involved. This part also needs more discussion. HAVE_RESIZABLE_SHMEM and have_resizable_shmem --------------------------------------------- The patch defines a build time macro HAVE_RESIZABLE_SHMEM to avoid compilation failure on a build platform where the required system calls or constants are not available. But we also need a run time check to see whether the shared memory type being used allows resizing. Hence we provide a GUC have_resizable_shmem which tells whether a running server supports resizable shared memory or not. Most of the code which deals with the shared memory is under HAVE_RESIZABLE_SHMEM, but there is some related to resizable shared memory outside the macro as well. For example, the members maximum_size, minimum_size are not under this macro. This is mostly to allow writing readable code in both the modes and still provide the same user visible views etc. This division of code within and without macro needs more discussion. TODOs ===== There are some TODOs in the code still. We will address them as we finalize that part of design/code. Comments/suggestions on those is welcome. Idea of using mmap to reserve address space in shared memory segments and allocating memory within that address space on demand was proposed by Dmitry Dolgov <9erthalion6@gmail.com>. Author: Ashutosh Bapat Reviewed-by: Matthias van de Meent --- configure.ac | 4 + doc/src/sgml/config.sgml | 102 +++- doc/src/sgml/ref/postgres-ref.sgml | 8 +- doc/src/sgml/runtime.sgml | 9 +- doc/src/sgml/system-views.sgml | 57 +- doc/src/sgml/xfunc.sgml | 92 +++ meson.build | 16 + src/backend/port/sysv_shmem.c | 259 +++++++- src/backend/port/win32_shmem.c | 64 +- src/backend/postmaster/auxprocess.c | 46 +- src/backend/storage/ipc/ipci.c | 114 ++-- src/backend/storage/ipc/shmem.c | 555 ++++++++++++++++-- src/backend/storage/lmgr/proc.c | 16 + src/backend/utils/init/postinit.c | 10 + src/backend/utils/misc/guc_parameters.dat | 57 +- src/backend/utils/misc/guc_tables.c | 15 +- src/include/catalog/pg_proc.dat | 4 +- src/include/pg_config.h.in | 8 + src/include/pg_config_manual.h | 14 + src/include/storage/ipc.h | 2 +- src/include/storage/pg_shmem.h | 6 +- src/include/storage/shmem.h | 19 + src/include/storage/shmem_internal.h | 2 +- src/test/modules/test_shmem/meson.build | 3 +- ...mem_alloc.pl => 001_fixed_shmem_struct.pl} | 31 + .../t/002_resizable_shmem_struct.pl | 382 ++++++++++++ .../modules/test_shmem/test_shmem--1.0.sql | 55 ++ src/test/modules/test_shmem/test_shmem.c | 504 +++++++++++++++- src/test/regress/expected/rules.out | 7 +- src/tools/pgindent/typedefs.list | 1 + 30 files changed, 2294 insertions(+), 168 deletions(-) rename src/test/modules/test_shmem/t/{001_late_shmem_alloc.pl => 001_fixed_shmem_struct.pl} (58%) create mode 100644 src/test/modules/test_shmem/t/002_resizable_shmem_struct.pl diff --git a/configure.ac b/configure.ac index a331749fcb5..bba12b8c8f4 100644 --- a/configure.ac +++ b/configure.ac @@ -1912,6 +1912,10 @@ AC_CHECK_DECLS([memset_s], [], [], [#define __STDC_WANT_LIB_EXT1__ 1 # This is probably only present on macOS, but may as well check always AC_CHECK_DECLS(F_FULLFSYNC, [], [], [#include ]) +# Linux-specific madvise constants needed for resizable shared memory. See similar checks in meson.build for explanation of why these checks are here. +AC_CHECK_DECLS([MADV_POPULATE_WRITE], [], [], [#include ]) +AC_CHECK_DECLS([MADV_REMOVE], [], [], [#include ]) + AC_REPLACE_FUNCS(m4_normalize([ explicit_bzero getopt diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index aa7b1bd75d2..5b6cc870093 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -12373,6 +12373,20 @@ dynamic_library_path = '/usr/local/lib/postgresql:$libdir' + + have_resizable_shmem (boolean) + + have_resizable_shmem configuration parameter + + + + + Reports whether PostgreSQL supports Resizable shared memory structures. + + + + huge_pages_status (enum) @@ -12552,32 +12566,100 @@ dynamic_library_path = '/usr/local/lib/postgresql:$libdir' - - shared_memory_size (integer) + + shared_memory_initial_size (integer) + + shared_memory_initial_size configuration parameter + + + + + Reports the amount of memory, rounded up to the nearest megabyte, allocated at the server startup in main shared memory area. + + + + + + shared_memory_minimum_size (integer) + + shared_memory_minimum_size configuration parameter + + + + + Reports the minimum amount of memory, rounded up to the nearest megabyte, required in the main shared memory area. + + + + + + shared_memory_maximum_size (integer) + + shared_memory_maximum_size configuration parameter + + + + + Reports the maximum size of the main shared memory area, rounded up + to the nearest megabyte. This is the amount of address space that + must be reserved for the main shared memory area. This is also the maximum amount of memory that may be allocated in the main shared memory area. + + + + + + shared_memory_initial_size_in_huge_pages (integer) - shared_memory_size configuration parameter + shared_memory_initial_size_in_huge_pages configuration parameter - Reports the size of the main shared memory area, rounded up to the - nearest megabyte. + Reports the number of huge pages needed in the main shared memory area + at the startup based on the specified . If huge pages are not supported, this + will be -1. + + + This setting is supported only on Linux. It + is always set to -1 on other platforms. For more + details about using huge pages on Linux, see + . - - shared_memory_size_in_huge_pages (integer) + + shared_memory_minimum_size_in_huge_pages (integer) - shared_memory_size_in_huge_pages configuration parameter + shared_memory_minimum_size_in_huge_pages configuration parameter - Reports the number of huge pages that are needed for the main shared - memory area based on the specified . + Reports the minimum number of huge pages needed in the main shared memory area based on the specified . If huge pages are not supported, this will be -1. + + This setting is supported only on Linux. It + is always set to -1 on other platforms. + + + + + + shared_memory_maximum_size_in_huge_pages (integer) + + shared_memory_maximum_size_in_huge_pages configuration parameter + + + + + Reports the maximum number of huge pages needed in the + main shared memory area based on the specified + . If huge pages are not + supported, this will be -1. + This setting is supported only on Linux. It is always set to -1 on other platforms. For more diff --git a/doc/src/sgml/ref/postgres-ref.sgml b/doc/src/sgml/ref/postgres-ref.sgml index b13a16a117f..3eefd629e06 100644 --- a/doc/src/sgml/ref/postgres-ref.sgml +++ b/doc/src/sgml/ref/postgres-ref.sgml @@ -143,8 +143,12 @@ PostgreSQL documentation This can be used on a running server for most parameters. However, the server must be shut down for some runtime-computed parameters - (e.g., , - , and + (e.g., , + , + , + , + , + , and ). diff --git a/doc/src/sgml/runtime.sgml b/doc/src/sgml/runtime.sgml index d9984910cc4..8ba4d233509 100644 --- a/doc/src/sgml/runtime.sgml +++ b/doc/src/sgml/runtime.sgml @@ -1450,11 +1450,12 @@ export PG_OOM_ADJUST_VALUE=0 CONFIG_HUGETLB_PAGE=y. You will also have to configure the operating system to provide enough huge pages of the desired size. The runtime-computed parameter - reports the number - of huge pages required. This parameter can be viewed before starting the + reports the + number of huge pages that must be available in the pool for postmaster + startup to succeed. This parameter can be viewed before starting the server with a postgres command like: -$ postgres -D $PGDATA -C shared_memory_size_in_huge_pages +$ postgres -D $PGDATA -C shared_memory_maximum_size_in_huge_pages 3170 $ grep ^Hugepagesize /proc/meminfo Hugepagesize: 2048 kB @@ -1465,7 +1466,7 @@ hugepages-1048576kB hugepages-2048kB In this example the default is 2MB, but you can also explicitly request either 2MB or 1GB with to adapt the number of pages calculated by - shared_memory_size_in_huge_pages. + shared_memory_maximum_size_in_huge_pages. While we need at least 3170 huge pages in this example, a larger setting would be appropriate if other programs on the machine diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml index 6b905498337..7c4f4aeba14 100644 --- a/doc/src/sgml/system-views.sgml +++ b/doc/src/sgml/system-views.sgml @@ -4333,8 +4333,46 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx Size of the allocation in bytes including padding. For anonymous allocations, no information about padding is available, so the size and allocated_size columns - will always be equal. Padding is not meaningful for free memory, so - the columns will be equal in that case also. + will always be equal. Padding is not meaningful for free memory, so the + columns will be equal in that case also. For resizable allocations which + may span multiple memory pages, the padding includes the padding due to + page alignment. + + + + + + minimum_size int8 + + + Minimum size in bytes that the resizable allocation can shrink to. Equals + size for fixed-size allocations, anonymous + allocations, and free memory. + + + + + + maximum_size int8 + + + Maximum size in bytes that the resizable allocation can grow to. Equals + size for fixed-size allocations, anonymous + allocations, and free memory. + + + + + + reserved_space int8 + + + Address space reserved for the allocation in bytes. For resizable + structures, this is the total address space reserved to accommodate + growth up to maximum_size, and is greater + than or equal to allocated_size. For + fixed-size allocations, anonymous allocations, and free memory this + is same as allocated_size. @@ -4348,6 +4386,21 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx ShmemRequestHash(). + + Resizable structures are allocations whose size can change while the server + is running. indicates whether the + running server supports resizable allocations. Each such allocation has a + minimum size give by minimum_size it can shrink to + and a maximum size given by maximum_size it can + grow to; enough address space is reserved up front to accommodate its + maximum_size. The aggregate initial, minimum and + maximum sizes of all shared memory allocations are reported by , and respectively (and their + _in_huge_pages counterparts). + + By default, the pg_shmem_allocations view can be read only by superusers or roles with privileges of the diff --git a/doc/src/sgml/xfunc.sgml b/doc/src/sgml/xfunc.sgml index 2b8a11e7ad0..2bfc664d7ca 100644 --- a/doc/src/sgml/xfunc.sgml +++ b/doc/src/sgml/xfunc.sgml @@ -3744,6 +3744,98 @@ my_shmem_init(void *arg) + + Resizable shared memory structures + + + A resizable memory structure can be requested using + ShmemRequestStruct by passing + .maximum_size along with + .size. .maximum_size is + maximum size upto which the structure can grow where as + .size is the initial size of the structure. + Optionally, .minimum_size can be set to the minimum + size that the structure can shrink to. While + contiguous address space worth maximum_size is + allocated to the structure, only memory worth size + bytes is allocated initially. The init_fn should only + initialize the size amount of memory. The actual + memory allocated to this structure at any point in time is given by pg_shmem_allocations.allocated_size + and the address space reserved for this structure is given by pg_shmem_allocations.reserved_space. + + + + The structure can be resized using ShmemResizeStruct by + passing it the structure's name and the + new size which can be anywhere between minimum_size + and maximum_size. If the new size is smaller than the + current size of the structure, the memory between the new size and current + size is freed while keeping the contents of the memory upto new size intact. + If the new size is greater than the current size, memory is allocated upto + new size while keeping the current contents of the structure intact. The + starting address of the structure does not change because of resizing + operation. + + + + ShmemResizeStruct returns true on + success. If the operating system cannot supply the additional memory + required to expand the structure, it emits a WARNING + naming the structure and the number of bytes that could not be allocated, + leaves the structure at its previous size, and returns + false. + + + + ShmemResizeStruct does not coordinate with other + backends that may be accessing the shared structure. The caller is + responsible for ensuring that no backend is accessing the parts of the + structure between old size and the new size while + ShmemResizeStruct is executing. Similarly after the + function has finished, the caller must ensure that each backend knows the + new size before accessing the parts of the structure between the old size + and the new size. Accessing the range [current_size, + maximum_size) results in an undefined operating-system dependent + behaviour. + + + + ShmemProtectStruct can be used to make the range beyond + the current size inaccessible in a backend's address space, so that stray + accesses cause a fault instead of silently succeeding. When a backend + starts, protections for all resizable structures are adjusted according to + their current sizes, so subsystems need not call + ShmemProtectStruct from their per-backend + initialization routines. During runtime after calling + ShmemResizeStruct from one backend, + ShmemProtectStruct should be called from all the + backends that may access the structure. + + + + PostgreSQL's + ProcSignalBarrier mechanism (see + src/include/storage/procsignal.h) may be used to + coordinate the resize across backends and adjusting access to the address + space. + + + + This functionality is available only on the platforms which provide the APIs + necessary to reserve contiguous address space and to allocate or free memory + in that address space on demand. Macro HAVE_RESIZABLE_SHMEM + is defined on such platforms. It can be used to guard code related to + resizing a shared memory structure. The functionality is available on with + mmap'ed memory, so subsystems which use resizable structures may have to + addtionally disable resizable memory usage when shared_memory_type is not + SHMEM_TYPE_MMAP. A GUC is set to + on when this functionality is available in a running + server, off otherwise. + + + Allocating Dynamic Shared Memory after Startup diff --git a/meson.build b/meson.build index f4cde249242..4e0c736470c 100644 --- a/meson.build +++ b/meson.build @@ -2894,6 +2894,22 @@ decl_checks = [ ['timingsafe_bcmp', 'string.h'], ] +# Linux-specific madvise constants needed for resizable shared memory. +# Usually we use AC_CHECK_DECLS to check for function declarations, but in this +# case we are using it to detect existence of constants. These constants are +# used to define HAVE_RESIZABLE_SHMEM which is used in storage/pg_shmem.h as +# well as storage/shmem.h. The first abstracts the APIs to allocate shared +# memory segments from the operating system whereas the second abstracts APIs to +# allocate shared memory to various subsystems. Since they are related but +# orthogonal to each other, including any one of them in the other file doesn't +# make sense. pg_config_manual.h is the only place where HAVE_RESIZABLE_SHMEM +# can be defined and made available to both without including sys/mman.h. But +# for that we need constants that indicate the existence of following defines. +decl_checks += [ + ['MADV_POPULATE_WRITE', 'sys/mman.h'], + ['MADV_REMOVE', 'sys/mman.h'], +] + # Need to check for function declarations for these functions, because # checking for library symbols wouldn't handle deployment target # restrictions on macOS diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c index 2e3886cf9fe..c052776e94c 100644 --- a/src/backend/port/sysv_shmem.c +++ b/src/backend/port/sysv_shmem.c @@ -589,44 +589,113 @@ check_huge_page_size(int *newval, void **extra, GucSource source) return true; } +/* + * Get the page size being used by the shared memory. + * + * The function should be called only after the shared memory has been setup. + */ +size_t +GetOSPageSize(void) +{ + size_t os_page_size; + + Assert(huge_pages_status != HUGE_PAGES_UNKNOWN); + + os_page_size = sysconf(_SC_PAGESIZE); + + /* If huge pages are actually in use, use huge page size */ + if (huge_pages_status == HUGE_PAGES_ON) + GetHugePageSize(&os_page_size, NULL); + + return os_page_size; +} + /* * Creates an anonymous mmap()ed shared memory segment. * - * Pass the requested size in *size. This function will modify *size to the - * actual size of the allocation, if it ends up allocating a segment that is - * larger than requested. + * initial_size is the amount of memory required at the start of the server. + * + * *size is the size of the anonymous memory mapping to create. It will be + * updated to the actual size of the allocation, if it ends up allocating a + * segment that is larger than the requested. When *size > initial_size we want + * to reserve a large memory segment while allocating only initial_size amount of + * memory at the server start. + * + * When using huge pages, we make sure that there are enough huge pages + * configured to cover the initial_size worth of memory. */ static void * -CreateAnonymousSegment(Size *size) +CreateAnonymousSegment(Size initial_size, Size *size) { Size allocsize = *size; void *ptr = MAP_FAILED; int mmap_errno = 0; + bool resizable = (initial_size < allocsize); int mmap_flags = MAP_SHARED | MAP_ANONYMOUS | MAP_HASSEMAPHORE; + Assert(initial_size <= allocsize); + #ifndef MAP_HUGETLB /* PGSharedMemoryCreate should have dealt with this case */ Assert(huge_pages != HUGE_PAGES_ON); #else if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY) { - /* - * Round up the request size to a suitable large value. - */ Size hugepagesize; int huge_mmap_flags; + bool probe_ok = true; + /* Round up the request size to a suitable large value. */ GetHugePageSize(&hugepagesize, &huge_mmap_flags); - if (allocsize % hugepagesize != 0) allocsize = add_size(allocsize, hugepagesize - (allocsize % hugepagesize)); + if (initial_size % hugepagesize != 0) + initial_size = add_size(initial_size, hugepagesize - (initial_size % hugepagesize)); - ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE, - mmap_flags | huge_mmap_flags, -1, 0); - mmap_errno = errno; - if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED) - elog(DEBUG1, "mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m", - allocsize); + /* + * When the total amount of space requested is larger than the initial + * memory requirement, we do not allocate memory worth the entire + * requested space upfront. But we will need to make sure that there + * are enough huge pages configured to cover the initial memory + * requirement. Otherwise, we will choose huge pages map instead of + * falling back to normal pages and the server will not start. Hence + * we first try to map and allocate the initial memory requirement. If + * it fails we fall back to normal pages. If it succeeds, we unmap the + * memory and then try to map the entire requested space without + * allocating memory. + */ + if (resizable) + { + void *probe; + + probe = mmap(NULL, initial_size, PROT_READ | PROT_WRITE, + mmap_flags | huge_mmap_flags, + -1, 0); + if (probe == MAP_FAILED) + { + mmap_errno = errno; + probe_ok = false; + if (huge_pages == HUGE_PAGES_TRY) + elog(DEBUG1, + "huge-page probe mmap(%zu) failed, huge pages disabled: %m", + initial_size); + } + else if (munmap(probe, initial_size) != 0) + elog(ERROR, "munmap(%p, %zu) huge page probe failed: %m", probe, initial_size); + } + + if (probe_ok) + { + if (resizable) + mmap_flags |= MAP_NORESERVE; + ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE, + mmap_flags | huge_mmap_flags, -1, 0); + mmap_errno = errno; + if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED) + elog(DEBUG1, + "mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m", + allocsize); + } } #endif @@ -645,6 +714,10 @@ CreateAnonymousSegment(Size *size) * to non-huge pages. */ allocsize = *size; + + /* Use MAP_NORESERVE when we do not need all the memory up front. */ + if (resizable) + mmap_flags |= MAP_NORESERVE; ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE, mmap_flags, -1, 0); mmap_errno = errno; @@ -693,13 +766,18 @@ AnonymousShmemDetach(int status, Datum arg) * standard header. Also, register an on_shmem_exit callback to release * the storage. * + * initial_size is the amount of memory required at the start of the server. + * Used only when we want to reserve a large memory segment while allocating + * only initial_size amount of memory at the server start. That facility is only + * available when using anonymous shared memory segments. + * * Dead Postgres segments pertinent to this DataDir are recycled if found, but * we do not fail upon collision with foreign shmem segments. The idea here * is to detect and re-use keys that may have been assigned by a crashed * postmaster or backend. */ PGShmemHeader * -PGSharedMemoryCreate(Size size, +PGSharedMemoryCreate(Size initial_size, Size size, PGShmemHeader **shim) { IpcMemoryKey NextShmemSegID; @@ -738,7 +816,7 @@ PGSharedMemoryCreate(Size size, if (shared_memory_type == SHMEM_TYPE_MMAP) { - AnonymousShmem = CreateAnonymousSegment(&size); + AnonymousShmem = CreateAnonymousSegment(initial_size, &size); AnonymousShmemSize = size; /* Register on-exit routine to unmap the anonymous segment */ @@ -754,6 +832,10 @@ PGSharedMemoryCreate(Size size, /* huge pages are only available with mmap */ SetConfigOption("huge_pages_status", "off", PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT); + + /* resizable shared memory is only available with mmap */ + SetConfigOption("have_resizable_shmem", "off", + PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT); } /* @@ -991,3 +1073,148 @@ PGSharedMemoryDetach(void) AnonymousShmem = NULL; } } + +/* + * Make sure that the memory of given size from the given address is released. + * + * The address and size are expected to be page aligned. + * + * Only supported on platforms that support anonymous shared memory. + * + * On success returns true. false if system call fails. + */ +bool +PGSharedMemoryEnsureFreed(void *addr, Size size) +{ +#ifndef HAVE_RESIZABLE_SHMEM + ereport(ERROR, + errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("resizable shared memory is not supported on this platform")); +#else + if (!AnonymousShmem) + ereport(ERROR, + errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("only anonymous shared memory can be freed")); + +#ifdef USE_ASSERT_CHECKING + { + size_t os_page_size = GetOSPageSize(); + + Assert(addr == (void *) TYPEALIGN(os_page_size, addr)); + Assert(size == TYPEALIGN(os_page_size, size)); + } +#endif + + if (madvise(addr, size, MADV_REMOVE) == -1) + { + ereport(WARNING, errmsg("could not free shared memory: %m")); + return false; + } + + return true; +#endif +} + +/* + * Make sure that the memory of given size from the given address is allocated. + * + * The address and size are expected to be page aligned. The caller is + * responsible for ensuring that the range is already writable; this function + * only populates it. + * + * Only supported on platforms that support anonymous shared memory. + * + * On success returns true, false if system call fails. + */ +bool +PGSharedMemoryEnsureAllocated(void *addr, Size size) +{ +#ifndef HAVE_RESIZABLE_SHMEM + ereport(ERROR, + errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("resizable shared memory is not supported on this platform")); +#else + if (!AnonymousShmem) + ereport(ERROR, + errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("only anonymous shared memory can be allocated at runtime")); + +#ifdef USE_ASSERT_CHECKING + { + size_t os_page_size = GetOSPageSize(); + + Assert(addr == (void *) TYPEALIGN(os_page_size, addr)); + Assert(size == TYPEALIGN(os_page_size, size)); + } +#endif + + if (madvise(addr, size, MADV_POPULATE_WRITE) == -1) + { + ereport(WARNING, errmsg("could not allocate shared memory: %m")); + return false; + } + + return true; +#endif +} + +/* + * Set memory protection on the given region of shared memory. + * + * Makes [rw_start, rw_end) readable and writable, and [rw_end, prot_end) + * inaccessible. + * + * All addresses are expected to be page aligned. + * + * Only supported on platforms that support resizable shared memory. + * + * On success returns true, false if system call fails. + */ +bool +PGSharedMemoryProtect(void *rw_start, void *rw_end, void *prot_end) +{ +#ifndef HAVE_RESIZABLE_SHMEM + ereport(ERROR, + errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("resizable shared memory is not supported on this platform")); +#else + + if (!AnonymousShmem) + ereport(ERROR, + errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("only anonymous shared memory can be protected at runtime")); + +#ifdef USE_ASSERT_CHECKING + { + size_t os_page_size = GetOSPageSize(); + + Assert(rw_start == (void *) TYPEALIGN(os_page_size, rw_start)); + Assert(rw_end == (void *) TYPEALIGN(os_page_size, rw_end)); + Assert(prot_end == (void *) TYPEALIGN(os_page_size, prot_end)); + } +#endif + Assert(rw_end >= rw_start); + + if (rw_end > rw_start) + { + if (mprotect(rw_start, (char *) rw_end - (char *) rw_start, + PROT_READ | PROT_WRITE) != 0) + { + ereport(WARNING, errmsg("could not protect shared memory: %m")); + return false; + } + } + + if (prot_end > rw_end) + { + if (mprotect(rw_end, (char *) prot_end - (char *) rw_end, + PROT_NONE) != 0) + { + ereport(WARNING, errmsg("could not protect shared memory: %m")); + return false; + } + } + + return true; +#endif +} diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c index 794e4fcb2ad..d266cdbe226 100644 --- a/src/backend/port/win32_shmem.c +++ b/src/backend/port/win32_shmem.c @@ -200,11 +200,13 @@ EnableLockPagesPrivilege(int elevel) /* * PGSharedMemoryCreate * - * Create a shared memory segment of the given size and initialize its - * standard header. + * Create a shared memory segment of the given size and initialize its standard + * header. initial_size is only relevant when we want to create a larger memory + * segment with only a part of it being allocated initially. We don't support + * that on Windows, so we ignore it. */ PGShmemHeader * -PGSharedMemoryCreate(Size size, +PGSharedMemoryCreate(Size initial_size, Size size, PGShmemHeader **shim) { void *memAddress; @@ -648,3 +650,59 @@ check_huge_page_size(int *newval, void **extra, GucSource source) } return true; } + +/* + * Get the page size used by the shared memory. + * + * The function should be called only after the shared memory has been setup. + */ +size_t +GetOSPageSize(void) +{ + SYSTEM_INFO sysinfo; + size_t os_page_size; + + Assert(huge_pages_status != HUGE_PAGES_UNKNOWN); + + GetSystemInfo(&sysinfo); + os_page_size = sysinfo.dwPageSize; + + /* If huge pages are actually in use, use huge page size */ + if (huge_pages_status == HUGE_PAGES_ON) + GetHugePageSize(&os_page_size, NULL); + + return os_page_size; +} + +/* + * PGSharedMemoryEnsureFreed / PGSharedMemoryEnsureAllocated + * + * Not supported on Windows. These are only meaningful on platforms with + * resizable shared memory (mmap + madvise). + */ +bool +PGSharedMemoryEnsureFreed(void *addr, Size size) +{ + ereport(ERROR, + errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("resizable shared memory is not supported on this platform")); + return false; /* keep compiler quiet */ +} + +bool +PGSharedMemoryEnsureAllocated(void *addr, Size size) +{ + ereport(ERROR, + errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("resizable shared memory is not supported on this platform")); + return false; /* keep compiler quiet */ +} + +bool +PGSharedMemoryProtect(void *rw_start, void *rw_end, void *prot_end) +{ + ereport(ERROR, + errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("resizable shared memory is not supported on this platform")); + return false; /* keep compiler quiet */ +} diff --git a/src/backend/postmaster/auxprocess.c b/src/backend/postmaster/auxprocess.c index ad4bf4bd2a8..07a3b5c5923 100644 --- a/src/backend/postmaster/auxprocess.c +++ b/src/backend/postmaster/auxprocess.c @@ -69,33 +69,43 @@ AuxiliaryProcessMainCommon(void) BaseInit(); /* - * Prevent consuming interrupts between setting ProcSignalInit and setting - * the initial local data checksum value. If a barrier is emitted, and - * absorbed, before local cached state is initialized the state transition - * can be invalid. - */ - HOLD_INTERRUPTS(); - - ProcSignalInit(NULL, 0); - - /* - * Initialize a local cache of the data_checksum_version, to be updated by - * the procsignal-based barriers. + * Prevent consuming interrupts between ProcSignalInit() and the + * initialization of local states that are kept in sync with shared memory + * via procsignal-based barriers. If a barrier is emitted, and absorbed, + * before local cached state is initialized the state transition can be + * invalid. * - * This intentionally happens after initializing the procsignal, otherwise - * we might miss a state change. This means we can get a barrier for the - * state we've just initialized - but it can happen only once. + * These initializations intentionally happen after ProcSignalInit(), + * otherwise we might miss a state change. This means we may also receive + * a barrier for the state we've just initialized, but it can happen only + * once. * * The postmaster (which is what gets forked into the new child process) - * does not handle barriers, therefore it may not have the current value - * of LocalDataChecksumState value (it'll have the value read from the - * control file, which may be arbitrarily old). + * does not handle barriers, therefore its local states may not reflect + * the current state of the shared memory. * * NB: Even if the postmaster handled barriers, the value might still be * stale, as it might have changed after this process forked. */ + HOLD_INTERRUPTS(); + + ProcSignalInit(NULL, 0); + + /* + * LocalDataChecksumState inherited from Postmaster will have the value + * read from the control file, which may be arbitrarily old. Update it. + */ InitLocalDataChecksumState(); + /* + * Refresh per-backend protections for resizable shmem structures, in case + * these structures have been resized since the startup. Structures are + * expected to be kept in sync by respective subsystems using + * procsignal-based barriers. But we modify their protections en-masse + * here, rather than relying on individual subsystems to do it. + */ + ShmemReprotectResizableStructs(); + RESUME_INTERRUPTS(); /* diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c index e149a738c8d..e5f20c4603e 100644 --- a/src/backend/storage/ipc/ipci.c +++ b/src/backend/storage/ipc/ipci.c @@ -52,31 +52,50 @@ RequestAddinShmemSpace(Size size) /* * CalculateShmemSize * Calculates the amount of shared memory needed. + * + * - `initial` is the amount of memory needed when the server startup. + * - `min` is the minimum amount of memory needed when all the resizable + * structures are shrunk to their respective minimum sizes. + * - `max` is the maximum amount of memory needed when all the resizable + * structures are expanded to their respective maximum sizes. It is also the + * size of address space that must be reserved for the shared memory segment. + * + * When no resizable structures are requested, all three totals are identical. + * + * We take some care to ensure that the total size request doesn't overflow + * size_t. If this gets through, we don't need to be so careful during the + * actual allocation phase. */ -Size -CalculateShmemSize(void) +void +CalculateShmemSize(size_t *initial, size_t *min, size_t *max) { - Size size; + size_t initial_req; + size_t min_req; + size_t max_req; + size_t fixed_addins; + + ShmemGetRequestedSize(&initial_req, &min_req, &max_req); /* * Size of the Postgres shared-memory block is estimated via moderately- * accurate estimates for the big hogs, plus 100K for the stuff that's too * small to bother with estimating. * - * We take some care to ensure that the total size request doesn't - * overflow size_t. If this gets through, we don't need to be so careful - * during the actual allocation phase. + * Also include additional requested shmem from preload libraries. + * + * These are not resizable, so they contribute equally to all three + * totals. */ - size = 100000; - size = add_size(size, ShmemGetRequestedSize()); - - /* include additional requested shmem from preload libraries */ - size = add_size(size, total_addin_request); + fixed_addins = add_size(100000, total_addin_request); - /* might as well round it off to a multiple of a typical page size */ - size = add_size(size, 8192 - (size % 8192)); + *initial = add_size(initial_req, fixed_addins); + *min = add_size(min_req, fixed_addins); + *max = add_size(max_req, fixed_addins); - return size; + /* might as well round each off to a multiple of a typical page size */ + *initial = add_size(*initial, 8192 - (*initial % 8192)); + *min = add_size(*min, 8192 - (*min % 8192)); + *max = add_size(*max, 8192 - (*max % 8192)); } #ifdef EXEC_BACKEND @@ -121,18 +140,24 @@ CreateSharedMemoryAndSemaphores(void) { PGShmemHeader *shim; PGShmemHeader *seghdr; - Size size; + size_t initial_size; + size_t min_size; + size_t max_size; Assert(!IsUnderPostmaster); /* Compute the size of the shared-memory block */ - size = CalculateShmemSize(); - elog(DEBUG3, "invoking IpcMemoryCreate(size=%zu)", size); + CalculateShmemSize(&initial_size, &min_size, &max_size); + elog(DEBUG3, "invoking IpcMemoryCreate(initial size=%zu, minimum size=%zu, maximum size=%zu)", + initial_size, min_size, max_size); /* - * Create the shmem segment + * Create the shmem segment. + * + * Reserve enough shared address space to accommodate every requested + * structure grown to its maximum size. */ - seghdr = PGSharedMemoryCreate(size, &shim); + seghdr = PGSharedMemoryCreate(initial_size, max_size, &shim); /* * Make sure that huge pages are never reported as "unknown" while the @@ -189,32 +214,49 @@ void InitializeShmemGUCs(void) { char buf[64]; - Size size_b; - Size size_mb; - Size hp_size; + size_t initial_b; + size_t min_b; + size_t max_b; + size_t hp_size; + size_t size_mb; - /* - * Calculate the shared memory size and round up to the nearest megabyte. - */ - size_b = CalculateShmemSize(); - size_mb = add_size(size_b, (1024 * 1024) - 1) / (1024 * 1024); + CalculateShmemSize(&initial_b, &min_b, &max_b); + + /* Round each size up to the nearest megabyte. */ + size_mb = add_size(initial_b, (1024 * 1024) - 1) / (1024 * 1024); sprintf(buf, "%zu", size_mb); - SetConfigOption("shared_memory_size", buf, + SetConfigOption("shared_memory_initial_size", buf, PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT); - /* - * Calculate the number of huge pages required. - */ + size_mb = add_size(min_b, (1024 * 1024) - 1) / (1024 * 1024); + sprintf(buf, "%zu", size_mb); + SetConfigOption("shared_memory_minimum_size", buf, + PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT); + + size_mb = add_size(max_b, (1024 * 1024) - 1) / (1024 * 1024); + sprintf(buf, "%zu", size_mb); + SetConfigOption("shared_memory_maximum_size", buf, + PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT); + + /* Calculate the number of huge pages required for each size. */ GetHugePageSize(&hp_size, NULL); if (hp_size != 0) { - Size hp_required; + size_t hp_required; + + hp_required = initial_b / hp_size + (initial_b % hp_size != 0); + sprintf(buf, "%zu", hp_required); + SetConfigOption("shared_memory_initial_size_in_huge_pages", buf, + PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT); + + hp_required = min_b / hp_size + (min_b % hp_size != 0); + sprintf(buf, "%zu", hp_required); + SetConfigOption("shared_memory_minimum_size_in_huge_pages", buf, + PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT); - hp_required = size_b / hp_size; - if (size_b % hp_size != 0) - hp_required = add_size(hp_required, 1); + hp_required = max_b / hp_size + (max_b % hp_size != 0); sprintf(buf, "%zu", hp_required); - SetConfigOption("shared_memory_size_in_huge_pages", buf, + SetConfigOption("shared_memory_maximum_size_in_huge_pages", buf, PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT); } diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c index a3d56cf55dd..fbc05f0dc13 100644 --- a/src/backend/storage/ipc/shmem.c +++ b/src/backend/storage/ipc/shmem.c @@ -19,11 +19,11 @@ * methods). The routines in this file are used for allocating and * binding to shared memory data structures. * - * This module provides facilities to allocate fixed-size structures in shared - * memory, for things like variables shared between all backend processes. - * Each such structure has a string name to identify it, specified when it is - * requested. shmem_hash.c provides a shared hash table implementation on top - * of that. + * This module provides facilities to allocate fixed-size as well as resizable + * structures in shared memory, for things like variables shared between all + * backend processes. Each such structure has a string name to identify it, + * specified when it is requested. shmem_hash.c provides a shared hash table + * implementation on top of fixed-size structures. * * Shared memory areas should usually not be allocated after postmaster * startup, although we do allow small allocations later for the benefit of @@ -102,6 +102,23 @@ * (*options->ptr), and calls the attach_fn callback, if any, for additional * per-backend setup. * + * Resizable shared memory structures + * ---------------------------------- + * + * In order to allocate resizable shared memory structures, set + * ShmemRequestStructOpts::maximum_size to the maximum size that the structure + * can grow to. The address space for the maximum size will be reserved at + * startup, but memory is allocated or freed as the structure grows or shrinks + * respectively. ShmemRequestStructOpts::size should be set to the initial size + * of the structure, which is the amount of memory allocated at the startup. + * Optionally, ShmemRequestStructOpts::minimum_size can be set to the minimum + * size that the structure can shrink to. After startup, the structure can be + * resized by calling ShmemResizeStruct(). ShmemResizeStruct() enforces that the + * new size is within [minimum_size, maximum_size]. + * + * While resizable structures can be created after the startup, the memory + * available for them is quite limited. + * * Legacy ShmemInitStruct()/ShmemInitHash() functions * -------------------------------------------------- * @@ -268,6 +285,10 @@ typedef struct void *location; /* location in shared mem */ Size size; /* # bytes requested for the structure */ Size allocated_size; /* # bytes actually allocated */ + Size minimum_size; /* the minimum size the structure can shrink + * to */ + Size maximum_size; /* the maximum size the structure can grow to */ + Size reserved_space; /* the total address space reserved */ } ShmemIndexEnt; /* To get reliable results for NUMA inquiry we need to "touch pages" once */ @@ -276,6 +297,8 @@ static bool firstNumaTouch = true; static void CallShmemCallbacksAfterStartup(const ShmemCallbacks *callbacks); static void InitShmemIndexEntry(ShmemRequest *request); static bool AttachShmemIndexEntry(ShmemRequest *request, bool missing_ok); +static Size EstimateAllocatedSize(ShmemIndexEnt *entry); +static void ShmemProtectStructInternal(ShmemIndexEnt *entry); Datum pg_numa_available(PG_FUNCTION_ARGS); @@ -341,25 +364,63 @@ ShmemRequestInternal(ShmemStructOpts *options, ShmemRequestKind kind) if (options->name == NULL) elog(ERROR, "shared memory request is missing 'name' option"); +#ifndef HAVE_RESIZABLE_SHMEM + if (options->maximum_size > 0) + elog(ERROR, "resizable shared memory is not supported on this platform"); + if (options->minimum_size > 0) + elog(ERROR, "resizable shared memory is not supported on this platform"); +#else + if (options->maximum_size > 0 && shared_memory_type != SHMEM_TYPE_MMAP) + elog(ERROR, "resizable shared memory requires shared_memory_type = mmap"); +#endif + if (IsUnderPostmaster) { if (options->size <= 0 && options->size != SHMEM_ATTACH_UNKNOWN_SIZE) elog(ERROR, "invalid size %zd for shared memory request for \"%s\"", options->size, options->name); + if (options->minimum_size < 0 && options->minimum_size != SHMEM_ATTACH_UNKNOWN_SIZE) + elog(ERROR, "invalid minimum_size %zd for shared memory request for \"%s\"", + options->minimum_size, options->name); + if (options->maximum_size < 0 && options->maximum_size != SHMEM_ATTACH_UNKNOWN_SIZE) + elog(ERROR, "invalid maximum_size %zd for shared memory request for \"%s\"", + options->maximum_size, options->name); } else { - if (options->size == SHMEM_ATTACH_UNKNOWN_SIZE) + if (options->size == SHMEM_ATTACH_UNKNOWN_SIZE || + options->minimum_size == SHMEM_ATTACH_UNKNOWN_SIZE || + options->maximum_size == SHMEM_ATTACH_UNKNOWN_SIZE) elog(ERROR, "SHMEM_ATTACH_UNKNOWN_SIZE cannot be used during startup"); if (options->size <= 0) elog(ERROR, "invalid size %zd for shared memory request for \"%s\"", options->size, options->name); + if (options->minimum_size < 0) + elog(ERROR, "invalid minimum_size %zd for shared memory request for \"%s\"", + options->minimum_size, options->name); + if (options->maximum_size < 0) + elog(ERROR, "invalid maximum_size %zd for shared memory request for \"%s\"", + options->maximum_size, options->name); } if (options->alignment != 0 && pg_nextpower2_size_t(options->alignment) != options->alignment) elog(ERROR, "invalid alignment %zu for shared memory request for \"%s\"", options->alignment, options->name); + if (options->minimum_size > 0 && options->size != SHMEM_ATTACH_UNKNOWN_SIZE && + options->minimum_size > options->size) + elog(ERROR, "resizable shared memory structure \"%s\" should have minimum size (%zd) less than or equal to size (%zd)", + options->name, options->minimum_size, options->size); + + if (options->maximum_size > 0 && options->size > options->maximum_size) + elog(ERROR, "resizable shared memory structure \"%s\" should have maximum size (%zd) greater than size (%zd)", + options->name, options->maximum_size, options->size); + + if (options->minimum_size > 0 && options->maximum_size > 0 && + options->minimum_size > options->maximum_size) + elog(ERROR, "resizable shared memory structure \"%s\" should have minimum size (%zd) less than or equal to maximum size (%zd)", + options->name, options->minimum_size, options->maximum_size); + /* Check that we're in the right state */ if (shmem_request_state != SRS_REQUESTING) elog(ERROR, "ShmemRequestStruct can only be called from a shmem_request callback"); @@ -381,36 +442,70 @@ ShmemRequestInternal(ShmemStructOpts *options, ShmemRequestKind kind) } /* - * ShmemGetRequestedSize() --- estimate the total size of all registered shared - * memory structures. + * ShmemGetRequestedSize() --- estimate total size of all registered shared + * memory structures. + * + * Returns three totals: + * - initial - the sum of initial sizes of the requested structures which is the + * total amount of memory required at the startup. + * - min - the total of minimum sizes of structures + * - max - the sum of maximum sizes of the structures, which is the address + * space that must be reserved. + * + * When there are no resizable structures or on the platforms that do not + * support resizable structures, all three totals are the same. * * This is called at postmaster startup, before the shared memory segment has * been created. */ -size_t -ShmemGetRequestedSize(void) +void +ShmemGetRequestedSize(size_t *initial, size_t *min, size_t *max) { - size_t size; + size_t initial_size; + size_t min_size; + size_t max_size; /* memory needed for the ShmemIndex */ - size = hash_estimate_size(list_length(pending_shmem_requests) + SHMEM_INDEX_ADDITIONAL_SIZE, - sizeof(ShmemIndexEnt)); - size = CACHELINEALIGN(size); + initial_size = hash_estimate_size(list_length(pending_shmem_requests) + SHMEM_INDEX_ADDITIONAL_SIZE, + sizeof(ShmemIndexEnt)); + initial_size = CACHELINEALIGN(initial_size); + min_size = initial_size; + max_size = initial_size; /* memory needed for all the requested areas */ foreach_ptr(ShmemRequest, request, pending_shmem_requests) { size_t alignment = request->options->alignment; + size_t req_min; + size_t req_max; + size_t req_initial = request->options->size; + + if (request->options->maximum_size > 0) + { + req_min = request->options->minimum_size; + req_max = request->options->maximum_size; + } + else + { + req_min = req_initial; + req_max = req_initial; + } /* pad the start address for alignment like ShmemAllocRaw() does */ if (alignment < PG_CACHE_LINE_SIZE) alignment = PG_CACHE_LINE_SIZE; - size = TYPEALIGN(alignment, size); + initial_size = TYPEALIGN(alignment, initial_size); + min_size = TYPEALIGN(alignment, min_size); + max_size = TYPEALIGN(alignment, max_size); - size = add_size(size, request->options->size); + initial_size = add_size(initial_size, req_initial); + min_size = add_size(min_size, req_min); + max_size = add_size(max_size, req_max); } - return size; + *initial = initial_size; + *min = min_size; + *max = max_size; } /* @@ -431,6 +526,23 @@ ShmemInitRequested(void) * Initialize the ShmemIndex entries and perform basic initialization of * all the requested memory areas. There are no concurrent processes yet, * so no need for locking. + * + * TODO: If we have resizable structures, we will mmap with MAP_NORESERVE. + * In case there is not enough memory to cover the initial sizes of the + * shared structures, the mmap will succeed but the initialization will + * fail with SIGBUS. Instead we should allocate the memory worth the + * initial size of each structure using madvise(MADV_WRITE_POPOULATE). But + * instead of doing that for each structure, we should combine contiguous + * structures and do it once for the whole range. The algorithm to use is + * as follows 1. start from the first request 2. note the start address of + * the first structure in the range 3. walk upto a resizable structure or + * the last request, noting the initial end address of the last structure + * in the range 4. call madvise(MADV_WRITE_POPULATE) for the range (start + * address, end address) 5. Next structure becomes the first structure in + * the next range, repeat from step 2 + * + * If there are no resizable structure, no need to do anything, as all the + * memory is reserved during mmap itself. */ foreach_ptr(ShmemRequest, request, pending_shmem_requests) { @@ -514,6 +626,7 @@ InitShmemIndexEntry(ShmemRequest *request) ShmemIndexEnt *index_entry; bool found; size_t allocated_size; + size_t requested_size; void *structPtr; /* look it up in the shmem index */ @@ -531,10 +644,19 @@ InitShmemIndexEntry(ShmemRequest *request) } /* - * We inserted the entry to the shared memory index. Allocate requested - * amount of shared memory for it, and initialize the index entry. + * We inserted the entry to the shared memory index. Allocate requested + * amount of address space in the shared memory segment for it, and do + * basic initializion. The memory gets allocated during initialization as + * the corresponding memory pages are written to. Allocate enough space + * for a resizable structure to grow to its maximum size. It is expected + * that the initialization callback will use only as much memory as the + * initial size of the resizable structure. (Well, if it doesn't, more + * memory will be allocated initially than expected, no further harm is + * done.) */ - structPtr = ShmemAllocRaw(request->options->size, + requested_size = request->options->maximum_size > 0 ? + request->options->maximum_size : request->options->size; + structPtr = ShmemAllocRaw(requested_size, request->options->alignment, &allocated_size); if (structPtr == NULL) @@ -543,13 +665,36 @@ InitShmemIndexEntry(ShmemRequest *request) hash_search(ShmemIndex, name, HASH_REMOVE, NULL); ereport(ERROR, (errcode(ERRCODE_OUT_OF_MEMORY), - errmsg("not enough shared memory for data structure" + errmsg("not enough shared memory space for data structure" " \"%s\" (%zd bytes requested)", - name, request->options->size))); + name, requested_size))); } index_entry->size = request->options->size; index_entry->allocated_size = allocated_size; index_entry->location = structPtr; + index_entry->reserved_space = allocated_size; + if (request->options->maximum_size > 0) + { + index_entry->minimum_size = request->options->minimum_size; + index_entry->maximum_size = request->options->maximum_size; + + /* Adjust allocated size of a resizable structure. */ + index_entry->allocated_size = EstimateAllocatedSize(index_entry); + + /* + * Protect the unused part of the reserved address space for a + * resizable structure. If the structure has same minimum and maximum + * size, it is effectively a fixed-size structure without any unused + * space; no protection is required. + */ + if (index_entry->minimum_size != index_entry->maximum_size) + ShmemProtectStructInternal(index_entry); + } + else + { + index_entry->minimum_size = request->options->size; + index_entry->maximum_size = request->options->size; + } /* Initialize depending on the kind of shmem area it is */ switch (request->kind) @@ -594,7 +739,7 @@ AttachShmemIndexEntry(ShmemRequest *request, bool missing_ok) return false; } - /* Check that the size in the index matches the request */ + /* Check that the sizes in the index match the request. */ if (index_entry->size != request->options->size && request->options->size != SHMEM_ATTACH_UNKNOWN_SIZE) { @@ -604,6 +749,43 @@ AttachShmemIndexEntry(ShmemRequest *request, bool missing_ok) name, index_entry->size, request->options->size))); } + /* + * For resizable structures, also check that minimum_size and maximum_size + * match. For fixed-size structures, these are derived (set to size) in + * the index entry and not meaningful in the request. + */ + if (request->options->maximum_size != 0) + { + if (index_entry->minimum_size != request->options->minimum_size && + request->options->minimum_size != SHMEM_ATTACH_UNKNOWN_SIZE) + { + ereport(ERROR, + errmsg("shared memory struct \"%s\" was created with" + " different minimum_size: existing %zu, requested %zu", + name, index_entry->minimum_size, + request->options->minimum_size)); + } + + if (index_entry->maximum_size != request->options->maximum_size && + request->options->maximum_size != SHMEM_ATTACH_UNKNOWN_SIZE) + { + ereport(ERROR, + errmsg("shared memory struct \"%s\" was created with" + " different maximum_size: existing %zu, requested %zu", + name, index_entry->maximum_size, + request->options->maximum_size)); + } + } + else + { + if (index_entry->minimum_size != index_entry->maximum_size) + elog(ERROR, "shared memory struct \"%s\" was created as resizable, but requested as fixed-size", + name); + } + + if (index_entry->minimum_size != index_entry->maximum_size) + ShmemProtectStructInternal(index_entry); + /* * Re-establish the caller's pointer variable, or do other actions to * attach depending on the kind of shmem area it is. @@ -625,6 +807,291 @@ AttachShmemIndexEntry(ShmemRequest *request, bool missing_ok) return true; } +/* + * Estimate the actual memory allocated for a resizable structure. + * + * ... based on the assumption that the memory is allocated in pages. + * + * The memory pages covered by the current size of a resizable structure are + * considered to be allocated. The memory page where the maximal structure ends + * also hosts the next structure, unless the maximal structure ends on a page + * boundary. Hence that page is allocated because of the next structure. The + * memory pages between the page where the current structure ends and the page + * where the next structure starts remain unallocated. Thus the memory allocated + * for a resizable structure can be estimated as the total address space + * reserved for the structure minus the unallocated memory pages between the + * current end and the next structure. + */ +static Size +EstimateAllocatedSize(ShmemIndexEnt *entry) +{ + Size page_size = GetOSPageSize(); + char *align_end = (char *) TYPEALIGN(page_size, (char *) entry->location + entry->size); + char *floor_max_end = (char *) TYPEALIGN_DOWN(page_size, (char *) entry->location + entry->maximum_size); + + Assert(entry->maximum_size >= entry->size); + Assert(entry->reserved_space >= entry->maximum_size); + + if (align_end < floor_max_end) + return entry->reserved_space - (floor_max_end - align_end); + + return entry->reserved_space; +} + +/* + * ShmemResizeStruct() --- resize a resizable shared memory structure. + * + * The new size must be within [minimum_size, maximum_size]. If the structure + * is being shrunk, the memory pages that are no longer needed are freed. If + * the structure is being expanded, the memory pages that are needed for the + * new size are allocated. See EstimateAllocatedSize() for explanation of which + * pages are allocated for a resizable structure. + * + * The caller must ensure that no other backend is accessing the part of the + * structure between the old size and the new size while this function is + * running. It should also ensure that all backends that may access the + * structure have observed the new size before they access the range between + * the old size and the new size. + * + * If we can not allocate memory pages when expanding the structure, this + * function will return false. On success it returns true. Instead + * of returning false, an error is raised if we can not free the memory pages + * when shrinking the structure, which should not happen in practice. + */ +bool +ShmemResizeStruct(const char *name, Size new_size) +{ +#ifndef HAVE_RESIZABLE_SHMEM + ereport(ERROR, + errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("resizable shared memory is not supported on this platform")); + pg_unreachable(); +#else + ShmemIndexEnt *result; + bool found; + Size page_size = GetOSPageSize(); + char *new_end; + bool success = true; + + Assert(new_size > 0); + + /* + * Resizable shared memory structures are only supported with mmap'ed + * memory. + */ + Assert(shared_memory_type == SHMEM_TYPE_MMAP); + + /* look it up in the shmem index */ + LWLockAcquire(ShmemIndexLock, LW_EXCLUSIVE); + result = (ShmemIndexEnt *) hash_search(ShmemIndex, name, HASH_FIND, &found); + if (!found) + ereport(ERROR, + errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("shmem struct \"%s\" is not initialized", name)); + + Assert(result); + + if (result->minimum_size == result->maximum_size) + ereport(ERROR, + errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("shared memory struct \"%s\" is not resizable", name)); + + if (new_size < result->minimum_size) + ereport(ERROR, + errcode(ERRCODE_INSUFFICIENT_RESOURCES), + errmsg("cannot shrink shared memory structure \"%s\" below minimum size" + "(requested %zu bytes, minimum %zu bytes)", + name, new_size, result->minimum_size)); + + if (result->maximum_size < new_size) + ereport(ERROR, + errcode(ERRCODE_INSUFFICIENT_RESOURCES), + errmsg("not enough address space is reserved for resizing structure \"%s\"" + " (required %zu bytes, reserved %zu bytes)", + name, new_size, result->maximum_size)); + + /* + * A structure requires memory pages from the page containing the start of + * the structure and current end of the structure to be allocated. When + * expanding, we make sure that pages are allocated up to the new end. + * When shrinking, release memory pages beyond the new end, but not the + * page containing maximal end of the structure, as it may be used by the + * next structure. + * + * We do not consider the current end of the structure as it simplifies + * the calculations. Instead we rely on the underlying APIs not to touch + * the memory pages that will not be affected by the change in size. + */ + new_end = (char *) TYPEALIGN(page_size, (char *) result->location + new_size); + if (new_size < result->size) + { + char *max_end = (char *) TYPEALIGN_DOWN(page_size, (char *) result->location + result->maximum_size); + + if (max_end > new_end) + { + if (!PGSharedMemoryEnsureFreed(new_end, max_end - new_end)) + ereport(ERROR, + errcode(ERRCODE_SYSTEM_ERROR), + errmsg("could not free %zu bytes of shared memory from structure \"%s\"", + result->size - new_size, name)); + } + } + else if (new_size > result->size) + { + char *struct_start = (char *) TYPEALIGN_DOWN(page_size, (char *) result->location); + + if (new_end > struct_start) + { + ShmemIndexEnt entry_copy = *result; + + /* + * Allocating memory pages in the expanded range may require the + * corresponding address space to have read-write access. + */ + entry_copy.size = new_size; + ShmemProtectStructInternal(&entry_copy); + + if (!PGSharedMemoryEnsureAllocated(struct_start, new_end - struct_start)) + { + ShmemProtectStructInternal(result); + ereport(WARNING, + errcode(ERRCODE_OUT_OF_MEMORY), + errmsg("could not allocate %zu bytes of shared memory to structure \"%s\"", + new_size - result->size, name)); + success = false; + } + } + } + + /* Update shmem index entry. */ + if (success) + { + result->size = new_size; + result->allocated_size = EstimateAllocatedSize(result); + } + + LWLockRelease(ShmemIndexLock); + + return success; +#endif +} + +/* + * ShmemProtectStruct() --- protect the unused portion of the given resizable + * structure. + * + * Makes the region beyond the current size up to maximum_size inaccessible, and + * ensures the region up to the current size is readable and writable. Depending + * upon the platform, the protection honours the page boundaries. So it may be + * more permissible than strictly needed. + * + * This function only affects the calling backend's address space. After each + * ShmemResizeStruct(), every backend that may access the structure should call + * this function before its next access. When backends are still in the middle + * of that round of ShmemProtectStruct() calls, ShmemResizeStruct() should not + * be called. + */ +void +ShmemProtectStruct(const char *name) +{ +#ifndef HAVE_RESIZABLE_SHMEM + ereport(ERROR, + errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("resizable shared memory is not supported on this platform")); +#else + ShmemIndexEnt *result; + bool found; + + LWLockAcquire(ShmemIndexLock, LW_SHARED); + + result = (ShmemIndexEnt *) hash_search(ShmemIndex, name, HASH_FIND, &found); + if (!found) + ereport(ERROR, + errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("shmem struct \"%s\" is not initialized", name)); + + if (result->minimum_size == result->maximum_size) + ereport(ERROR, + errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("shared memory struct \"%s\" is not resizable", name)); + + ShmemProtectStructInternal(result); + + LWLockRelease(ShmemIndexLock); +#endif +} + +/* + * ShmemProtectStructInternal --- same as ShmemProtectStruct() + * + * ..., but called when the ShmemIndexEnt of the struct is available. The caller + * should hold ShmemIndexLock if required. + */ +static void +ShmemProtectStructInternal(ShmemIndexEnt *entry) +{ +#ifndef HAVE_RESIZABLE_SHMEM + ereport(ERROR, + errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("resizable shared memory is not supported on this platform")); +#else + Size page_size = GetOSPageSize(); + char *rw_start; + char *rw_end; + char *prot_end; + + Assert(shared_memory_type == SHMEM_TYPE_MMAP); + Assert(entry->minimum_size != entry->maximum_size); + + /* Make at least [location, location+size) readable and writable */ + rw_start = (char *) TYPEALIGN_DOWN(page_size, entry->location); + rw_end = (char *) TYPEALIGN(page_size, + (char *) entry->location + entry->size); + + /* + * Make remaining portion inaccessible while making sure that the portion + * after maximum_size is not affected since it may be used by other + * structures. + */ + prot_end = (char *) TYPEALIGN_DOWN(page_size, + (char *) entry->location + entry->maximum_size); + + if (!PGSharedMemoryProtect(rw_start, rw_end, prot_end)) + ereport(ERROR, + errcode(ERRCODE_SYSTEM_ERROR), + errmsg("could not protect shared memory structure \"%s\"", entry->key)); +#endif +} + +/* + * ShmemReprotectResizableStructs() --- re-apply per-backend protection to every + * resizable shmem structure. + * + * On !EXEC_BACKEND platforms, a new backend inherits shmem mappings and their + * per-process protections from the postmaster. If any resizable structure has + * been resized since postmaster start, the inherited protections no longer + * match the current size. This is called early in per-backend startup so the + * new backend sees the protections according to the current sizes of the + * resizable structures. + */ +void +ShmemReprotectResizableStructs(void) +{ +#ifdef HAVE_RESIZABLE_SHMEM + HASH_SEQ_STATUS status; + ShmemIndexEnt *entry; + + LWLockAcquire(ShmemIndexLock, LW_SHARED); + hash_seq_init(&status, ShmemIndex); + while ((entry = (ShmemIndexEnt *) hash_seq_search(&status)) != NULL) + { + if (entry->minimum_size != entry->maximum_size) + ShmemProtectStructInternal(entry); + } + LWLockRelease(ShmemIndexLock); +#endif +} + /* * InitShmemAllocator() --- set up basic pointers to shared memory. * @@ -731,6 +1198,9 @@ InitShmemAllocator(PGShmemHeader *seghdr) Assert(!found); result->size = ShmemAllocator->index_size; result->allocated_size = ShmemAllocator->index_size; + result->minimum_size = result->size; + result->maximum_size = result->size; + result->reserved_space = result->allocated_size; result->location = ShmemAllocator->index; } } @@ -1047,7 +1517,7 @@ ShmemInitStruct(const char *name, Size size, bool *foundPtr) Datum pg_get_shmem_allocations(PG_FUNCTION_ARGS) { -#define PG_GET_SHMEM_SIZES_COLS 4 +#define PG_GET_SHMEM_SIZES_COLS 7 ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; HASH_SEQ_STATUS hstat; ShmemIndexEnt *ent; @@ -1069,7 +1539,20 @@ pg_get_shmem_allocations(PG_FUNCTION_ARGS) values[1] = Int64GetDatum((char *) ent->location - (char *) ShmemSegHdr); values[2] = Int64GetDatum(ent->size); values[3] = Int64GetDatum(ent->allocated_size); - named_allocated += ent->allocated_size; + values[4] = Int64GetDatum(ent->minimum_size); + values[5] = Int64GetDatum(ent->maximum_size); + values[6] = Int64GetDatum(ent->reserved_space); + + /* + * Anonymous areas are allocated in the area remaining after all named + * areas have been allocated. Thus the amount of shared memory + * allocated for anonymous areas can be calculated as the total amount + * of space allocated minus the amount of space allocated for named + * areas. The amount of free shared memory at the end of the segment + * can be calculated as the total size of the segment minus the total + * amount of space allocated. + */ + named_allocated += ent->reserved_space; tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls); @@ -1080,6 +1563,9 @@ pg_get_shmem_allocations(PG_FUNCTION_ARGS) nulls[1] = true; values[2] = Int64GetDatum(ShmemAllocator->free_offset - named_allocated); values[3] = values[2]; + values[4] = values[2]; + values[5] = values[2]; + values[6] = values[2]; tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls); /* output as-of-yet unused shared memory */ @@ -1088,6 +1574,9 @@ pg_get_shmem_allocations(PG_FUNCTION_ARGS) nulls[1] = false; values[2] = Int64GetDatum(ShmemSegHdr->totalsize - ShmemAllocator->free_offset); values[3] = values[2]; + values[4] = values[2]; + values[5] = values[2]; + values[6] = values[2]; tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls); LWLockRelease(ShmemIndexLock); @@ -1274,23 +1763,9 @@ pg_get_shmem_allocations_numa(PG_FUNCTION_ARGS) Size pg_get_shmem_pagesize(void) { - Size os_page_size; -#ifdef WIN32 - SYSTEM_INFO sysinfo; - - GetSystemInfo(&sysinfo); - os_page_size = sysinfo.dwPageSize; -#else - os_page_size = sysconf(_SC_PAGESIZE); -#endif - Assert(IsUnderPostmaster); - Assert(huge_pages_status != HUGE_PAGES_UNKNOWN); - - if (huge_pages_status == HUGE_PAGES_ON) - GetHugePageSize(&os_page_size, NULL); - return os_page_size; + return GetOSPageSize(); } Datum diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c index 9d6e69175a5..780cdb5e9ff 100644 --- a/src/backend/storage/lmgr/proc.c +++ b/src/backend/storage/lmgr/proc.c @@ -575,6 +575,14 @@ InitProcess(void) if (IsUnderPostmaster) AttachSharedMemoryStructs(); #endif + + /* + * Update access to the address space occupied by the resizable shared + * structures. We do it here so that the structures can be accessed safely + * by this backend. But we will do this again after ProcSignalInit() for + * the reasons mentioned there. + */ + ShmemReprotectResizableStructs(); } /* @@ -756,6 +764,14 @@ InitAuxiliaryProcess(void) if (IsUnderPostmaster) AttachSharedMemoryStructs(); #endif + + /* + * Update access to the address space occupied by the resizable shared + * structures. We do it here so that the structures can be accessed safely + * by this backend. But we will do this again after ProcSignalInit() for + * the reasons mentioned there. + */ + ShmemReprotectResizableStructs(); } /* diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c index 3d8c9bdebd5..815b865aa38 100644 --- a/src/backend/utils/init/postinit.c +++ b/src/backend/utils/init/postinit.c @@ -787,6 +787,16 @@ InitPostgres(const char *in_dbname, Oid dboid, */ InitLocalDataChecksumState(); + /* + * Refresh per-backend protections for resizable shmem structures. Usually + * the subsystems using resizable shared structures will use + * ProcSignalBarrier mechanism to coordinate resizing which would involve + * adjusting the protections as well. Like InitLocalDataChecksumState() + * above, this must run after ProcSignalInit so as not to miss a barrier + * for protection change. + */ + ShmemReprotectResizableStructs(); + RESUME_INTERRUPTS(); /* diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index dccc4c82507..2cbec1981c5 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -1226,6 +1226,13 @@ max => '1000.0', }, +{ name => 'have_resizable_shmem', type => 'bool', context => 'PGC_INTERNAL', group => 'PRESET_OPTIONS', + short_desc => 'Shows whether the running server supports resizable shared memory.', + flags => 'GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE', + variable => 'have_resizable_shmem_enabled', + boot_val => 'HAVE_RESIZABLE_SHMEM_ENABLED', +}, + { name => 'hba_file', type => 'string', context => 'PGC_POSTMASTER', group => 'FILE_LOCATIONS', short_desc => 'Sets the server\'s "hba" configuration file.', flags => 'GUC_SUPERUSER_ONLY', @@ -2722,20 +2729,58 @@ max => 'INT_MAX / 2', }, -{ name => 'shared_memory_size', type => 'int', context => 'PGC_INTERNAL', group => 'PRESET_OPTIONS', - short_desc => 'Shows the size of the server\'s main shared memory area (rounded up to the nearest MB).', +{ name => 'shared_memory_initial_size', type => 'int', context => 'PGC_INTERNAL', group => 'PRESET_OPTIONS', + short_desc => 'Shows the amount of memory allocated at server startup in the main shared memory area (rounded up to the nearest MB).', + flags => 'GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE | GUC_UNIT_MB | GUC_RUNTIME_COMPUTED', + variable => 'shared_memory_initial_size_mb', + boot_val => '0', + min => '0', + max => 'INT_MAX', +}, + +{ name => 'shared_memory_initial_size_in_huge_pages', type => 'int', context => 'PGC_INTERNAL', group => 'PRESET_OPTIONS', + short_desc => 'Shows the number of huge pages needed in the main shared memory area at server startup.', + long_desc => '-1 means huge pages are not supported.', + flags => 'GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE | GUC_RUNTIME_COMPUTED', + variable => 'shared_memory_initial_size_in_huge_pages', + boot_val => '-1', + min => '-1', + max => 'INT_MAX', +}, + +{ name => 'shared_memory_maximum_size', type => 'int', context => 'PGC_INTERNAL', group => 'PRESET_OPTIONS', + short_desc => 'Shows the size of the main shared memory area and maximum memory that can be allocated in that area (rounded up to the nearest MB).', + flags => 'GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE | GUC_UNIT_MB | GUC_RUNTIME_COMPUTED', + variable => 'shared_memory_maximum_size_mb', + boot_val => '0', + min => '0', + max => 'INT_MAX', +}, + +{ name => 'shared_memory_maximum_size_in_huge_pages', type => 'int', context => 'PGC_INTERNAL', group => 'PRESET_OPTIONS', + short_desc => 'Shows the maximum number of huge pages needed in the main shared memory area.', + long_desc => '-1 means huge pages are not supported.', + flags => 'GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE | GUC_RUNTIME_COMPUTED', + variable => 'shared_memory_maximum_size_in_huge_pages', + boot_val => '-1', + min => '-1', + max => 'INT_MAX', +}, + +{ name => 'shared_memory_minimum_size', type => 'int', context => 'PGC_INTERNAL', group => 'PRESET_OPTIONS', + short_desc => 'Shows the minimum amount of memory required in the main shared memory area (rounded up to the nearest MB).', flags => 'GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE | GUC_UNIT_MB | GUC_RUNTIME_COMPUTED', - variable => 'shared_memory_size_mb', + variable => 'shared_memory_minimum_size_mb', boot_val => '0', min => '0', max => 'INT_MAX', }, -{ name => 'shared_memory_size_in_huge_pages', type => 'int', context => 'PGC_INTERNAL', group => 'PRESET_OPTIONS', - short_desc => 'Shows the number of huge pages needed for the main shared memory area.', +{ name => 'shared_memory_minimum_size_in_huge_pages', type => 'int', context => 'PGC_INTERNAL', group => 'PRESET_OPTIONS', + short_desc => 'Shows the minimum number of huge pages needed in the main shared memory area.', long_desc => '-1 means huge pages are not supported.', flags => 'GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE | GUC_RUNTIME_COMPUTED', - variable => 'shared_memory_size_in_huge_pages', + variable => 'shared_memory_minimum_size_in_huge_pages', boot_val => '-1', min => '-1', max => 'INT_MAX', diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c index 1ec460b6a82..17d50ffde38 100644 --- a/src/backend/utils/misc/guc_tables.c +++ b/src/backend/utils/misc/guc_tables.c @@ -643,8 +643,12 @@ static int max_index_keys; static int max_identifier_length; static int block_size; static int segment_size; -static int shared_memory_size_mb; -static int shared_memory_size_in_huge_pages; +static int shared_memory_initial_size_mb; +static int shared_memory_minimum_size_mb; +static int shared_memory_maximum_size_mb; +static int shared_memory_initial_size_in_huge_pages; +static int shared_memory_minimum_size_in_huge_pages; +static int shared_memory_maximum_size_in_huge_pages; static int wal_block_size; static int num_os_semaphores; static int effective_wal_level = WAL_LEVEL_REPLICA; @@ -664,6 +668,13 @@ static bool assert_enabled = DEFAULT_ASSERT_ENABLED; #endif static bool exec_backend_enabled = EXEC_BACKEND_ENABLED; +#ifdef HAVE_RESIZABLE_SHMEM +#define HAVE_RESIZABLE_SHMEM_ENABLED true +#else +#define HAVE_RESIZABLE_SHMEM_ENABLED false +#endif +static bool have_resizable_shmem_enabled = HAVE_RESIZABLE_SHMEM_ENABLED; + static char *recovery_target_timeline_string; static char *recovery_target_string; static char *recovery_target_xid_string; diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index f8a021987b5..712172760b3 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8692,8 +8692,8 @@ { oid => '5052', descr => 'allocations from the main shared memory segment', proname => 'pg_get_shmem_allocations', prorows => '50', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,int8,int8}', proargmodes => '{o,o,o,o}', - proargnames => '{name,off,size,allocated_size}', + proallargtypes => '{text,int8,int8,int8,int8,int8,int8}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,off,size,allocated_size,minimum_size,maximum_size,reserved_space}', prosrc => 'pg_get_shmem_allocations', proacl => '{POSTGRES=X,pg_read_all_stats=X}' }, diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in index 661c4a9b168..661c12e9bf3 100644 --- a/src/include/pg_config.h.in +++ b/src/include/pg_config.h.in @@ -85,6 +85,14 @@ don't. */ #undef HAVE_DECL_F_FULLFSYNC +/* Define to 1 if you have the declaration of `MADV_POPULATE_WRITE', and to 0 + if you don't. */ +#undef HAVE_DECL_MADV_POPULATE_WRITE + +/* Define to 1 if you have the declaration of `MADV_REMOVE', and to 0 if you + don't. */ +#undef HAVE_DECL_MADV_REMOVE + /* Define to 1 if you have the declaration of `memset_s', and to 0 if you don't. */ #undef HAVE_DECL_MEMSET_S diff --git a/src/include/pg_config_manual.h b/src/include/pg_config_manual.h index 521b49b8888..ab944babe2b 100644 --- a/src/include/pg_config_manual.h +++ b/src/include/pg_config_manual.h @@ -131,6 +131,20 @@ #define EXEC_BACKEND #endif +/* + * HAVE_RESIZABLE_SHMEM indicates whether resizable shared memory structures are + * supported. The implementation requires Linux-specific madvise constants + * (MADV_REMOVE and MADV_POPULATE_WRITE) and existence of mprotect() API. + * + * TODO: We may want to remove EXEC_BACKEND from the condition to test attaching + * and resizing resizable shared memory structures in EXEC_BACKEND mode. Windows + * will anyway won't have HAVE_RESIZABLE_SHMEM defined since it won't have + * MADV_REMOVE and MADV_POPULATE_WRITE. + */ +#if HAVE_DECL_MADV_REMOVE && HAVE_DECL_MADV_POPULATE_WRITE && !defined(EXEC_BACKEND) +#define HAVE_RESIZABLE_SHMEM +#endif + /* * USE_POSIX_FADVISE controls whether Postgres will attempt to use the * posix_fadvise() kernel call. Usually the automatic configure tests are diff --git a/src/include/storage/ipc.h b/src/include/storage/ipc.h index b205b00e7a1..46ae87fe863 100644 --- a/src/include/storage/ipc.h +++ b/src/include/storage/ipc.h @@ -78,7 +78,7 @@ extern void check_on_shmem_exit_lists_are_empty(void); extern PGDLLIMPORT shmem_startup_hook_type shmem_startup_hook; extern void RegisterBuiltinShmemCallbacks(void); -extern Size CalculateShmemSize(void); +extern void CalculateShmemSize(size_t *initial, size_t *min, size_t *max); extern void CreateSharedMemoryAndSemaphores(void); #ifdef EXEC_BACKEND extern void AttachSharedMemoryStructs(void); diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h index 10c7b065861..50474c2d579 100644 --- a/src/include/storage/pg_shmem.h +++ b/src/include/storage/pg_shmem.h @@ -85,10 +85,14 @@ extern void PGSharedMemoryReAttach(void); extern void PGSharedMemoryNoReAttach(void); #endif -extern PGShmemHeader *PGSharedMemoryCreate(Size size, +extern PGShmemHeader *PGSharedMemoryCreate(Size initial_size, Size max_size, PGShmemHeader **shim); extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2); extern void PGSharedMemoryDetach(void); +extern bool PGSharedMemoryEnsureFreed(void *addr, Size size); +extern bool PGSharedMemoryEnsureAllocated(void *addr, Size size); +extern bool PGSharedMemoryProtect(void *rw_start, void *rw_end, void *prot_end); extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags); +extern size_t GetOSPageSize(void); #endif /* PG_SHMEM_H */ diff --git a/src/include/storage/shmem.h b/src/include/storage/shmem.h index 43b636868c7..947debcede4 100644 --- a/src/include/storage/shmem.h +++ b/src/include/storage/shmem.h @@ -57,6 +57,22 @@ typedef struct ShmemStructOpts */ size_t alignment; + /* + * Minimum size this structure can shrink to. Should be set to 0 for + * fixed-size structures. + */ + ssize_t minimum_size; + + /* + * Maximum size this structure can grow upto in future. The memory is not + * allocated right away but the corresponding address space is reserved so + * that memory can be mapped to it when the structure grows. Typically + * should be used for large resizable structures which need several pages + * worth of contiguous memory. Should be set to 0 for fixed-size + * structures. + */ + ssize_t maximum_size; + /* * When the shmem area is initialized or attached to, pointer to it is * stored in *ptr. It usually points to a global variable, used to access @@ -168,6 +184,9 @@ typedef struct ShmemCallbacks extern void RegisterShmemCallbacks(const ShmemCallbacks *callbacks); extern bool ShmemAddrIsValid(const void *addr); +extern bool ShmemResizeStruct(const char *name, Size new_size); +extern void ShmemProtectStruct(const char *name); +extern void ShmemReprotectResizableStructs(void); /* * These macros provide syntactic sugar for calling the underlying functions diff --git a/src/include/storage/shmem_internal.h b/src/include/storage/shmem_internal.h index 8746b614fa3..6c8c81812a8 100644 --- a/src/include/storage/shmem_internal.h +++ b/src/include/storage/shmem_internal.h @@ -36,7 +36,7 @@ extern void ResetShmemAllocator(void); extern void ShmemRequestInternal(ShmemStructOpts *options, ShmemRequestKind kind); -extern size_t ShmemGetRequestedSize(void); +extern void ShmemGetRequestedSize(size_t *initial, size_t *min, size_t *max); extern void ShmemInitRequested(void); #ifdef EXEC_BACKEND extern void ShmemAttachRequested(void); diff --git a/src/test/modules/test_shmem/meson.build b/src/test/modules/test_shmem/meson.build index fb4bf328b8f..1f795aae7eb 100644 --- a/src/test/modules/test_shmem/meson.build +++ b/src/test/modules/test_shmem/meson.build @@ -27,7 +27,8 @@ tests += { 'bd': meson.current_build_dir(), 'tap': { 'tests': [ - 't/001_late_shmem_alloc.pl', + 't/001_fixed_shmem_struct.pl', + 't/002_resizable_shmem_struct.pl', ], }, } diff --git a/src/test/modules/test_shmem/t/001_late_shmem_alloc.pl b/src/test/modules/test_shmem/t/001_fixed_shmem_struct.pl similarity index 58% rename from src/test/modules/test_shmem/t/001_late_shmem_alloc.pl rename to src/test/modules/test_shmem/t/001_fixed_shmem_struct.pl index 5cf07d071ec..d231821e0b4 100644 --- a/src/test/modules/test_shmem/t/001_late_shmem_alloc.pl +++ b/src/test/modules/test_shmem/t/001_fixed_shmem_struct.pl @@ -56,5 +56,36 @@ else ); } +### +# Test that a fixed-size shared memory structure cannot be resized. +# Only relevant on platforms that support resizable shmem. +### +my $have_resizable_shmem = + $node->safe_psql('postgres', 'SHOW have_resizable_shmem;') eq 'on'; + +if ($have_resizable_shmem) +{ + # Try expanding the fixed-size structure + my ($ret, $stdout, $stderr) = + $node->psql("postgres", "SELECT test_shmem_resize_fixed(1000);"); + isnt($ret, 0, "expanding a fixed-size structure fails"); + like($stderr, qr/is not resizable/, "expand error message mentions not resizable"); + + # Try shrinking the fixed-size structure + ($ret, $stdout, $stderr) = + $node->psql("postgres", "SELECT test_shmem_resize_fixed(1);"); + isnt($ret, 0, "shrinking a fixed-size structure fails"); + like($stderr, qr/is not resizable/, "shrink error message mentions not resizable"); +} + +### +# Test that minimum_size and maximum_size equal size for a fixed-size structure +# in pg_shmem_allocations. +### +is($node->safe_psql('postgres', + "SELECT minimum_size = size AND maximum_size = size FROM pg_shmem_allocations WHERE name = 'test_shmem area';"), + 't', "fixed-size structure has minimum_size = maximum_size = size"); + $node->stop; + done_testing(); diff --git a/src/test/modules/test_shmem/t/002_resizable_shmem_struct.pl b/src/test/modules/test_shmem/t/002_resizable_shmem_struct.pl new file mode 100644 index 00000000000..4d2fd3c4282 --- /dev/null +++ b/src/test/modules/test_shmem/t/002_resizable_shmem_struct.pl @@ -0,0 +1,382 @@ +# Copyright (c) 2025-2026, PostgreSQL Global Development Group + +use strict; +use warnings FATAL => 'all'; + +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; +use Test::More; + +# Test resizable shared memory functionality, both when loaded at startup via +# shared_preload_libraries and when loaded after startup (late allocation). + +# Verify that enough shared memory is allocated to cover the resizable_shmem +# structure at its current size but does not exceed the memory required by the +# current sizes of all shared memory structures. We expect that the backend +# where we run the query will have touched the entire resizable_shmem structure, +# so that all the memory pages covering the resizable structure are mapped to +# the backend's address space. +# +# Since we have configured the server so that resizable shared struture +# dominates the main shared memory segment, the total memory allocated to other +# shared memory structures does not result in false positive tests below. +sub check_shmem_usage +{ + my ($session, $label, $node) = @_; + + my $shmem_usage = $session->query_safe('SELECT test_shmem_usage();', verbose => 0); + my $total_alloc = $node->safe_psql('postgres', "SELECT sum(allocated_size) FROM pg_shmem_allocations;"); + my $resizable_alloc = $node->safe_psql('postgres', + "SELECT allocated_size FROM pg_shmem_allocations WHERE name = 'resizable_shmem';"); + + diag "$label: shmem_usage=$shmem_usage, resizable_shmem allocated=$resizable_alloc, sum(allocated_size)=$total_alloc"; + ok($shmem_usage <= $total_alloc, + "$label: allocated shared memory does not exceed total allocated size"); + ok($shmem_usage >= $resizable_alloc, + "$label: shared memory usage covers the resizable_shmem allocation"); +} + +# Test a resize operation: resize, verify old data, write new data, verify +# new data, and check shmem usage. Returns updated ($num_entries, $value). +sub test_resize +{ + my ($node, $prefix, $old_num_entries, $old_value, $new_num_entries, $new_value, $label) = @_; + + $label = "$prefix: $label"; + + my $session1 = $node->background_psql('postgres'); + my $session2 = $node->background_psql('postgres'); + + $session1->query_safe("SELECT resizable_shmem_resize($new_num_entries);", + verbose => 0); + + # Old data should still be intact in the (possibly smaller) area + my $readable_entries = ($new_num_entries < $old_num_entries) ? $new_num_entries : $old_num_entries; + is($session1->query_safe("SELECT resizable_shmem_read($readable_entries, $old_value);", + verbose => 0), + 't', "old data readable after $label"); + + $session2->query_safe("SELECT resizable_shmem_write($new_value);", + verbose => 0); + is($session1->query_safe("SELECT resizable_shmem_read($new_num_entries, $new_value);", + verbose => 0), + 't', "new data readable after $label"); + + check_shmem_usage($session1, "$label (session 1)", $node); + check_shmem_usage($session2, "$label (session 2)", $node); + + $session1->quit; + $session2->quit; + + return ($new_num_entries, $new_value); +} + +# Verify that reads or writes past the current size, but within the reserved +# maximum, fault when they reach the protected region. +sub test_fault_beyond_size +{ + my ($node, $initial_entries, $prefix) = @_; + + # Enable restart_after_crash to test postmaster driven restart with + # resizable shared memory. + $node->safe_psql('postgres', + 'ALTER SYSTEM SET restart_after_crash = on;'); + $node->reload; + + for my $mode ('write', 'read') + { + my ($ret, $stdout, $stderr) = $node->psql('postgres', + "SELECT resizable_shmem_access_beyond_size('$mode');"); + ok($ret != 0, "$prefix: $mode past current size crashes the backend"); + like($stderr, + qr/server closed the connection unexpectedly|connection to server was lost/, + "$prefix: $mode crash reports lost connection"); + + $node->poll_query_until('postgres', 'SELECT 1', '1') + or die "server did not come back after $mode crash"; + } + + is($node->safe_psql('postgres', + "SELECT resizable_shmem_read($initial_entries, 0);"), + 't', "$prefix: read succeeds after crash recovery"); + + $node->safe_psql('postgres', 'ALTER SYSTEM RESET restart_after_crash;'); + $node->reload; +} + +# Run the full suite of resizable shared memory tests on the given node. +sub run_resizable_tests +{ + my ($node, $initial_entries, $max_entries, $prefix) = @_; + my $have_resizable_shmem = $node->safe_psql('postgres', 'SHOW have_resizable_shmem;') eq 'on'; + + my $num_entries = $initial_entries; + + # Basic read/write should work on all platforms + my $value = 100; + $node->safe_psql('postgres', "SELECT resizable_shmem_write($value);"); + is($node->safe_psql('postgres', "SELECT resizable_shmem_read($num_entries, $value);"), + 't', "$prefix: data read after write successful"); + + if ($have_resizable_shmem) + { + # Initial structure state + my $session1 = $node->background_psql('postgres'); + my $session2 = $node->background_psql('postgres'); + + $value = 100; + # Write and read the initial set of entries. + $session1->query_safe("SELECT resizable_shmem_write($value);", verbose => 0); + is($session2->query_safe("SELECT resizable_shmem_read($num_entries, $value);", + verbose => 0), + 't', "$prefix: data read after write successful"); + check_shmem_usage($session1, "$prefix: initial write (session 1)", $node); + check_shmem_usage($session2, "$prefix: initial write (session 2)", $node); + $session1->quit; + $session2->quit; + + # Verify no other structure is resizable + is($node->safe_psql('postgres', "SELECT count(*) FROM pg_shmem_allocations WHERE name <> 'resizable_shmem' AND maximum_size <> minimum_size;"), + '0', "$prefix: no other resizable structures"); + + # Resize to maximum + ($num_entries, $value) = test_resize($node, $prefix, $num_entries, $value, + $max_entries, 500, 'resize to maximum'); + + # Shrink to 75% of max + my $shrink_entries = int($max_entries * 3 / 4); + ($num_entries, $value) = test_resize($node, $prefix, $num_entries, $value, + $shrink_entries, 999, 'shrinking'); + + # Resize to the same size (no-op) + ($num_entries, $value) = test_resize($node, $prefix, $num_entries, $value, + $num_entries, 1999, 'no-op resize'); + + # Shrink to minimum i.e. zero entries and grow back + ($num_entries, $value) = test_resize($node, $prefix, $num_entries, $value, + 0, 0, 'shrink to minimum'); + ($num_entries, $value) = test_resize($node, $prefix, $num_entries, $value, + $initial_entries, 2999, + 'grow back from minimum'); + + # Test resize failure (attempt to resize beyond max - should fail) + my ($ret, $stdout, $stderr) = + $node->psql('postgres', "SELECT resizable_shmem_resize(" . ($max_entries * 2) . ");"); + ok($ret != 0 || $stderr =~ /ERROR/, "$prefix: Resize beyond maximum fails"); + + # Resize to a size below minimum_size must fail. + ($ret, $stdout, $stderr) = + $node->psql('postgres', 'SELECT resizable_shmem_resize(-1);'); + ok($ret != 0, "$prefix: resize below minimum_size fails"); + like($stderr, + qr/cannot shrink shared memory structure "resizable_shmem" below minimum size/, + "$prefix: resize-below-minimum error comes from ShmemResizeStruct"); + + # The fault test relies on a hole being present between the current end + # of the structure and its maximal end. Skip when the structure does not + # span multiple pages. + my $spans_pages = $node->safe_psql('postgres', qq{ + SELECT (maximum_size - minimum_size) >= test_shmem_pagesize() + FROM pg_shmem_allocations WHERE name = 'resizable_shmem'; + }); + if ($spans_pages ne 't') + { + diag "$prefix: skipping fault-beyond-size test: resizable_shmem does not span multiple shmem pages"; + } + else + { + test_fault_beyond_size($node, $initial_entries, $prefix); + } + } + else + { + # On unsupported platforms, resizing should fail with a clear error + my ($ret, $stdout, $stderr) = + $node->psql('postgres', "SELECT resizable_shmem_resize($num_entries);"); + ok($ret != 0, "$prefix: resize fails on unsupported platform"); + like($stderr, qr/not supported/, "$prefix: resize error mentions not supported"); + } +} + +# Check the runtime-computed shared_memory_{initial,minimum,maximum}_size GUC +# invariants. min <= initial <= max must always hold. When a resizable +# structure has been registered on a server that supports resizable shared +# memory structures, min must additionally be strictly less than max; +# otherwise all three GUCs must be equal. +sub check_shmem_size_gucs +{ + my ($node, $label) = @_; + my $pgdata = $node->data_dir; + + my $get = sub { + my ($guc) = @_; + my ($stdout, $stderr) = run_command([ 'postgres', '-D' => $pgdata, '-C' => $guc ]); + + return $stdout; + }; + + my $have_resizable_shmem = $get->('have_resizable_shmem'); + my $ini = 0 + $get->('shared_memory_initial_size'); + my $min = 0 + $get->('shared_memory_minimum_size'); + my $max = 0 + $get->('shared_memory_maximum_size'); + my $have_resizable_struct = ($get->('resizable_shmem.max_entries') ne ''); + + ok($min <= $ini && $ini <= $max, "$label: shared_memory size GUCs in expected order"); + + if ($have_resizable_struct && $have_resizable_shmem eq 'on') + { + ok($min < $max, "$label: min < max with resizable structures"); + } + else + { + ok($min == $ini && $ini == $max, + "$label: all shared_memory size GUCs equal when no resizable structures"); + } +} + +# Log the runtime shared_memory_{initial,minimum,maximum}_size GUCs and huge +# pages usage information for easier debugging. +sub diag_shmem_sizes +{ + my ($node, $label) = @_; + + my $vals = $node->safe_psql('postgres', q{ + SELECT format('initial=%s minimum=%s maximum=%s huge_pages_status=%s huge_page_size=%s shmem_page_size=%s', + current_setting('shared_memory_initial_size'), + current_setting('shared_memory_minimum_size'), + current_setting('shared_memory_maximum_size'), + current_setting('huge_pages_status'), + current_setting('huge_page_size'), + test_shmem_pagesize()); + }); + diag "$label: $vals"; +} + +### Set up a test node. +# +# Configure minimal shared memory so that the resizable_shmem structure dominates +# and any unexpected increase is easy to detect. +# +# If we turn on huge pages and the machine where the test is running does not +# have huge pages available, the test will fail midway because it will not be +# able to allocate memory pages when expanding the resizable_shmem structure. +# Hence we turn off huge pages for this test. The test outputs the GUCs +# shared_memory_{initial,minimum,maximum}_size and information about huge pages. +# By provisioning enough huge pages, and by changing huge_pages = try/on, the +# test can be run with huge pages enabled. +### +my $node = PostgreSQL::Test::Cluster->new('resizable_shmem'); +$node->init; + +$node->append_conf('postgresql.conf', 'huge_pages = off'); +$node->append_conf('postgresql.conf', 'shared_buffers = 128kB'); +$node->append_conf('postgresql.conf', 'max_connections = 5'); +$node->append_conf('postgresql.conf', 'max_worker_processes = 0'); +$node->append_conf('postgresql.conf', 'max_wal_senders = 0'); +$node->append_conf('postgresql.conf', 'max_prepared_transactions = 0'); +$node->append_conf('postgresql.conf', 'max_locks_per_transaction = 10'); +$node->append_conf('postgresql.conf', 'max_pred_locks_per_transaction = 10'); +$node->append_conf('postgresql.conf', 'wal_buffers = 32kB'); + +### +# Test 1: Startup allocation via shared_preload_libraries +### +my $startup_initial = 25 * 1024 * 1024; +my $startup_max = 100 * 1024 * 1024; + +$node->append_conf('postgresql.conf', 'shared_preload_libraries = test_shmem'); +$node->append_conf('postgresql.conf', "resizable_shmem.initial_entries = $startup_initial"); +$node->append_conf('postgresql.conf', "resizable_shmem.max_entries = $startup_max"); + +check_shmem_size_gucs($node, 'startup preload'); + +$node->start; +$node->safe_psql('postgres', 'CREATE EXTENSION test_shmem;'); +diag_shmem_sizes($node, 'startup'); +run_resizable_tests($node, $startup_initial, $startup_max, 'startup'); + +my $have_resizable_shmem = $node->safe_psql('postgres', 'SHOW have_resizable_shmem;') eq 'on'; + +### +# Test 2: Late allocation (loaded after startup, not in shared_preload_libraries). +# Use much smaller sizes since only ~100KB of shared memory is available for +# structures allocated after startup. +### +my $late_initial = 5 * 1024; +my $late_max = 12 * 1024; + +$node->safe_psql('postgres', qq{ + ALTER SYSTEM RESET shared_preload_libraries; + ALTER SYSTEM SET resizable_shmem.initial_entries = $late_initial; + ALTER SYSTEM SET resizable_shmem.max_entries = $late_max; +}); +$node->safe_psql('postgres', 'DROP EXTENSION test_shmem;'); +$node->restart; + +$node->safe_psql('postgres', 'CREATE EXTENSION test_shmem;'); +diag_shmem_sizes($node, 'late'); +run_resizable_tests($node, $late_initial, $late_max, 'late'); + +### +# Test sysv shared memory does not support resizable shmem. Only relevant on +# platforms that support resizable shmem (HAVE_RESIZABLE_SHMEM), since the +# module only sets maximum_size in that case. +### +if ($have_resizable_shmem) +{ + ### + # Test 3: Verify that CREATE EXTENSION fails with sysv shared memory + # when loaded after startup (not in shared_preload_libraries). + ### + $node->safe_psql('postgres', 'DROP EXTENSION test_shmem;'); + + # Remove settings that would cause the library to auto-load at startup: + # shared_preload_libraries and module-prefixed GUCs. ALTER SYSTEM RESET + # only affects postgresql.auto.conf, so we must use adjust_conf to remove + # from postgresql.conf. + $node->adjust_conf('postgresql.conf', 'shared_preload_libraries', undef); + $node->adjust_conf('postgresql.conf', 'resizable_shmem.initial_entries', undef); + $node->adjust_conf('postgresql.conf', 'resizable_shmem.max_entries', undef); + $node->adjust_conf('postgresql.auto.conf', 'shared_preload_libraries', undef); + $node->adjust_conf('postgresql.auto.conf', 'resizable_shmem.initial_entries', undef); + $node->adjust_conf('postgresql.auto.conf', 'resizable_shmem.max_entries', undef); + $node->safe_psql('postgres', qq{ + ALTER SYSTEM SET shared_memory_type = 'sysv'; + }); + + $node->stop; + + check_shmem_size_gucs($node, 'sysv'); + + $node->start; + + is($node->safe_psql('postgres', 'SHOW have_resizable_shmem;'), + 'off', + 'have_resizable_shmem reports off with shared_memory_type = sysv'); + + my ($ret, $stdout, $stderr) = + $node->psql('postgres', 'CREATE EXTENSION test_shmem;'); + ok($ret != 0, 'CREATE EXTENSION fails with resizable shmem on sysv'); + like($stderr, qr/resizable shared memory requires shared_memory_type = mmap/, + 'CREATE EXTENSION error mentions shared_memory_type = mmap requirement'); + + ### + # Test 4: Verify that resizable structures are also rejected with sysv + # shared memory when loaded at startup via shared_preload_libraries. + ### + $node->safe_psql('postgres', qq{ + ALTER SYSTEM SET shared_preload_libraries = 'test_shmem'; + ALTER SYSTEM SET resizable_shmem.initial_entries = $startup_initial; + ALTER SYSTEM SET resizable_shmem.max_entries = $startup_max; + }); + $node->stop; + + ok(!$node->start(fail_ok => 1), + 'server fails to start with resizable shmem on sysv'); + + my $log = slurp_file($node->logfile); + like($log, qr/resizable shared memory requires shared_memory_type = mmap/, + 'log mentions shared_memory_type = mmap requirement'); +} + +done_testing(); diff --git a/src/test/modules/test_shmem/test_shmem--1.0.sql b/src/test/modules/test_shmem/test_shmem--1.0.sql index 2d01fd9256c..eb695604b1d 100644 --- a/src/test/modules/test_shmem/test_shmem--1.0.sql +++ b/src/test/modules/test_shmem/test_shmem--1.0.sql @@ -4,6 +4,61 @@ \echo Use "CREATE EXTENSION test_shmem" to load this file. \quit +-- =================================================================== +-- Fixed-size shared memory structure +-- =================================================================== + CREATE FUNCTION get_test_shmem_attach_count() RETURNS pg_catalog.int4 STRICT AS 'MODULE_PATHNAME' LANGUAGE C; + +CREATE FUNCTION test_shmem_resize_fixed(pg_catalog.int4) +RETURNS pg_catalog.void STRICT +AS 'MODULE_PATHNAME' LANGUAGE C; + + +-- =================================================================== +-- Resizable shared memory structure +-- =================================================================== + +-- Function to resize the test structure in the shared memory +CREATE FUNCTION resizable_shmem_resize(new_entries integer) +RETURNS bool +AS 'MODULE_PATHNAME' +LANGUAGE C STRICT; + +-- Function to write data to all entries in the test structure in shared memory +-- Writing all the entries makes sure that the memory is actually allocated and +-- mapped to the process, so that we can later measure the memory usage. +CREATE FUNCTION resizable_shmem_write(entry_value integer) +RETURNS void +AS 'MODULE_PATHNAME' +LANGUAGE C STRICT; + +-- Function to verify that specified number of initial entries have expected value. +-- Reading all the entries makes sure that the memory is actually mapped to the +-- process, so that we can later measure the memory usage. +CREATE FUNCTION resizable_shmem_read(entry_count integer, entry_value integer) +RETURNS boolean +AS 'MODULE_PATHNAME' +LANGUAGE C STRICT; + +-- Function to report memory mapped against the main shared memory segment in +-- the backend where this function runs. +CREATE FUNCTION test_shmem_usage() +RETURNS bigint +AS 'MODULE_PATHNAME' +LANGUAGE C STRICT; + +-- Function to get the shared memory page size +CREATE FUNCTION test_shmem_pagesize() +RETURNS integer +AS 'MODULE_PATHNAME' +LANGUAGE C STRICT; + +-- Function to crash the backend by walking entries past the current size up to +-- the reserved maximum, reading or writing each one as decided by mode. +CREATE FUNCTION resizable_shmem_access_beyond_size(mode text) +RETURNS integer +AS 'MODULE_PATHNAME' +LANGUAGE C STRICT; diff --git a/src/test/modules/test_shmem/test_shmem.c b/src/test/modules/test_shmem/test_shmem.c index 9bd4012b435..83f8ea9cc3c 100644 --- a/src/test/modules/test_shmem/test_shmem.c +++ b/src/test/modules/test_shmem/test_shmem.c @@ -1,11 +1,10 @@ /*------------------------------------------------------------------------- * * test_shmem.c - * Helpers to test shmem allocation routines + * Helpers to test shmem management routines * - * Test basic memory allocation in an extension module. One notable feature - * that is not exercised by any other module in the repository is the - * allocating (non-DSM) shared memory after postmaster startup. + * Test fixed-size and resizable shared memory structures created during + * postmaster startup and after startup respectively. * * Copyright (c) 2020-2026, PostgreSQL Global Development Group * @@ -17,13 +16,26 @@ #include "postgres.h" +#include + +#include "commands/extension.h" #include "fmgr.h" #include "miscadmin.h" +#include "storage/fd.h" +#include "storage/pg_shmem.h" #include "storage/shmem.h" +#include "utils/builtins.h" +#include "utils/guc.h" PG_MODULE_MAGIC; + +/* ---------------------------------------------------------------- + * Fixed-size shared memory structure + * ---------------------------------------------------------------- + */ + typedef struct TestShmemData { int value; @@ -35,17 +47,6 @@ static TestShmemData *TestShmem; static bool attached_or_initialized = false; -static void test_shmem_request(void *arg); -static void test_shmem_init(void *arg); -static void test_shmem_attach(void *arg); - -static const ShmemCallbacks TestShmemCallbacks = { - .flags = SHMEM_CALLBACKS_ALLOW_AFTER_STARTUP, - .request_fn = test_shmem_request, - .init_fn = test_shmem_init, - .attach_fn = test_shmem_attach, -}; - static void test_shmem_request(void *arg) { @@ -60,6 +61,17 @@ static void test_shmem_init(void *arg) { elog(LOG, "init callback called"); + + /* + * Reset the per-process flag and the shared "initialized" marker during + * postmaster induced restart. + */ + if (!IsUnderPostmaster) + { + attached_or_initialized = false; + TestShmem->initialized = false; + } + if (TestShmem->initialized) elog(ERROR, "shmem area already initialized"); TestShmem->initialized = true; @@ -82,12 +94,12 @@ test_shmem_attach(void *arg) attached_or_initialized = true; } -void -_PG_init(void) -{ - elog(LOG, "test_shmem module's _PG_init called"); - RegisterShmemCallbacks(&TestShmemCallbacks); -} +static const ShmemCallbacks TestShmemCallbacks = { + .flags = SHMEM_CALLBACKS_ALLOW_AFTER_STARTUP, + .request_fn = test_shmem_request, + .init_fn = test_shmem_init, + .attach_fn = test_shmem_attach, +}; PG_FUNCTION_INFO_V1(get_test_shmem_attach_count); Datum @@ -99,3 +111,453 @@ get_test_shmem_attach_count(PG_FUNCTION_ARGS) elog(ERROR, "shmem area not yet initialized"); PG_RETURN_INT32(TestShmem->attach_count); } + +/* + * Attempt to resize the fixed-size shared memory structure. This should + * fail because the structure was not allocated with a maximum_size. + */ +PG_FUNCTION_INFO_V1(test_shmem_resize_fixed); +Datum +test_shmem_resize_fixed(PG_FUNCTION_ARGS) +{ + int32 new_size = PG_GETARG_INT32(0); + + ShmemResizeStruct("test_shmem area", new_size); + PG_RETURN_VOID(); +} + + +/* ---------------------------------------------------------------- + * Resizable shared memory structure + * ---------------------------------------------------------------- + */ + +/* + * The test module may be loaded after postmaster startup in which case only + * 100K of shared memory is available for the extension. Keep the default + * initial and maximum sizes small enough to fit in that space. + */ +#define TEST_INITIAL_ENTRIES_DEFAULT 1 +#define TEST_MAX_ENTRIES_DEFAULT 1024 + +#define TEST_ENTRY_SIZE sizeof(int32) /* Size of each entry */ + +/* + * Resizable test data structure stored in shared memory. + * + * The test performs resizing, reads or writes, only one at a time and never + * concurrently. Hence, there is no need for locks in the test structure. + */ +typedef struct TestResizableShmemStruct +{ + /* Metadata */ + int32 num_entries; /* Number of entries that can fit */ + + /* Data area - variable size */ + int32 data[FLEXIBLE_ARRAY_MEMBER]; +} TestResizableShmemStruct; + +static TestResizableShmemStruct *resizable_shmem = NULL; + +/* GUC variables controlling the size of the test structure */ +static int test_initial_entries; +static int test_max_entries; + +/* Whether to use SHMEM_ATTACH_UNKNOWN_SIZE when attaching to the shared memory */ +/* TODO: We may use opaque_arg to pass this value to the request function.*/ +static bool use_unknown_size = false; + +/* + * Request shared memory resources. + */ +static void +resizable_shmem_request(void *arg) +{ + Size initial_size = add_size(offsetof(TestResizableShmemStruct, data), + mul_size(test_initial_entries, TEST_ENTRY_SIZE)); + +/* + * Create resizable structure on the platforms which support it. Otherwise create + * as a fixed-size structure. Other way would be to conditionally include + * .maximum_size in the call to ShmemRequestStruct(). + */ +#ifdef HAVE_RESIZABLE_SHMEM + Size max_size = add_size(offsetof(TestResizableShmemStruct, data), + mul_size(test_max_entries, TEST_ENTRY_SIZE)); + Size min_size = offsetof(TestResizableShmemStruct, data); +#else + Size max_size = 0; + Size min_size = 0; +#endif + + ShmemRequestStruct(.name = "resizable_shmem", + .size = use_unknown_size ? SHMEM_ATTACH_UNKNOWN_SIZE : initial_size, + .minimum_size = min_size, + .maximum_size = max_size, + .ptr = (void **) &resizable_shmem, + ); +} + +/* + * Initialize shared memory structure. + */ +static void +resizable_shmem_shmem_init(void *arg) +{ + Assert(resizable_shmem != NULL); + + resizable_shmem->num_entries = test_initial_entries; + memset(resizable_shmem->data, 0, mul_size(test_initial_entries, TEST_ENTRY_SIZE)); +} + +/* + * Attach to the already-allocated shared memory structure. + */ +static void +resizable_shmem_shmem_attach(void *arg) +{ + Assert(resizable_shmem != NULL); +} + +static ShmemCallbacks resizable_shmem_callbacks = { + .request_fn = resizable_shmem_request, + .init_fn = resizable_shmem_shmem_init, + .attach_fn = resizable_shmem_shmem_attach, +}; + +/* + * Resize the shared memory structure to accommodate the specified number of + * entries. + * + * Negative value for new_entries can be used to test resizing below the + * minimum size. + * + * Returns true if the resize was successful, false if ShmemResizeStruct() + * could not allocate the requested memory. On platforms that do not support + * resizable shared memory, ShmemResizeStruct() raises an error. + */ +PG_FUNCTION_INFO_V1(resizable_shmem_resize); +Datum +resizable_shmem_resize(PG_FUNCTION_ARGS) +{ + int32 new_entries = PG_GETARG_INT32(0); + Size new_size; + + if (!resizable_shmem) + ereport(ERROR, + errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("resizable_shmem is not initialized")); + + if (new_entries < 0) + new_size = 1; + else + new_size = add_size(offsetof(TestResizableShmemStruct, data), + mul_size(new_entries, TEST_ENTRY_SIZE)); + if (!ShmemResizeStruct("resizable_shmem", new_size)) + PG_RETURN_BOOL(false); + + ShmemProtectStruct("resizable_shmem"); + resizable_shmem->num_entries = new_entries; + + PG_RETURN_BOOL(true); +} + +/* + * Write the given integer value to all entries in the data array. + */ +PG_FUNCTION_INFO_V1(resizable_shmem_write); +Datum +resizable_shmem_write(PG_FUNCTION_ARGS) +{ + int32 entry_value = PG_GETARG_INT32(0); + int32 i; + + if (!resizable_shmem) + ereport(ERROR, + errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("resizable_shmem is not initialized")); + +#ifdef HAVE_RESIZABLE_SHMEM + + /* + * Ideally the structure should be protected through a synchronization + * cycle across all the backends that may access the structure. But we + * don't implement any such synchronization in this test module to keep it + * simple. Given that ProcSignalBarrier mechanism is not extensible, we + * may not be able to do that as well here. Hence add protect just before + * accessing the structure. + */ + ShmemProtectStruct("resizable_shmem"); +#endif + + for (i = 0; i < resizable_shmem->num_entries; i++) + resizable_shmem->data[i] = entry_value; + + PG_RETURN_VOID(); +} + +/* + * Check whether the first 'entry_count' entries all have the expected 'entry_value'. + * Returns true if all match, false otherwise. + */ +PG_FUNCTION_INFO_V1(resizable_shmem_read); +Datum +resizable_shmem_read(PG_FUNCTION_ARGS) +{ + int32 entry_count = PG_GETARG_INT32(0); + int32 entry_value = PG_GETARG_INT32(1); + int32 i; + + if (resizable_shmem == NULL) + ereport(ERROR, + errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("resizable_shmem is not initialized")); + + if (entry_count < 0 || entry_count > resizable_shmem->num_entries) + ereport(ERROR, + errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("entry_count %d is out of range (0..%d)", entry_count, resizable_shmem->num_entries)); + +#ifdef HAVE_RESIZABLE_SHMEM + + /* + * Ideally the structure should be protected through a synchronization + * cycle across all the backends that may access the structure. But we + * don't implement any such synchronization in this test module to keep it + * simple. Given that ProcSignalBarrier mechanism is not extensible, we + * may not be able to do that as well here. Hence add protect just before + * accessing the structure. + */ + ShmemProtectStruct("resizable_shmem"); +#endif + + for (i = 0; i < entry_count; i++) + { + if (resizable_shmem->data[i] != entry_value) + PG_RETURN_BOOL(false); + } + + PG_RETURN_BOOL(true); +} + +/* + * Return the memory mapped against the main shared memory segment in this + * backend. + * + * The VMA containing our resizable_shmem pointer identifies the start of the + * main shared-memory segment. + * + * mprotect() calls issued when the resizable structure grows and shrinks can + * split the original mmap into several adjacent VMAs, so we sum the accounting + * fields across the base VMA and any VMAs contiguous with it. + */ +PG_FUNCTION_INFO_V1(test_shmem_usage); +Datum +test_shmem_usage(PG_FUNCTION_ARGS) +{ + FILE *f; + char line[256]; + uintptr_t target = (uintptr_t) resizable_shmem; + bool in_target_vma = false; + bool use_hugetlb = (huge_pages_status == HUGE_PAGES_ON); + unsigned long prev_end = 0; + int64 total_rss_kb = 0; + int64 total_swap_kb = 0; + int64 total_shared_hugetlb_kb = 0; + int64 val; + size_t result; + + f = AllocateFile("/proc/self/smaps", "r"); + if (f == NULL) + ereport(ERROR, + errcode_for_file_access(), + errmsg("could not open /proc/self/smaps: %m")); + + while (fgets(line, sizeof(line), f) != NULL) + { + unsigned long start; + unsigned long end; + + if (sscanf(line, "%lx-%lx", &start, &end) == 2) + { + if (in_target_vma) + { + /* + * Continue accumulating only across VMAs that are contiguous + * with the previous one; stop as soon as we hit a gap or a + * different mapping. + */ + if (start != prev_end) + break; + } + else + in_target_vma = (target >= start && target < end); + + prev_end = end; + } + else if (in_target_vma) + { + if (use_hugetlb) + { + if (sscanf(line, "Shared_Hugetlb: %ld kB", &val) == 1) + total_shared_hugetlb_kb += val; + } + else + { + if (sscanf(line, "Rss: %ld kB", &val) == 1) + total_rss_kb += val; + else if (sscanf(line, "Swap: %ld kB", &val) == 1) + total_swap_kb += val; + } + } + } + + FreeFile(f); + + if (use_hugetlb) + result = mul_size(total_shared_hugetlb_kb, 1024); + else + { + result = mul_size(total_rss_kb, 1024); + result = add_size(result, mul_size(total_swap_kb, 1024)); + } + + PG_RETURN_INT64(result); +} + +/* + * Return the shared memory page size. + */ +PG_FUNCTION_INFO_V1(test_shmem_pagesize); +Datum +test_shmem_pagesize(PG_FUNCTION_ARGS) +{ + PG_RETURN_INT32(pg_get_shmem_pagesize()); +} + +/* + * Walk the entries between the current size and the reserved maximum, accessing + * each one. Ideally, this function should (seg)fault the moment we try to access + * the entry outside the currently allocated size, but the memory allocation and + * protection mechanisms work on page basis. Hence it may only (seg)fault when a + * page boundary is crossed. The mode argument selects between "read" and + * "write" access. + * + * When the current end of the structure and end of maximal structure are on the + * same page, this function may not (seg)fault at all. + */ +PG_FUNCTION_INFO_V1(resizable_shmem_access_beyond_size); +Datum +resizable_shmem_access_beyond_size(PG_FUNCTION_ARGS) +{ + text *mode_txt = PG_GETARG_TEXT_PP(0); + const char *mode = text_to_cstring(mode_txt); + bool do_write; + int32 sink = 0; + + if (!resizable_shmem) + ereport(ERROR, + errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("resizable_shmem is not initialized")); + + if (strcmp(mode, "read") == 0) + do_write = false; + else if (strcmp(mode, "write") == 0) + do_write = true; + else + ereport(ERROR, + errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("mode must be \"read\" or \"write\"")); + +#ifdef HAVE_RESIZABLE_SHMEM + + /* + * Ideally the structure should be protected through a synchronization + * cycle across all the backends that may access the structure. But we + * don't implement any such synchronization in this test module to keep it + * simple. Given that ProcSignalBarrier mechanism is not extensible, we + * may not be able to do that as well here. Hence add protect just before + * accessing the structure. + */ + ShmemProtectStruct("resizable_shmem"); +#endif + + for (int i = resizable_shmem->num_entries; i < test_max_entries; i++) + { + if (do_write) + resizable_shmem->data[i] = 0xdead; + else + sink = resizable_shmem->data[i]; + } + + /* + * Return the last read value so that compiler doesn't optimize away the + * assignment to sink. + */ + PG_RETURN_INT32(sink); +} + + +/* ---------------------------------------------------------------- + * Module initialization + * ---------------------------------------------------------------- + */ + +void +_PG_init(void) +{ + int guc_context; + + elog(LOG, "test_shmem module's _PG_init called"); + + RegisterShmemCallbacks(&TestShmemCallbacks); + + /* + * Use PGC_POSTMASTER when loaded at startup so the values are fixed once + * the shared memory segment is created. When loaded after startup + * PGC_POSTMASTER is not allowed, so we use PGC_SIGHUP instead. Although + * we do not intend to change these values at config reload, PGC_SIGHUP is + * the least permissive context that allows defining the GUC after startup + * and still prevents it from being changed via SET. + */ + if (process_shared_preload_libraries_in_progress) + guc_context = PGC_POSTMASTER; + else + { + guc_context = PGC_SIGHUP; + resizable_shmem_callbacks.flags = SHMEM_CALLBACKS_ALLOW_AFTER_STARTUP; + } + + DefineCustomIntVariable("resizable_shmem.initial_entries", + "Initial number of entries in the test structure.", + NULL, + &test_initial_entries, + TEST_INITIAL_ENTRIES_DEFAULT, + 1, + INT_MAX, + guc_context, + 0, + NULL, NULL, NULL); + + DefineCustomIntVariable("resizable_shmem.max_entries", + "Maximum number of entries in the test structure.", + NULL, + &test_max_entries, + TEST_MAX_ENTRIES_DEFAULT, + 1, + INT_MAX, + guc_context, + 0, + NULL, NULL, NULL); + + /* + * When loaded after startup by a backend that is not creating the + * extension, the shared memory might have been resized to a size other + * than the initial size. Use SHMEM_ATTACH_UNKNOWN_SIZE to attach without + * knowing the exact size. + */ + if (!process_shared_preload_libraries_in_progress && !creating_extension) + use_unknown_size = true; + + RegisterShmemCallbacks(&resizable_shmem_callbacks); +} diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index 6a3341356da..f2c47ac1c28 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1770,8 +1770,11 @@ pg_shadow| SELECT pg_authid.rolname AS usename, pg_shmem_allocations| SELECT name, off, size, - allocated_size - FROM pg_get_shmem_allocations() pg_get_shmem_allocations(name, off, size, allocated_size); + allocated_size, + minimum_size, + maximum_size, + reserved_space + FROM pg_get_shmem_allocations() pg_get_shmem_allocations(name, off, size, allocated_size, minimum_size, maximum_size, reserved_space); pg_shmem_allocations_numa| SELECT name, numa_node, size diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 56c1f997f88..65f0fa4d5cb 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -3186,6 +3186,7 @@ TestDSMRegistryHashEntry TestDSMRegistryStruct TestDecodingData TestDecodingTxnData +TestResizableShmemStruct TestShmemData TestSpec TestValueType -- 2.34.1