From: | Zhu Yang <fairyfar(at)msn(dot)com> |
---|---|
To: | "pgsql-bugs(at)lists(dot)postgresql(dot)org" <pgsql-bugs(at)lists(dot)postgresql(dot)org> |
Subject: | The memory usage is positively correlated with the number of partition tables when pg_get_expr is called. |
Date: | 2025-08-20 06:16:53 |
Message-ID: | MEWP300MB002272E8DEAD2FCFA14A3518D533A@MEWP300MB0022.AUSP300.PROD.OUTLOOK.COM |
Views: | Whole Thread | Raw Message | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-bugs |
Unreasonable memory usage.
# SUMMARY INFO
OS: Red Hat Enterprise Linux 7.6
PostgreSQL version: 19devel
[yz(at)bogon postgresql]$ psql postgres
psql (19devel)
Type "help" for help.
postgres=# select version();
version
-----------------------------------------------------------------------------------------------------------
PostgreSQL 19devel on x86_64-pc-linux-gnu, compiled by gcc (GCC) 9.3.1 20200408 (Red Hat 9.3.1-2), 64-bit
(1 row)
# TEST CASE
1. Modify postgresql.conf, and set GUCs:
```
max_locks_per_transaction = 2048
shared_buffers = '10GB'
```
2. Restart PostgreSQL
Make the parameter modification take effect.
3. Create database
```
CREATE DATABASE mem;
```
4. Create 500 partition tables (it is recommended to create them in segments), with each partitioned table having 200 sub-partitions.
```
[yz(at)bogon postgresql]$ psql mem
psql (19devel)
Type "help" for help.
mem=#
DO $$
DECLARE
part_start bigint := 1;
part_end bigint := 500;
part_curr bigint := part_start;
sub_start bigint := 1;
sub_end bigint := 200;
sub_curr bigint := sub_start;
BEGIN
WHILE part_curr < part_end LOOP
EXECUTE format('
CREATE TABLE sales%s(
id int,
sale_price bigint,
product_name text
)partition by range (sale_price)', part_curr
);
sub_curr := sub_start;
WHILE sub_curr < sub_end LOOP
EXECUTE format('
CREATE TABLE sales%s_%s PARTITION OF sales%s
FOR VALUES FROM (%s) TO (%s)',
part_curr,
sub_curr,
part_curr,
sub_curr,
sub_curr + 1
);
sub_curr := sub_curr + 1;
END LOOP;
part_curr := part_curr + 1;
END LOOP;
END $$;
```
After successful creation, exit the psql connection.
5. Call pg_get_expr
```
[yz(at)bogon postgresql]$ psql mem
psql (19devel)
Type "help" for help.
mem=# SELECT pg_get_expr(relpartbound, oid) FROM pg_class WHERE relname like 'sales%';
```
Wait for the query to complete.
6. Check the query memory usage in Step 5
Find the PID corresponding to the query:
```
[yz(at)bogon ~]$ ps ux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
yz 1739 0.0 1.8 5619732 144672 ? Ss 07:22 0:01 /home/yz/postgresql/pg_bin/bin/postgres -D pg_data
yz 1740 0.0 0.0 5619868 3704 ? Ss 07:22 0:00 postgres: io worker 1
yz 1741 0.0 0.0 5619868 6580 ? Ss 07:22 0:00 postgres: io worker 0
yz 1742 0.0 0.0 5619868 2924 ? Ss 07:22 0:00 postgres: io worker 2
yz 1743 0.8 12.1 5661680 971236 ? Ds 07:22 3:42 postgres: checkpointer
yz 1744 0.0 1.0 5620820 87232 ? Ss 07:22 0:11 postgres: background writer
yz 1746 0.1 0.2 5619868 18212 ? Ss 07:22 0:29 postgres: walwriter
yz 1747 0.0 0.0 5658196 3408 ? Ss 07:22 0:01 postgres: autovacuum launcher
yz 1748 0.0 0.0 5621196 3436 ? Ss 07:22 0:00 postgres: logical replication launcher
yz 30192 1.4 0.0 124644 3636 pts/0 S+ 14:31 0:00 psql mem
yz 30193 21.3 10.5 5942276 841712 ? Ss 14:31 0:10 postgres: yz mem [local] idle
```
Print the memory count of CacheMemoryContext using gdb:
```
[yz(at)bogon ~]$ gdb -p 30193
...
(gdb) p *CacheMemoryContext
$1 = {type = T_AllocSetContext, isReset = false, allowInCritSection = false, mem_allocated = 286261312, methods = 0xecbff0 <mcxt_methods+240>, parent = 0x2b2af50,
firstchild = 0x2b5a3d0, prevchild = 0x2b8f2c0, nextchild = 0x2b8b2a0, name = 0xea853f "CacheMemoryContext", ident = 0x0, reset_cbs = 0x0}
```
It can be seen that the value of mem_allocated in CacheMemoryContext is approximately 280MB.
When the number of partition tables or sub-partitions increases, the memory usage also increases.
Yours Sincerely,
Yang Zhu
From | Date | Subject | |
---|---|---|---|
Next Message | Richard Guo | 2025-08-20 09:37:14 | Re: BUG #19007: Planner fails to choose partial index with spurious 'not null' |
Previous Message | Zane Duffield | 2025-08-20 05:38:34 | Re: Lock timeouts and unusual spikes in replication lag with logical parallel transaction streaming |