VACUUM memory management

From: Ibrar Ahmed <ibrar(dot)ahmad(at)gmail(dot)com>
To: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: VACUUM memory management
Date: 2019-12-09 18:30:20
Message-ID: CALtqXTdbBKRLdUuekg-2iYN06EX7BrodU1PheNgB6Bx6SuvCvg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

The memory consumption of VACUUM has some issues and could be improved.
Some of its limitations are recorded in the comments of the “vacuumlazy.c”
file. The current design of VACUUM memory usage is that it stores the TID
in a fixed-size array which is allocated at the start, based upon
maintenance_work_mem. There are three problems with that design

- If the value of maintenance_work_mem is too large then it is a waste of
memory for small tables.
- If the value of maintenance_work_mem is too small or “TIDs” do not fit
in the array then multiple scans happen.
- In cases where maintainess_work_mem is set too large, and we have a
bigger value of vacuume_count, then the system can be out-of-memory.

There are two solutions for these problems. The first is to use a list
instead of a fixed size array. The second solution is to allocate the
memory in chunks.
The attached WIP patch creates an array of ItemPointers and allocates
memory in chunks by dividing the maintenance_work_mem into multiple chunks.

Comments?
--
Ibrar Ahmed

Attachment Content-Type Size
vacuum_dyn_mem_ibrar_v1.patch application/x-patch 11.0 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2019-12-09 18:31:47 Re: Windows buildfarm members vs. new async-notify isolation test
Previous Message Karl O. Pinc 2019-12-09 18:25:23 Re: proposal: minscale, rtrim, btrim functions for numeric