DSA failed to allocate memory

From: Dongming Liu <ldming101(at)gmail(dot)com>
To: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: DSA failed to allocate memory
Date: 2022-01-24 09:58:44
Message-ID: CAL1p7e8LzB2LSeAXo2pXCW4+Rya9s0sJ3G_ReKOU=AjSUWjHWQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

I am using Dynamic shared memory areas(DSA) to manage some variable
length shared memory, I've found that in some cases allocation fails even
though there are enough contiguous pages.

The steps to reproduce are as follows:
1. create a dsa area with a 1MB DSM segment
2. set its size limit to 1MB
3. allocate 4KB memory until fails
4. free all allocated memory in step 3
5. repeat step 3 and step 4

When I first run the step 3, there is 240 4KB memory allocated successfully.
But when I free all and allocate again, no memory can be allocated even
though there are 252 contiguous pages. IMO, this should not be expected
to happen, right?

The call stack is as follows:
#0 get_best_segment (area=0x200cc70, npages=16) at dsa.c:1972
#1 0x0000000000b46b36 in ensure_active_superblock (area=0x200cc70,
pool=0x7fa7b51f46f0,
size_class=33) at dsa.c:1666
#2 0x0000000000b46555 in alloc_object (area=0x200cc70, size_class=33)
at dsa.c:1460
#3 0x0000000000b44f05 in dsa_allocate_extended (area=0x200cc70,
size=4096, flags=2) at dsa.c:795

I read the relevant code and found that get_best_segment re-bin the segment
to segment index 4 when first run the step 3. But when free all and run the
step 3 again, get_best_segment search from the first bin that *might*
have enough
contiguous pages, it is calculated by contiguous_pages_to_segment_bin(),
for a superblock with 16 pages, contiguous_pages_to_segment_bin is 5.
So the second time, get_best_segment search bin from segment index 5 to 16,
but the suitable segment has been re-bin to 4 that we do not check.
Finally, the get_best_segment return NULL and dsa_allocate_extended return
a invalid dsa pointer.

Maybe we can use one of the following methods to fix it:
1. re-bin segment to suitable segment index when called dsa_free
2. get_best_segment search all bins

I wrote a simple test code that is attached to reproduce it.

Ant thoughts?

--
Best Regards
Dongming(https://www.aliyun.com/)

Attachment Content-Type Size
0001-add-dsa-test-module.patch application/octet-stream 5.6 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Pavel Borisov 2022-01-24 10:23:33 Re: Make mesage at end-of-recovery less scary.
Previous Message Masahiko Sawada 2022-01-24 09:54:20 Re: Skipping logical replication transactions on subscriber side