RE: Copy data to DSA area

From: "Ideriha, Takeshi" <ideriha(dot)takeshi(at)jp(dot)fujitsu(dot)com>
To: 'Kyotaro HORIGUCHI' <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>, "thomas(dot)munro(at)enterprisedb(dot)com" <thomas(dot)munro(at)enterprisedb(dot)com>, "robertmhaas(at)gmail(dot)com" <robertmhaas(at)gmail(dot)com>
Cc: "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: RE: Copy data to DSA area
Date: 2019-05-08 05:29:30
Message-ID: 4E72940DA2BF16479384A86D54D0988A7DB4FA55@G01JPEXMBKW04
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

>From: Ideriha, Takeshi [mailto:ideriha(dot)takeshi(at)jp(dot)fujitsu(dot)com]
>Sent: Friday, April 26, 2019 11:50 PM
>Well, after developing PoC, I realized that this PoC doesn't solve the local process is
>crashed before the context becomes shared because local process keeps track of
>pointer to chunks.
>Maybe all of you have already noticed and pointed out this case :) So it needs another
>work but this poc is a good step for me to advance more.

I think the point to prevent memory leak is allocating memory and storing its
address into a structure at the same time. This structure should be trackable from
other process.

So I'm going to change the design. I'll allocate a buffer of pointers to
dsa objects which are not permanent yet. This buffer is located on shared
memory. For simplicity, the buffer is allocated per process. That is the
number of backends equals to MaxBackends.

Developer calls API to make objects permanent. If objects become permanent,
corresponding pointers are deleted from the buffer. If they fail to become
permanent, they are freed from shared memory by checking the buffer and
corresponding pointers also deleted from the buffer.

There are two cases of failure: one is transaction abort, the other is process
crash. If transaction aborts, that process itself has responsibility to free
the objects. The process free the objects. In case of process crash, another
backend needs to take care of it. I'm assuming on_shmem_exit callback can be
used to free objects by postmaster.

Regarding MemoryContext, one Shared MemoryContext is on the shared memory
and each chunk has a back pointer to it to pfree and so on.

I'm going to make a new patch and send it later.

Regards,
Takeshi Ideriha

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Thomas Munro 2019-05-08 06:04:06 Re: Copy data to DSA area
Previous Message Thomas Munro 2019-05-08 04:50:02 Re: We're leaking predicate locks in HEAD