Re: POC: Cleaning up orphaned files using undo logs

On Wed, Jul 17, 2019 at 3:53 AM Andres Freund <andres(at)anarazel(dot)de> wrote:
> On 2019-07-15 12:26:21 -0400, Robert Haas wrote:

Responding again with some more details.

> >
> > But, my understanding of the current design being implemented is that
> > there is a hard limit on the number of transactions that can be
> > pending undo and the in-memory data structures are sized accordingly.
>
> My understanding is that that's really just an outcome of needing to
> maintain oldestXidHavingUndo accurately, right?
>

Yes.

> I know I asked this
> before, but I didn't feel like the answer was that clear (probably due
> to my own haziness). To me it seems very important to understand whether
> / how much we can separate the queuing/worker logic from the question of
> how to maintain oldestXidHavingUndo.
>

I am not sure if there is any tight coupling between queuing/worker
logic and computing oldestXid* value. The main thing to compute
oldestXid* value is that we need to know the xids of all the pending
abort transactions. We have already decided from the very beginning
that the hash table will have all the abort requests irrespective of
whether it is being processed by the foreground process or background
process. This will help us to avoid duplicate entries by backend and
background workers. Later, we decided that if we can have a hard
limit on how many pending undo requests can be present in a system,
then we can find the value of oldestXid* from the hash table.

I don't know how much it helps and you might already know all of this,
but I thought it is better to summarize to avoid any confusion.

--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com