Skip site navigation (1) Skip section navigation (2)

Re: Dead Space Map version 3 (simplified)

From: Heikki Linnakangas <heikki(at)enterprisedb(dot)com>
To: ITAGAKI Takahiro <itagaki(dot)takahiro(at)oss(dot)ntt(dot)co(dot)jp>
Cc: pgsql-patches(at)postgresql(dot)org
Subject: Re: Dead Space Map version 3 (simplified)
Date: 2007-04-20 15:21:09
Message-ID: 4628DA65.3080208@enterprisedb.com (view raw or flat)
Thread:
Lists: pgsql-hackerspgsql-patches
ITAGAKI Takahiro wrote:
> Attached is an updated DSM patch. I've left the core function of DSM only
> and dropped other complicated features in this release.

We discussed it a long time ago already, but I really wished the DSM 
wouldn't need a fixed size shared memory area. It's one more thing the 
DBA needs to tune manually. It also means we need to have an algorithm 
for deciding what to keep in the DSM and what to leave out. And I don't 
see a good way to extend the current approach to implement the 
index-only-scans that we've been talking about, and the same goes for 
recovery. :(

The way you update the DSM is quite interesting. When a page is dirtied, 
the BM_DSM_DIRTY flag is set in the buffer descriptor. The corresponding 
bit in the DSM is set lazily in FlushBuffer whenever BM_DSM_DIRTY is 
set. That's a clever way to avoid contention on updates. But does it 
work for tables that have a small hot part that's updated very 
frequently? That's exactly the scenario where the DSM is the most 
useful. Hot pages stay in the buffer cache because they're frequently 
accessed, which means that FlushBuffer isn't getting called for them and 
the bits in the DSM aren't getting set until checkpoint. This could lead 
to unnecessary bloating of the hot part. A straightforward fix would be 
to scan the buffer cache for buffers marked with BM_DSM_DIRTY to update 
the DSM before starting the vacuum scan.

It might not be a problem in practice, but it bothers me that the DSM 
isn't 100% accurate. You end up having a page with dead tuples on it 
marked as non-dirty in the DSM at least when a page is vacuumed but 
there's some RECENTLY_DEAD tuples on it that become dead later on. There 
might be other scenarios as well.

If I'm reading the code correctly, DSM makes no attempt to keep the 
chunks ordered by block number. If that's the case, vacuum needs to be 
modified because it currently relies on the fact that blocks are scanned 
and the dead tuple list is therefore populated in order.

-- 
   Heikki Linnakangas
   EnterpriseDB   http://www.enterprisedb.com

In response to

Responses

pgsql-hackers by date

Next:From: Magnus HaganderDate: 2007-04-20 16:46:09
Subject: Re: BUG #3242: FATAL: could not unlock semaphore: error code 298
Previous:From: Tom LaneDate: 2007-04-20 14:58:35
Subject: Re: BUG #3242: FATAL: could not unlock semaphore: error code 298

pgsql-patches by date

Next:From: Gregory StarkDate: 2007-04-20 23:03:34
Subject: Test case for slow planning time
Previous:From: Andrew DunstanDate: 2007-04-20 13:47:12
Subject: Re: [HACKERS] parser dilemma

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group