Re: A thought on Index Organized Tables

Gokulakannan Somasundaram wrote:
>> If you have a scenario where the visibility map incurs a measurable
>> overhead, let's hear it. I didn't see any in the tests I performed, but
>> it's certainly possible that if the circumstances are just right it
>> makes a difference.
>>
>> Heikki,
> The obvious one, i could observe is that it would increase the WAL
> contention. Am i missing something?

Yes. The visibility map doesn't need any new WAL records to be written.

We probably will need to add some WAL logging to close the holes with
crash recovery, required for relying on it for index-only-scans, but
AFAICS only for VACUUM and probably only one WAL record for a whole
bunch of heap pages, so it should be pretty insignificant.

> All i am suggesting is to reduce the
> unnecessary work required in those tables, where the visibility map is not
> required. For example, in data warehouses, people might even have a tables
> without any indexes. Why do we ask them to incur the overhead of visibility
> map?

To make it possible to do partial VACUUMs. That's why the visibility map
was put into 8.4.

Let me repeat myself: if you think the overhead of a visibility map is
noticeable or meaningful in any scenario, the onus is on you to show
what that scenario is. I am not aware of such a scenario, which doesn't
mean that it doesn't exist, of course, but hand-waving is not helpful.

> Also since you have made the visibility maps without any page
> level locking, have you considered whether it would make sure the correct
> order of inserts into the WAL? i have looked at some random threads, but i
> couldn't get the complete design of visibility map to be used for index only
> scans.

I'm not sure what you mean with "without any page level locking".
Whenever a visibility map page is read or modified, a lock is taken on
the buffer.

I believe the current visibility map is free of race conditions, even if
it was used for index-only-scans, if that's what you mean. The critical
part is when a bit is cleared in the visibility map. It is done just
after inserting/deleting the heap tuple, which is OK because in the
window between modifying the heap page and clearing bit in the
visibility map, no other backend could see the actions of the modifying
transaction yet anyway. The index updates have not been made yet, so the
information in the indexes are still valid for the other transaction's
snapshot.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com