Re: Synchronized Scan WIP patch

On Thu, 2007-05-31 at 09:08 +0100, Heikki Linnakangas wrote:
> Here's a work-in-progress update of this patch.
>
> I haven't done any major changes, but a lot of little refactoring and
> commenting, including:
>
> * moved the sync scan stuff to a new file access/heapam/syncscan.c.
> heapam.c is long enough already, and in theory the same mechanism could
> be used for large bitmap heap scans in the future.

Good idea, I hadn't thought of that. It seems like the bitmaps in two
bitmap scans would have to match very closely, but that sounds
plausible.

This is similar to another idea I had considered (I forget who thought
of it) to try to have a bitmap of "tuples still needed" and then try to
optimize based on that information somehow (read the ones in cache
first, etc). Seems substantially more complex though, more like a
prefetch system at that point.

I expected the general refactoring. Hopefully my next patch is a little
closer to the code expectations and places less burden on the reviewers.

> Testing:
> * Multiple scans on different tables, causing movement in the LRU list
> * Measure the CPU overhead for a single scan
> * Measure the lock contention with multiple scanners
>

Is there any way to measure the necessity of the hash table? I would
think the conditions for that would be a large number of tables being
actively scanned causing a lot of LRU activity such that the locks are
held too long.

I also think the optimization of only reporting when the block is not
found in cache would be useful to test if the lock contention is a problem.

Regards,
Jeff Davis