Re: old synchronized scan patch

Jeff,

On 12/4/06 3:38 PM, "Jeff Davis" <pgsql(at)j-davis(dot)com> wrote:

> I'll try to run a test like that this week. I will be doing this on my
> home hardware (bad, consumer-grade stuff), so if I gave you a patch
> against HEAD could you test it against some more real hardware (and
> data)?

Sure.

> To open up the implementation topic:
>
> My current patch starts a new sequential scan on a given relation at the
> page of an already-running scan. It makes no guarantees that the scans
> stay together, but in practice I don't think they deviate much. To try
> to enforce synchronization of scanning I fear would do more harm than
> good. Thoughts?

I think this is good enough - a "background scanner" approach would be the
logical alternative, but may not be necessary. I suspect you are correct
about the scans being nearly synced.

This may not be the case if and when we implement a priority based
scheduler, but in that case we're already managing the throughput based on
content anyway.

> Also, it's more of a "hint" system that uses a direct mapping of the
> relations Oid to hold the position of the scan. That means that, in rare
> cases, the page offset could be wrong, in which case it will degenerate
> to the current performance characteristics with no cost. The benefit of
> doing it this way is that it's simple code, with essentially no
> performance penalty or additional locking. Also, I can use a fixed
> amount of shared memory (1 page is about right).

If I understand correctly, Tom's concern is that this page is potentially
accessed once for every page read and may consequently become very hot. How
do you manage the scan position so this doesn't happen? Do we have any
readahead in the I/O layer? There is certainly readahead in the OS I/O
cache, but it's dynamic and we don't know the block position...

- Luke