Re: How to make ResourceOwnerForgetBuffer() O(1), instead of O(N^2) scale

> Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com> writes:
> > Idea-1) Put ResourceOwnerForgetBuffer() O(1) logic, instead of O(N^2).
> > The source of problem come from data structure in ResourceOwnerData,
> > so a straightforward way is to apply O(1) logic based on hashing,
> > instead of the linear search.
>
> I will bet that this is a dead loss for all normal usage patterns, because
> queries seldom have more than a few buffers pinned. More than that: I do
> not think we should encourage coding patterns that pin lots of buffers.
> There is no way that holding pins on thousands of buffers to do one operation
> is a sane design.
>
Yep, that's my pain. Even though usual query does not take many buffers pinned,
my use case needs to fetch megabytes scale data at once because of performance
reason; page-by-page synchronous scan makes GPU being idle.
It is also a reason why I cared about whether the O(1) logic is harmless to
existing (major portion of) workloads.

BTW, what is the reason why we assume small amount of pages are pined at same
time? Does it come from hardware assumption that has much less RAM capacity
than the data-set to be processed?
Probably, once we pay attention a hardware that has more than several dozen GB
RAM which can keep the data-set on memory, it may lead different results.

At least, it seems to me valuable to support both of the styles of buffer
usages, as long as it is harmless to the existing workloads.
I'd like to dig down the deeper reason of this.

Thanks,
--
NEC OSS Promotion Center / PG-Strom Project
KaiGai Kohei <kaigai(at)ak(dot)jp(dot)nec(dot)com>