Re: Parallel bitmap heap scan

On Sun, Feb 19, 2017 at 7:18 PM, Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
> I have observed one problem with 0002 and I though of sharing that
> before fixing the same because we might have encountered the same
> problem in some other patches i.e parallel shared hash and there might
> be already a way to handle that.
>
> The problem is that In ExecEndBitmapHeapScan we do tbm_free. Wherein,
> it frees local pagetable memory and the shared memory using callback
> routine and if other than the Backend (actual backend for the client
> which is executing gather node) node any other worker become the
> leader (worker which is responsible for creating the shared TBM) then
> it will finish it work and free the shared memory by calling
> ExecEndBitmapHeapScan, and it's possible that other worker are still
> operating on the shared memory.
>
> So far I never observed this issue in our test may be because either
> Backend become the leader or by the time leader (non backend) free the
> TBM others also finishes there work.
>
> Solution:
> - One solution to this problem can be that leader should not complete
> the scan unless all other worker have finished the work.
> - We can also think of that we don't make anyone wait but we make a
> arrangement that last worker to finish the bitmapscan only free the
> memory, but one problem with this solution is that last worker can be
> non-leader also, which will have access to shared memory but how to
> free the pagetable local memory (it's local to the leader).
>
> Any suggestion on this ?

It's probably OK if tbm_free() doesn't free the memory allocated from
DSA, and we just let cleanup at end of query do it. However, that
could cause some trouble if the Parallel Bitmap Heap Scan gets
executed over and over and keeps allocating more and more memory from
DSA. I think the way to fix that would be to maintain a reference
count that starts at 1 when the iterator arrays are created and gets
incremented every time a TBMSharedIteratorState is created. It gets
decremented when the TIDBitmap is destroyed that has iterator arrays
is destroyed, and each time a TBMSharedIteratorState is destroyed.
When it reaches 0, the process that reduces the reference count to 0
calls dsa_free on the DSA pointers for pagetable, spages, and schunks.
(Also, if a TIDBitmap is freed before iteration begins, it frees the
DSA pointer for the pagetable only; the others won't have values yet.)

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company