From: | Robert Berry <berrydigital(at)gmail(dot)com> |
---|---|
To: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, michael(dot)paquier(at)gmail(dot)com |
Cc: | pgsql-hackers(at)postgresql(dot)org |
Subject: | Re: calculating an aspect of shared buffer state from a background worker |
Date: | 2014-03-10 20:52:04 |
Message-ID: | CAM=RetHbzKpTa9ad0N6Pq42xnrb_FxP2oF7trCgx1nCO01dmXw@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Thank you both for the thoughtful and helpful responses.
The utility of the length of the free list is somewhat dubious. I imagine
it could be useful to answer the question of "is there a chance that
increasing shared buffers would be useless?" in an optimization context.
Agreed it's not useful in most steady state scenarios.
I saw the approach in the pg_buffercache contrib module and am looking for
lockless alternatives for at least estimating the size of free buffers.
I'm a relatively inexperienced so I'd be curious to know whether there is a
danger beyond an inconsistent result in traversing / sampling the
BufferDescriptors without a lock?
Also I got the impression that there is a ring approach to freeing buffers,
and that assuming the descriptors are allocated in sequential addresses,
taking the difference in the first and last could be used to get a rough
estimate accounting for sizes or other shenanigans?
Thank you again, the clues to look at buffer descriptors and ShmemInitStruct
are very helpful.
Best Regards,
Robert
On Mon, Mar 10, 2014 at 7:33 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Robert Berry <berrydigital(at)gmail(dot)com> writes:
> > I'm looking at doing a calculation to determine the number of free
> buffers
> > available. A n example ratio that is based on some data structures in
> > freelist.c as follows:
>
> > (StrategyControl->lastFreeBuffer - StrategyControl->firstFreeBuffer) /
> > (double) NBuffers
>
> > Is there a way to get access to the StrategyControl pointer in the
> context
> > of a background worker?
>
> The BufferStrategyControl struct is in shared memory, so you can certainly
> get at it. One way would be to modify freelist.c to export its static
> pointer variable. Alternatively, you could call ShmemInitStruct an extra
> time to look up the struct for yourself, and then save it in your own
> static variable.
>
> Having said that, though, I'm pretty dubious of the premise. I trust you
> realize that the above calculation is entirely wrong; firstFreeBuffer and
> lastFreeBuffer are list head and tail pointers, and have no numerical
> relation to the list length. The only way to determine the list length
> accurately would be to chase down the whole list, which you'd have to hold
> the BufFreelistLock while doing, which'd be disastrous for performance if
> the list was long. (If you're okay with modifying the backend code you
> could dodge this by teaching freelist.c to maintain a counter, I guess.)
>
> An even bigger issue is that it's not clear that the length of the free
> list is actually a useful number to have; in steady-state usage it
> frequently is always zero. Buffers only get put back on the freelist if
> they're invalidated, eg by dropping the relation they belonged to. Normal
> usage tends to allocate buffers by reclaiming ones whose usage_count has
> reached zero in the clock sweep algorithm. So a better picture of the
> availability of buffers would require scanning the buffer pool to see how
> many there are of each usage_count level.
>
> regards, tom lane
>
From | Date | Subject | |
---|---|---|---|
Next Message | Josh Berkus | 2014-03-10 20:55:53 | Re: Changeset Extraction v7.9.1 |
Previous Message | Alvaro Herrera | 2014-03-10 20:18:03 | Re: on_exit_reset fails to clear DSM-related exit actions |