Re: Converting postgresql.conf parameters to kilobytes

On Monday 31 May 2004 22:00, Tom Lane wrote:
> Shridhar Daithankar <shridhar(at)frodo(dot)hserus(dot)net> writes:
> > Right now following are measured in pages
> > wal_buffers
> > shared_buffers
> > effective_cachesize
> > while rest of the memory parameters are in kb. I thought being uniform
> > would be good. Besides it will make it independent of page size as well.
>
> It would probably be reasonable to change effective_cache_size, since we
> really do not know what the kernel's unit of buffering is (except on
> Linux, where we *do* know that it ain't 8K ;-)). Personally I'd opt for
> measuring it in MB not KB, though; that would be a much more convenient
> unit on modern machines. We could easily make it a float for anyone who
> thinks they know the cache size to sub-MB accuracy.

I have no problems with MB. Only thing I want to see is a bit more user
friendly and consistent configuration.

Initially I thought of bytes as oracle does but said 'let's be little more
practical' and put KB..:-) MB is perfectly OK. It would just change the
multiplier.

And I don't think specifying a float is such a good idea. It is just so
counter-intuitive. I mean how many people would care for fraction of an MB
wasted?

> As for the others, I'll side with Emerson: "a foolish consistency is the
> hobgoblin of little minds". We know very well what the unit of
> allocation of those is, and it's pages. There's no advantage to using
> KB except making it harder to work out what's really happening. We
> could measure max_connections in KB too if we had a mind to: there's
> a very definite shared-mem cost per connection slot. Or the FSM
> parameters, or checkpoint_segments, or max_locks_per_transaction.
> The fact that they have quantifiable space costs doesn't mean that space
> is the most useful way to measure them.

Agreed. This is not to change things left and right. It is only to put some
consistency in place.

> BTW, were you intending to convert KB to NBuffers by charging exactly 8K
> per buffer, or were you intending to allow for the additional shmem
> costs such as buffer headers, per-buffer LWLocks, etc? If not the
> latter, then what are you really measuring? For sure it's not shared
> memory size --- charging an artificial number isn't going to help anyone
> who's trying to pick shared_buffers to arrive at a particular actual
> shmem size. But if it is the latter then it'll become even more
> impossible to tell what's really happening, and we'll be forced to
> invent some way of reading out how many buffers really got allocated.

Well, for the purpose, this is beyond what I am trying to do. As of now there
is a int value sitting in postgresql.conf which is in page blocks. Now there
will be a conversion before it is used anytime so that it is bit more user
friendly. The change should be skin deep so as to be low impact.

As far putting a prefix such as K or M, I don't know much work that would be.
Does that mean we need to convert shared_buffers to a string parameter and
parse it? pgsql(at)mahawksoft(dot)com(I would gladly write a real name but alas)
said yesterday that there is a patch pending with Bruce for such a framework.
I don't know what and how it does.

For simplicity, I would convert all memory parameters to either KB or MB and
state so in postgresql.conf. No floats no suffixes. This is my opinion of
course. Any suggestions are always welcome..

Actually I need to find out few more things about it. It is not as simple as
adding a assign_hook. When I tried to initdb with changes, it demanded 64MB
of shared buffers which I (now) think that somewhere NBuffers are used before
postgresql.conf is parsed. So 8192*8000=64MB. But this is just guesswork.
Haven't looked in it there.

If this seems reasonably OK, then I would spend some more time on it. We would
need quite some documentation update then.

> So I disagree with the premise. Measuring these things in KB is not an
> improvement.

As I said, KBs or MBs is not the issue. Not having it in terms of pagesize is.

Shridhar