Re: Large writable variables

Hi,

On 2018-10-16 15:13:43 -0400, Robert Haas wrote:
> On Mon, Oct 15, 2018 at 4:08 PM Andres Freund <andres(at)anarazel(dot)de> wrote:
> > So we have 500kb of not-initialized memory mapped into every
> > process. That's, uh, not nothing.
>
> Thinking about this a bit more, why is this bad? I mean, if the
> memory is never touched, the OS does not really need to allocate or
> zero any pages, or even make any page table entries. If somebody
> actually accesses the data, then we'll take a page fault and have to
> really allocate, but otherwise I would think we could have 50MB of
> unused bss floating around and it wouldn't really matter, let alone
> 500kB.
>
> What am I missing?

For one the OS will actually reserve all that memory when you use memory
overcommit = 2 (which you should). There's no need to reserve memory for
data that's guaranteed to be shared. It also reduces the frequency of
page faults when looking at other global variables laid out nearby - we
could do something about that by giving the linker placement
instructions, but that's also work. The size of the pagetable also has
performance effects, although it'll often be dwarfed by the shared
buffers entry if not using huge pages. WRT .data entries, marking them
as const where appropriate both reduces memory usage, possibly allows
more compiler optimizations, and allows to catch bugs.

There's also the fact that not every OS has COW pagetables, or we use
processes in a way that don't allow that (say windows, where we don't
fork).

Greetings,

Andres Freund