Re: Providing catalog view to pg_hba.conf file - Patch submission

On Mon, Mar 2, 2015 at 7:42 PM, Stephen Frost <sfrost(at)snowman(dot)net> wrote:
> Uh, maybe because it's trying to allocate over 1GB and palloc() doesn't
> support that?

Nobody's allocating anything that big. It's a list of 25,000 pointers
to 472-byte structs. That should add up to about 11MB. Instead the
memory context is a total of 954606152 bytes which is still under a
gigabyte and the database does start up. It drives my laptop to a
crawl and Autovacuum crashes when it tries to start but the postmaster
is mostly happy. That's about 38k per line or about 80x as much as it
should be taking.

The tokenizer is run in a separate memory context which gets reset
when this is done. Then during parsing it copies things from that
memory context over as needed to fill in the struct. In my example
pg_hba.c it shouldn't have to copy anything over though so even if
there's a bug there I don't see how it would cause this.

My best guess is that some functions called allocate some temporary
storage. I haven't spotted them yet though I put this aside when I
first sent my message. It seems like it would be safest to use a
temporary memory context throughout the whole process and only copy
things to the hba memory context manually.

--
greg