RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed

Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> I think there are way too many moving parts here, and too few configuration details,
> to allow assigning blame confidently.
> I tried making a simple C test program that just connected, issued
> BEGIN/UPDATE/COMMIT, and disconnected in a tight loop. I see zero leakage with
> that. So either the leak isn't actually in libpq, or there's some critical environmental
> factor you didn't mention.
>
> Plausible such factors include connection type and authentication method. (For
> example, years ago libpq did leak memory while using GSSAPI encryption.) I tried
> both regular and SSL connections but didn't really push hard on that, since I'd just
> be guessing blindly about what your setup is.

Tom,

Thank you for taking the time to test this and for the feedback. Your C test showing no leak suggests the issue is specific to how psycopg2 uses libpq, not libpq itself. I apologize for not including enough environmental details. I used Kerberos/GSSAPI with SSL (TLS 1.2 connections). My connection string was: "postgresql://hostname/database" (no password, Kerberos auth).

Your mention of "years ago libpq did leak memory while using GSSAPI encryption" is interesting because we ARE using GSSAPI/Kerberos authentication.

Given your C test showed no leak, this appears to be specific to GSSAPI authentication, or perhaps specific to how psycopg2 uses libpq with GSSAPI.

I can test with non-GSSAPI authentication to try to isolate that variable. I can also create a pure psycopg2 reproducer (without SQLAlchemy). I can also test whether disabling GSSAPI encryption (but keeping GSSAPI auth) changes the behavior. Would testing with GSSAPI authentication help narrow this down? I can also report this to the psycopg2 project if you think it's their issue.

Thanks again for your help,
Matt