Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

From: "Nishant, Fnu" <nishantf(at)amazon(dot)com>
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, "Nasby, Jim" <nasbyj(at)amazon(dot)com>, "Alvaro Herrera" <alvherre(at)2ndquadrant(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>, Maxim Boguk <maxim(dot)boguk(at)gmail(dot)com>, "Peter Geoghegan" <pg(at)bowt(dot)ie>, Jeremy Finzel <finzelj(at)gmail(dot)com>
Subject: Re: found xmin from before relfrozenxid on pg_catalog.pg_authid
Date: 2018-05-29 18:06:12
Message-ID: 8CF9E2FB-B73B-4C66-8E56-529A162D47CC@amazon.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

Hi,

> To achieve this we can allocate Form_pg_class structures (for shared
> relations… a small number) on shared memory.

But why would this be necessary / a good idea? Even if we decided it
were, it seems like it'd end up being quite invasive. But I doubt it's
a good plan, because relcache entries want / need to be updated
differently in the transaction that does the changes (as it needs to see
the effect of catalog changes before commit) than other sessions (which
only should see them after commit).

It will be a good idea as, we can avoid maintaining file (creation/deletion/updation) for period of engine running and we do not need to invalidate other backend cache.
For transaction(which change catalog), we can allocate a new private Form_pg_class structure and do operations on it and update shared structure post commit.
Routine roughly will be-
1) A backend having a relcache entry for a shared rel where rd_rel part points to shared memory structure.
Rel->rd_rel is storing a shared memory pointer.
2) Wants to update the entry...allocate a new private structure and memcpy the shared content to this new memory and point rd_rel to private memory
Rel->rd_rel is storing a pointer to process specific structure.
3) Transaction committed and we copy the private memory content to shared memory area and point rd_rel again to shared memory.
4) free private memory.
5) Other backends do not do any invalidation but still get the latest updated values.

Why is this good idea?
Lets take an example (assuming we have 1000 postgres backends running).
With shared memory scheme-
Operation wise, for a transaction, we allocate/free once (private memory allocation) and memcpy data to and fro (from shared to private and back to shared)...
Overall memory footprint 1 shared copy and 1 private only when updating.
No file creation/deletion/updation.
With current file scheme-
Operation wise, for a transaction, we use private cache but we need to invalidate 1000 other caches( which will be atleast 1000 memcpy and allocate/free) and may involve reading back in page of pg_class.
Overall memory footprint 1000 private copies.
We have to create/delete/update init file and synchronize around it.

Having said that we may not worry about transaction for updating all values...(I think relfrozenxid can be updated by a CAS operation ...still thinking on it).

-Nishant

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Andres Freund 2018-05-29 18:13:15 Re: found xmin from before relfrozenxid on pg_catalog.pg_authid
Previous Message Laurenz Albe 2018-05-29 15:29:13 Re: Login with LDAP authentication takes 5 seconds

Browse pgsql-hackers by date

  From Date Subject
Next Message Andres Freund 2018-05-29 18:13:15 Re: found xmin from before relfrozenxid on pg_catalog.pg_authid
Previous Message Tom Lane 2018-05-29 18:00:20 Re: "column i.indnkeyatts does not exist" in pg_upgrade from 11dev to 11b1