Re: Initializing LWLock Array from Server Code

Hi Robert,

Thank you for your reply and sorry that I couldn't reply earlier.
Since I didn't get any response within a couple of days, I took the longer
route -- changed the lwlock.h and lwlock.c
for accommodating the lw locks for the shared hash table.

I'll describe what I modified in the lwlock.h and lwlock.c and it seems to
be working fine. Although I haven't got an opportunity
to test it extensively. If you could let me know if I missed out anything
that might cause problems later that would be great.

In lwlock.h, the following modifications were made:

*/* the number of partitions of the shared hash table */*
#define NUM_FILTERHASH_PARTITIONS 64

*/* Offsets for various chunks of preallocated lwlocks. */*
#define BUFFER_MAPPING_LWLOCK_OFFSET NUM_INDIVIDUAL_LWLOCKS
#define LOCK_MANAGER_LWLOCK_OFFSET \
(BUFFER_MAPPING_LWLOCK_OFFSET + NUM_BUFFER_PARTITIONS)
#define PREDICATELOCK_MANAGER_LWLOCK_OFFSET \
(LOCK_MANAGER_LWLOCK_OFFSET + NUM_LOCK_PARTITIONS)

*/* offset for shared filterhash lwlocks in the MainLWLockArray */*
#define FILTERHASH_OFFSET \
(PREDICATELOCK_MANAGER_LWLOCK_OFFSET + NUM_PREDICATELOCK_PARTITIONS)
*/* modified NUM_FIXED_LOCKS */*
#define NUM_FIXED_LWLOCKS \
(FILTERHASH_OFFSET + NUM_FILTERHASH_PARTITIONS)

*/* added LWTRANCHE_FILTERHASH in the BuiltinTrancheIds */*
typedef enum BuiltinTrancheIds
{
LWTRANCHE_CLOG_BUFFERS = NUM_INDIVIDUAL_LWLOCKS,
LWTRANCHE_COMMITTS_BUFFERS,
LWTRANCHE_SUBTRANS_BUFFERS,
LWTRANCHE_MXACTOFFSET_BUFFERS,
LWTRANCHE_MXACTMEMBER_BUFFERS,
LWTRANCHE_ASYNC_BUFFERS,
LWTRANCHE_OLDSERXID_BUFFERS,
LWTRANCHE_WAL_INSERT,
LWTRANCHE_BUFFER_CONTENT,
LWTRANCHE_BUFFER_IO_IN_PROGRESS,
LWTRANCHE_REPLICATION_ORIGIN,
LWTRANCHE_REPLICATION_SLOT_IO_IN_PROGRESS,
LWTRANCHE_PROC,
LWTRANCHE_BUFFER_MAPPING,
LWTRANCHE_LOCK_MANAGER,
LWTRANCHE_PREDICATE_LOCK_MANAGER,
*LWTRANCHE_FILTERHASH*,
LWTRANCHE_PARALLEL_HASH_JOIN,
LWTRANCHE_PARALLEL_QUERY_DSA,
LWTRANCHE_SESSION_DSA,
LWTRANCHE_SESSION_RECORD_TABLE,
LWTRANCHE_SESSION_TYPMOD_TABLE,
LWTRANCHE_SHARED_TUPLESTORE,
LWTRANCHE_TBM,
LWTRANCHE_PARALLEL_APPEND,
LWTRANCHE_FIRST_USER_DEFINED
} BuiltinTrancheIds;

In lwlock.c, the following modifications were made:

In function, static void InitializeLWLocks(void), the following lines were
added:

*/* Initialize filterhash LWLocks in main array */*
lock = MainLWLockArray + NUM_INDIVIDUAL_LWLOCKS +
NUM_BUFFER_PARTITIONS + NUM_LOCK_PARTITIONS +
NUM_FILTERHASH_PARTITIONS;
for (id = 0; id < NUM_FILTERHASH_PARTITIONS; id++, lock++)
LWLockInitialize(&lock->lock, LWTRANCHE_FILTERHASH);

In function void RegisterLWLockTranches(void), the following line was added:

LWLockRegisterTranche(LWTRANCHE_FILTERHASH, "filter_hash");

All of this was done after allocating the required of space in the shared
memory.

Thanks,
-SB

On Mon, Apr 29, 2019 at 1:59 PM Robert Haas <robertmhaas(at)gmail(dot)com> wrote:

> On Fri, Apr 26, 2019 at 2:58 PM Souvik Bhattacherjee <kivuosb(at)gmail(dot)com>
> wrote:
> > I have created a shared hash table in partitioned mode inside the
> postgres server code. In order to guard the partitions, I'm trying to
> initialize an array of LWLocks. The code that I'm trying to use for that is
> >
> > void RequestNamedLWLockTranche(const char *tranche_name, int
> num_lwlocks);
> > LWLockPadded *GetNamedLWLockTranche(const char *tranche_name);
> >
> > I'm not sure where exactly should this code be called from the server
> code. So I had placed it in
> >
> > void CreateSharedMemoryAndSemaphores(bool makePrivate, int port);
> >
> > within ipic.c. However, I'm getting the following error message when
> starting the database:
> >
> > FATAL: requested tranche is not registered
> >
> > So at this point, I'm a little confused as to where the methods should
> be called from inside the server code. Any pointers would be appreciated.
>
> RequestNamedLWLockTranche() changes the behavior of LWLockShmemSize()
> and CreateLWLocks(), so must be called before either of those.
> GetNamedLWLockTranche() must be called after CreateLWLocks().
>
> This machinery works for pg_stat_statements, so see that as an example
> of how to make this work from an extension. If you're hacking the
> core server code, then look at the places where the corresponding bits
> of pg_stat_statements code get called. IIRC, _PG_init() gets called
> from process_shared_preload_libraries(), so you might look at the
> placement of the calls to that function.
>
> --
> Robert Haas
> EnterpriseDB: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>