Support to define custom wait events for extensions

From: Masahiro Ikeda <ikedamsh(at)oss(dot)nttdata(dot)com>
To: pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Support to define custom wait events for extensions
Date: 2023-06-15 06:06:01
Message-ID: b9f5411acda0cf15c8fbb767702ff43e@oss.nttdata.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

Currently, only one PG_WAIT_EXTENSION event can be used as a
wait event for extensions. Therefore, in environments with multiple
extensions are installed, it could take time to identify which
extension is the bottleneck.

So, I'd like to support new APIs to define custom wait events
for extensions. It's discussed in [1].

I made patches to realize it. Although I have some TODOs,
I want to know your feedbacks. Please feel free to comment.

# Implementation of new APIs

I implemented 2 new APIs for extensions.
* RequestNamedExtensionWaitEventTranche()
* GetNamedExtensionWaitEventTranche()

Extensions can request custom wait events by calling
RequestNamedExtensionWaitEventTranche(). After that, use
GetNamedExtensionWaitEventTranche() to get the wait event information.

The APIs usage example by extensions are following.

```
shmem_request_hook = shmem_request;
shmem_startup_hook = shmem_startup;

static void
shmem_request(void)
{
/* request a custom wait event */
RequestNamedExtensionWaitEventTranche("custom_wait_event");
}

static void
shmem_startup(void)
{
/* get the wait event information */
custom_wait_event =
GetNamedExtensionWaitEventTranche("custom_wait_event");
}

void
extension_funtion()
{
(void) WaitLatch(MyLatch,
WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH,
10L * 1000,
custom_wait_event); /* notify core with custom wait event */
ResetLatch(MyLatch);
}
```

# Implementation overview

I referenced the implementation of
RequestNamedLWLockTranche()/GetNamedLWLockTranche().
(0001-Support-to-define-custom-wait-events-for-extensions.patch)

Extensions calls RequestNamedExtensionWaitEventTranche() in
shmem_request_hook() to request wait events to be used by each
extension.

In the core, the requested wait events are dynamically registered in
shared
memory. The extension then obtains the wait event information with
GetNamedExtensionWaitEventTranche() and uses the value to notify the
core
that it is waiting.

When a string representing of the wait event is requested,
it returns the name defined by calling
RequestNamedExtensionWaitEventTranche().

# PoC extension

I created the PoC extension and you can use it, as shown here:
(0002-Add-a-extension-to-test-custom-wait-event.patch)

1. start PostgreSQL with the following configuration
shared_preload_libraries = 'inject_wait_event'

2. check wait events periodically
psql-1=# SELECT query, wait_event_type, wait_event FROM pg_stat_activity
WHERE backend_type = 'client backend' AND pid != pg_backend_pid() ;
psql-1=# \watch

3. execute a function to inject a wait event
psql-2=# CREATE EXTENSION inject_wait_event;
psql-2=# SELECT inject_wait_event();

4. check the custom wait event
You can see the following results of psql-1.

(..snip..)

query | wait_event_type | wait_event
-----------------------------+-----------------+------------
SELECT inject_wait_event(); | Extension | Extension
(1 row)

(..snip..)

(...about 10 seconds later ..)

query | wait_event_type | wait_event
-----------------------------+-----------------+-------------------
SELECT inject_wait_event(); | Extension | custom_wait_event
# requested wait event by the extension!
(1 row)

(..snip..)

# TODOs

* tests on windows (since I tested on Ubuntu 20.04 only)
* add custom wait events for existing contrib modules (ex. postgres_fdw)
* add regression code (but, it seems to be difficult)
* others? (Please let me know)

[1]
https://www.postgresql.org/message-id/81290a48-b25c-22a5-31a6-3feff5864fe3%40gmail.com

Regards,

--
Masahiro Ikeda
NTT DATA CORPORATION

Attachment Content-Type Size
0001-Support-to-define-custom-wait-events-for-extensions.patch text/x-diff 16.7 KB
0003-Add-docs-to-define-custom-wait-events.patch text/x-diff 3.0 KB
0002-Add-a-extension-to-test-custom-wait-event.patch text/x-diff 5.7 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Peter Smith 2023-06-15 06:14:05 Re: Initial Schema Sync for Logical Replication
Previous Message Kyotaro Horiguchi 2023-06-15 05:30:34 Re: add non-option reordering to in-tree getopt_long