| From: | Nadav Shatz <nadav(at)tailorbrands(dot)com> |
|---|---|
| To: | pgpool-hackers(at)lists(dot)postgresql(dot)org |
| Subject: | Proposal: Recent mutated table tracking in memory |
| Date: | 2026-01-06 11:25:54 |
| Message-ID: | CACeKOO2hjPmstboJaa=rw8Erd7k5VhXyupU39bAosCPtUe1UBA@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgpool-hackers |
Hello,
As initially proposed under "Proposal: recent access based routing for
primary-replica setups" and then broken into separate tasks - i am adding
here a patch to implement tracking of latest mutated table, and then using
the replication lag as a base - deciding where to point queries when query
load balancing and parsing is enabled.
More details as in the patch:
Feature: add in-memory table tracking to prevent stale reads from replicas
Implement "memory map" feature that tracks recently-written database
tables in shared memory to prevent stale reads during replication lag.
When a write (INSERT/UPDATE/DELETE) occurs on a table, that table is
marked as "dirty" for a configurable TTL period. Any SELECT on a dirty
table within the TTL window is routed to primary instead of replica.
Key features:
- Shared memory hash table for tracking table mutations with TTL
- Query parse cache with LRU eviction for performance
- Cold start protection (routes all queries to primary initially)
- Automatic TTL calculation: replication_delay × configurable factor
- Per-table staleness tracking with microsecond precision
New configuration parameters:
- memory_map_enabled: Enable/disable the feature (default: off)
- memory_map_ttl_factor: TTL multiplier for replication delay (default: 5.0)
- memory_map_cold_start_duration: Cold start period in ms (default: 2000)
- memory_map_table_buckets: Hash buckets for table map (default: 1024)
- memory_map_table_size: Max tracked tables (default: 2048)
- memory_map_query_buckets: Hash buckets for query cache (default: 2048)
- memory_map_query_cache_size: Max cached queries (default: 10000)
Patch applies properly and tests pass.
Open to all feedback - thank you!
--
Nadav Shatz
Tailor Brands | CTO
| Attachment | Content-Type | Size |
|---|---|---|
| mutated_table.patch | application/octet-stream | 67.1 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Tatsuo Ishii | 2026-01-06 13:01:23 | Re: Feature: reduce sync messages |
| Previous Message | Tatsuo Ishii | 2026-01-06 06:43:20 | Re: Proposal: recent access based routing for primary-replica setups |