Re: Load distributed checkpoint

"Inaam Rana" <inaamrana(at)gmail(dot)com> wrote:

> Did you had a chance to look into this any further? We, at EnterpriseDB,
> have done some testing on this patch (dbt2 runs) and it looks like we are
> getting the desired results, particularly so when we spread out both sync
> and write phases.

Thank you for testing! Yes, I'm cleaning the patch. I changed configuration
parameters to delay each phase in checkpoints from setting absolute times
(checkpoint_xxx_duration) to setting relative to checkpoint_timeout
(checkpoint_xxx_percent). Delay factors strongly depend on total checkpoint
lengths, so I think the relative method is better.

I tested the patch on my machine, too. The belows are the results of
pgbench with/without the patch. As you see, the patch is not a complete
solution -- 12s of response time is not satisfiable yet -- but it was
better than other possible settings we could choose now.

pgbench -s100 -c16 -t100000
on the machine with 1GB ram and one SCSI drive

| A | B | C | D |
--------------------------+--------+--------+--------+--------+
bgwriter_all_maxpages | 5 | 5 | 60 | 120 |
checkpoint_write_percent | 50.0 | 0 | 0 | 0 |
checkpoint_nap_percent | 10.0 | 0 | 0 | 0 |
checkpoint_sync_percent | 20.0 | 0 | 0 | 0 |
--------------------------+--------+--------+--------+--------+
pgbench tps | 612.23 | 517.64 | 488.90 | 378.39 |
response average | 2.50ms | 2.89ms | 3.12ms | 3.99ms |
response maximum | 12.23s |123.66s | 55.09s | 36.72s |
--------------------------+--------+--------+--------+--------+

A: Proposed configurations with patch
B: Default configurations
C: Make bgwriter aggressive
D: Make bgwriter more aggressive

Other configurations:
- full_page_writes = off
- wal_buffers = 4MB
- checkpoint_segments = 32
- checkpoint_timeout = 15min

Regards,
---
ITAGAKI Takahiro
NTT Open Source Software Center