Re: Speed up Clog Access by increasing CLOG buffers

From: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: Ashutosh Sharma <ashu(dot)coek88(at)gmail(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Michael Paquier <michael(dot)paquier(at)gmail(dot)com>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, Jim Nasby <Jim(dot)Nasby(at)bluetreble(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Speed up Clog Access by increasing CLOG buffers
Date: 2017-08-30 07:24:03
Message-ID: CAA4eK1KQJbzQ2E9_hV5Ajjqk0Y4AmGzYhDd9=JnNzr7YgxgQ8g@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox
Thread:
Lists: pgsql-hackers

On Wed, Aug 30, 2017 at 2:43 AM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> On Tue, Jul 4, 2017 at 12:33 AM, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>> I have updated the patch to support wait events and moved it to upcoming CF.
>
> This patch doesn't apply any more, but I made it apply with a hammer
> and then did a little benchmarking (scylla, EDB server, Intel Xeon
> E5-2695 v3 @ 2.30GHz, 2 sockets, 14 cores/socket, 2 threads/core).
> The results were not impressive. There's basically no clog contention
> to remove, so the patch just doesn't really do anything.
>

Yeah, in such a case patch won't help.

> For example,
> here's a wait event profile with master and using Ashutosh's test
> script with 5 savepoints:
>
> 1 Lock | tuple
> 2 IO | SLRUSync
> 5 LWLock | wal_insert
> 5 LWLock | XidGenLock
> 9 IO | DataFileRead
> 12 LWLock | lock_manager
> 16 IO | SLRURead
> 20 LWLock | CLogControlLock
> 97 LWLock | buffer_content
> 216 Lock | transactionid
> 237 LWLock | ProcArrayLock
> 1238 IPC | ProcArrayGroupUpdate
> 2266 Client | ClientRead
>
> This is just a 5-minute test; maybe things would change if we ran it
> for longer, but if only 0.5% of the samples are blocked on
> CLogControlLock without the patch, obviously the patch can't help
> much. I did some other experiments too, but I won't bother
> summarizing the results here because they're basically boring. I
> guess I should have used a bigger machine.
>

That would have been better. In any case, will do the tests on some
higher end machine and will share the results.

> Given that we've changed the approach here somewhat, I think we need
> to validate that we're still seeing a substantial reduction in
> CLogControlLock contention on big machines.
>

Sure will do so. In the meantime, I have rebased the patch.

--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

Attachment Content-Type Size
group_update_clog_v14.patch application/octet-stream 16.4 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Etsuro Fujita 2017-08-30 08:20:31 Re: Tuple-routing for certain partitioned tables not working as expected
Previous Message Fabien COELHO 2017-08-30 06:39:30 Re: pgbench: Skipping the creating primary keys after initialization