RE: [Patch] Optimize dropping of relation buffers using dlist

From: "Tang, Haiying" <tanghy(dot)fnst(at)cn(dot)fujitsu(dot)com>
To: "Tang, Haiying" <tanghy(dot)fnst(at)cn(dot)fujitsu(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: "Tsunakawa, Takayuki" <tsunakawa(dot)takay(at)fujitsu(dot)com>, "Jamison, Kirk" <k(dot)jamison(at)fujitsu(dot)com>, Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>, "Andres Freund" <andres(at)anarazel(dot)de>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: RE: [Patch] Optimize dropping of relation buffers using dlist
Date: 2020-12-28 08:15:15
Message-ID: 66851e198f6b41eda59e6257182564b6@G08CNEXMBPEKD05.g08.fujitsu.local
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi Amit,

>I think one table with a varying amount of data is sufficient for the vacuum test.
>I think with more number of tables there is a greater chance of variation.
>We have previously used multiple tables in one of the tests because of
>the Truncate operation (which uses DropRelFileNodesAllBuffers that
>takes multiple relations as input) and that is not true for Vacuum operation which I suppose you are testing here.

I retested performance on single table for several times, the table size is varying with the BUF_DROP_FULL_SCAN_THRESHOLD for different shared buffers.
When shared_buffers is below 20G, there were no significant changes between master(HEAD) and patched.
And according to the results compared between 20G and 100G, we can get optimization up to NBuffers/128, but there is no benefit from NBuffers/256.
I've tested many times, most times the same results came out, I don't know why. But If I used 5 tables(each table size is set as BUF_DROP_FULL_SCAN_THRESHOLD), then we can get benefit from it(NBuffers/256).

Here is my test results for single table. If you have any question or suggestion, kindly let me know.

%reg= (patched- master(HEAD))/ patched
Optimized percentage:
shared_buffers %reg(NBuffers/512) %reg(NBuffers/256) %reg(NBuffers/128) %reg(NBuffers/64) %reg(NBuffers/32) %reg(NBuffers/16) %reg(NBuffers/8) %reg(NBuffers/4)
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
128M 0% 0% -1% 0% 1% 0% 0% 0%
1G -1% 0% -1% 0% 0% 0% 0% 0%
20G 0% 0% -33% 0% 0% -13% 0% 0%
100G -32% 0% -49% 0% 10% 30% 0% 6%

Result details(unit: second):
patched (sec)
shared_buffers NBuffers/512 NBuffers/256 NBuffers/128 NBuffers/64 NBuffers/32 NBuffers/16 NBuffers/8 NBuffers/4
128M 0.107 0.107 0.107 0.107 0.108 0.107 0.108 0.208
1G 0.107 0.107 0.107 0.108 0.208 0.208 0.308 0.409
20G 0.208 0.308 0.308 0.409 0.609 0.808 1.511 2.713
100G 0.309 0.408 0.609 1.010 2.011 5.017 6.620 13.931

master(HEAD) (sec)
shared_buffers NBuffers/512 NBuffers/256 NBuffers/128 NBuffers/64 NBuffers/32 NBuffers/16 NBuffers/8 NBuffers/4
128M 0.107 0.107 0.108 0.107 0.107 0.107 0.108 0.208
1G 0.108 0.107 0.108 0.108 0.208 0.207 0.308 0.409
20G 0.208 0.309 0.409 0.409 0.609 0.910 1.511 2.712
100G 0.408 0.408 0.909 1.010 1.811 3.515 6.619 13.032

Regards
Tang

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Masahiko Sawada 2020-12-28 08:43:26 Re: Add table AM 'tid_visible'
Previous Message Peter Geoghegan 2020-12-28 07:41:44 Re: New IndexAM API controlling index vacuum strategies