Re: TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

From: Dilip kumar <dilip(dot)kumar(at)huawei(dot)com>
To: Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: Andres Freund <andres(at)2ndquadrant(dot)com>, Magnus Hagander <magnus(at)hagander(dot)net>, Jan Lentfer <Jan(dot)Lentfer(at)web(dot)de>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, Sawada Masahiko <sawada(dot)mshk(at)gmail(dot)com>, Euler Taveira <euler(at)timbira(dot)com(dot)br>
Subject: Re: TODO : Allow parallel cores to be used by vacuumdb [ WIP ]
Date: 2015-01-23 11:23:57
Message-ID: 4205E661176A124FAF891E0A6BA91352663CB93B@szxeml509-mbs.china.huawei.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 22 January 2015 23:16, Alvaro Herrera Wrote,

> Here's v23.
>
> There are two things that continue to bother me and I would like you,
> dear patch author, to change them before committing this patch:
>
> 1. I don't like having vacuum_one_database() and a separate
> vacuum_one_database_parallel(). I think we should merge them into one
> function, which does either thing according to parameters. There's
> plenty in there that's duplicated.
>
> 2. in particular, the above means that run_parallel_vacuum can no
> longer exist as it is. Right now vacuum_one_database_parallel relies
> on run_parallel_vacuum to do the actual job parallellization. I would
> like to have that looping in the improved vacuum_one_database()
> function instead.

> Looking forward to v24,

Thanks you for your effort, I have tried to change the patch as per your instructions and come up with v24,

Changes:
1. In current patch vacuum_one_database (for table list), have the table loop outside and analyze_stage loop inside, so it will finish
All three stage for one table first and then pick the next table. But vacuum_one_database_parallel will do the stage loop outside and will call run_parallel_vacuum,
Which will have table loop, so for one stage all the tables will be vacuumed first, then go to next stage.
So for merging two function both functions behaviors should be identical, I think if user have given a list of tables in analyze-in-stages, than doing all the table
Atleast for one stage and then picking next stage will be better solution so I have done it that way.

2. in select_loop
For WIN32 TranslateSocketError function I replaced with
if (WSAGetLastError() == WSAEINTR)
errno == EINTR;

otherwise I have to expose TranslateSocketError function from socket and include extra header.

I have tested in windows also its working fine.

Regards,
Dilip

Attachment Content-Type Size
vacuumdb_parallel_v24.patch application/octet-stream 35.3 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Kapila 2015-01-23 11:42:51 Re: Parallel Seq Scan
Previous Message Abhijit Menon-Sen 2015-01-23 08:26:39 Re: pg_basebackup fails with long tablespace paths