Re: Create database from template very slow

From: Mark Phillips <mark(dot)phillips(at)mophilly(dot)com>
To: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
Cc: "David G(dot) Johnston" <david(dot)g(dot)johnston(at)gmail(dot)com>, PostgreSQL mailing lists <pgsql-bugs(at)lists(dot)postgresql(dot)org>
Subject: Re: Create database from template very slow
Date: 2019-12-05 19:52:10
Message-ID: 9B36A9B4-5DF9-4441-8B96-69E8E334946D@mophilly.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

Thank you both.

I will look into joining the general list.

We have observed the long execution time on several windows machines, including those of end users. Also, due to the nature of our deployment, the postgres version is the same for all users. All user databases are small, typically being less than 10MB.

The suggestion for timing the stages of CREATE DATABASE is good. I am not familiar with tracing postgres functionality, so I will dig into this and see how far I get.

Cheers!

> On Dec 3, 2019, at 9:16 PM, Thomas Munro <thomas(dot)munro(at)gmail(dot)com> wrote:
>
> On Wed, Dec 4, 2019 at 5:51 PM David G. Johnston
> <david(dot)g(dot)johnston(at)gmail(dot)com <mailto:david(dot)g(dot)johnston(at)gmail(dot)com>> wrote:
>> On Tue, Dec 3, 2019 at 9:27 PM Mark Phillips <mark(dot)phillips(at)mophilly(dot)com> wrote:
>>> This is likely not a bug, so please point me in the right direction if that is the case.
>>
>> The -general list is you best choice if you know its not a bug but don't know where else to send the email.
>>
>>> In our application, a copy of the local database is made at launch using...
>>> CREATE DATABASE myCopy FROM TEMPLATE currentDB;
>>>
>>> On linux and MacOS this operation completed in 5 seconds or so. On Windows (10 at present) it takes over a minute or more.
>>>
>>> I would like to understand why this would be, and what might be done to speed it up on Windows.
>>>
>>
>> Less capable hardware on the Windows box? Use better hardware? This isn't my area of expertise but I'm doubtful that any significant fault/difference exists within the PostgreSQL software. Is the data involved identical?
>
> CREATE DATABASE (createdb()) does a checkpoint, copies all the files
> with individual fsync, does another checkpoint. It'd be interesting
> to narrow down what's taking time. How long does manual CHECKPOINT
> take, how long does a recursive copy with the Windows command line
> take (xcopy?), how long does CREATE DATABASE take if you run with
> fsync = off (do this on a test database you don't care about)? If
> that doesn't reveal where the time is spent, I'd probably start
> investigating by using whatever Windows thing is like truss/strace
> (NtTrace seems to be a thing?) or adding a bunch of elog(LOG, "about
> to copy files ...") type calls into createdb(), if I had a compiler
> handy.

In response to

Browse pgsql-bugs by date

  From Date Subject
Next Message MOHAN KUMAR DORAIRAJ 2019-12-06 03:34:23 Re: BUG #16145: Not able to terminate active session
Previous Message Jeff Janes 2019-12-05 16:52:06 Re: BUG #16148: Query on Large table hangs in ETL flows and gives out of memory when run in pgAdmin4