Re: Disable WAL logging to speed up data loading

From: Laurenz Albe <laurenz(dot)albe(at)cybertec(dot)at>
To: Stephen Frost <sfrost(at)snowman(dot)net>, "tsunakawa(dot)takay(at)fujitsu(dot)com" <tsunakawa(dot)takay(at)fujitsu(dot)com>
Cc: 'David Steele' <david(at)pgmasters(dot)net>, "osumi(dot)takamichi(at)fujitsu(dot)com" <osumi(dot)takamichi(at)fujitsu(dot)com>, 'Kyotaro Horiguchi' <horikyota(dot)ntt(at)gmail(dot)com>, "sawada(dot)mshk(at)gmail(dot)com" <sawada(dot)mshk(at)gmail(dot)com>, "robertmhaas(at)gmail(dot)com" <robertmhaas(at)gmail(dot)com>, "masao(dot)fujii(at)oss(dot)nttdata(dot)com" <masao(dot)fujii(at)oss(dot)nttdata(dot)com>, "ashutosh(dot)bapat(dot)oss(at)gmail(dot)com" <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>, "pgsql-hackers(at)lists(dot)postgresql(dot)org" <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Disable WAL logging to speed up data loading
Date: 2021-03-22 14:59:04
Message-ID: 7c2723df1c62b6707055643483b54031d8cb840d.camel@cybertec.at
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, 2021-03-22 at 09:46 -0400, Stephen Frost wrote:
> * tsunakawa(dot)takay(at)fujitsu(dot)com (tsunakawa(dot)takay(at)fujitsu(dot)com) wrote:
> > From: Stephen Frost <sfrost(at)snowman(dot)net>
> > > The argument here seems to stem from the idea that issueing a 'TRUNCATE'
> > > inside the transaction before starting the 'COPY' command is 'too hard'.
> >
> > No, we can't ask using TRUNCATE because the user wants to add data to a table.
>
> First- what are you expecting would actually happen during crash
> recovery in this specific case with your proposed new WAL level?
>
> Second, use partitioning, or unlogged tables (with the patch discussed
> elsewhere to allow flipping them to logged without writing the entire
> thing to WAL).

Perhaps allowing to set unlogged tables to logged ones without writing WAL
is a more elegant way to do that, but I cannot see how that would be any
more crash safe than this patch. Besides, the feature doesn't exist yet.

So I think that argument doesn't carry much weight.

Yours,
Laurenz Albe

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Robert Haas 2021-03-22 15:05:19 Re: [HACKERS] Custom compression methods
Previous Message Fujii Masao 2021-03-22 14:59:02 Re: make the stats collector shutdown without writing the statsfiles if the immediate shutdown is requested.