RE: Disable WAL logging to speed up data loading

From: "tsunakawa(dot)takay(at)fujitsu(dot)com" <tsunakawa(dot)takay(at)fujitsu(dot)com>
To: 'Kyotaro Horiguchi' <horikyota(dot)ntt(at)gmail(dot)com>, "sfrost(at)snowman(dot)net" <sfrost(at)snowman(dot)net>
Cc: "osumi(dot)takamichi(at)fujitsu(dot)com" <osumi(dot)takamichi(at)fujitsu(dot)com>, "masao(dot)fujii(at)oss(dot)nttdata(dot)com" <masao(dot)fujii(at)oss(dot)nttdata(dot)com>, "ashutosh(dot)bapat(dot)oss(at)gmail(dot)com" <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>, "pgsql-hackers(at)lists(dot)postgresql(dot)org" <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: RE: Disable WAL logging to speed up data loading
Date: 2020-11-11 09:24:48
Message-ID: TYAPR01MB2990894C2F5FCAECAD837DADFEE80@TYAPR01MB2990.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

From: Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>
> Thanks! Since this feature is different from the feature that is
> being proposed in this thhead, I started another thread for this
> feature.
>
> https://www.postgresql.org/message-id/20201111.173317.460890039962481
> 381(dot)horikyota(dot)ntt(at)gmail(dot)com

Thank you, Horiguchi-san(^^) I was just about to ask you to separate the thread, because I think both features are good:

* ALTER TABLE SET UNLOGGED/LOGGED without data copy
Good:
- Does not require server restart (if this feature can be used in all wal_level settings).

Bad:
- The user has to maintain and modify some scripts to use ALTER TABLE when adding or removing the tables/partitions to load data into. For example, if the data loading job specifies a partitioned table, he may forget to add ALTER TABLE for new partitions, resulting in slow data loading.

* wal_level = none
Good:
- Easy to use. The user does not have to be aware of what tables are loaded. This can come in handy when migrating from an older version or another DBMS, building test databases, and consolidating databases.

Bad:
- Requires server restart.

* Both features
Bad:
- Requires taking database backup.
- Requires rebuilding the standby.

Sadly, the new thread's title includes a spelling mistake of extra r -- "In-placre". (I hope this won't prevent the search hit in the mail archive.)

I expect both features will be able to meet our customer's needs. The worst scenario (I don't want to imagine!) is that neither feature fails to be committed. So, let us continue both features. I'll join Horiguchi-san's new thread, and please help us here too. (I'll catch up with the recent discussion in this thread and reply.)

> Couldn't we have something like the following?
>
> ALTER TABLE table1, table2, table3 SET UNLOGGED;
>
> That is, multiple target object specification in ALTER TABLE sttatement.

Likewise, can't we do ALTER TABLE SET UNLOGGED/LOGGED against a partitioned table? Currently, the statement succeeds but none of the partitioned table nor its partitions is set unlogged (pg_class.relpersistence remains 'p'). Is this intended? If it's a bug, I'm willing to fix it so that it reports an eror. Of course, it's good to make all partitions unlogged at once.

Regards
Takayuki Tsunakawa

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Langote 2020-11-11 09:52:01 Re: ModifyTable overheads in generic plans
Previous Message Peter Eisentraut 2020-11-11 09:12:50 Clean up optional rules in grammar