Re: Survey on backing up unlogged tables: help us with PostgreSQL development!

From: "Marc Mamin" <M(dot)Mamin(at)intershop(dot)de>
To: "Glen Parker" <glenebob(at)nwlink(dot)com>, "PostgreSQL general" <pgsql-general(at)postgresql(dot)org>
Cc: "Josh Berkus" <josh(at)agliodbs(dot)com>
Subject: Re: Survey on backing up unlogged tables: help us with PostgreSQL development!
Date: 2010-11-17 21:18:50
Message-ID: C4DAC901169B624F933534A26ED7DF31034BB908@JENMAIL01.ad.intershop.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

I would like to choose the table behaviour on restart (restore/forget it)


Currently, I'm looking for a way to split large transaction on different threads (with dblink).
AN issue is to efficiently share temp data across the threads. unlogged tables would be here fine, something like global temp tables with shared data.

here an example to illustrate the current situation:

select cic_connect_me('c');
select dblink_exec ('c', 'drop table if exists my_share');
select dblink_exec ('c', 'create table my_share( a int)');
select dblink_disconnect ('c');

SELECT cic_multithread(ARRAY[
'insert into my_share select * from generate_series(1,10000)',
'insert into my_share select * from generate_series(1,10000)',
'insert into my_share select * from generate_series(1,10000)',
'insert into my_share select * from generate_series(1,10000)',
'insert into my_share select * from generate_series(1,10000)',
'insert into my_share select * from generate_series(1,10000)']
,max_threads=4);

create temp table my_result as select * from my_share;
drop table my_share;

select * from my_result;

For pg dump, I guess that having an optional flag is fine, but:
unlogged tables could also be useful to store very large 'raw' data to be processed,
whereas the client would only query the processed results.
In such a case, restoring the logged table has a higher priority.
The best solution in my opinion, would allow to dump/restore these 2 table types in separate processes (or threads..).

(and by the way: would it be possible to choose the compress tool as an option for pg_dump)

pgdump -F.. -Compress pigz -f out.dmp -f_unlogged out_unlogged.dmp.

regards,

Marc Mamin

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Marco Colombo 2010-11-17 22:15:30 Re: Linux x Windows LOCALE/ENCODING compatibility
Previous Message Dimitri Fontaine 2010-11-17 21:03:23 Re: Adding data from mysql to postgresql periodically