| From: | Simon Riggs <simon(at)2ndquadrant(dot)com> |
|---|---|
| To: | Josh Berkus <josh(at)agliodbs(dot)com> |
| Cc: | pgsql-performance(at)postgresql(dot)org, John A Meinel <john(at)arbash-meinel(dot)com>, bizgres-general <bizgres-general(at)pgfoundry(dot)org>, Dan Harris <fbsd(at)drivefaster(dot)net> |
| Subject: | Re: [Bizgres-general] Re: faster INSERT with possible |
| Date: | 2005-07-27 21:37:47 |
| Message-ID: | 1122500267.3670.184.camel@localhost.localdomain |
| Views: | Raw Message | Whole Thread | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-performance |
On Wed, 2005-07-27 at 09:29 -0700, Josh Berkus wrote:
> Luke,
>
> > Well - now that I test it, it appears you are correct, temp table COPY
> > bypasses WAL - thanks for pointing it out!
>
> RIght. The problem is bypassing WAL for loading new "scratch" tables which
> aren't TEMPORARY tables. We need to do this for multi-threaded ETL, since:
> a) Temp tables can't be shared by several writers, and
> b) you can't index a temp table.
The description of "scratch" tables might need some slight
clarification. It kindof makes it sound like temp tables.
I had in mind the extra tables that an application sometimes needs to
operate faster. Denormalisations, pre-joined tables, pre-calculated
results, aggregated data. These are not temporary tables, just part of
the application - multi-user tables that stay across shutdown/restart.
If you have gallons of GB, you will probably by looking to make use of
such tables.
You can use such tables for the style of ETL known as ELT, but that is
not the only use.
Best Regards, Simon Riggs
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Mark Wong | 2005-07-27 21:42:01 | Re: wal_buffer tests in |
| Previous Message | Sebastian Hennebrueder | 2005-07-27 21:21:39 | Re: Mirroring PostgreSQL database |