Skip site navigation (1) Skip section navigation (2)

Re: NOLOGGING option, or ?

From: Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
To: Luke Lonergan <llonergan(at)greenplum(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>,Alon Goldshuv <agoldshuv(at)greenplum(dot)com>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: NOLOGGING option, or ?
Date: 2005-06-02 01:05:11
Message-ID: 200506020105.j5215B703414@candle.pha.pa.us (view raw or flat)
Thread:
Lists: pgsql-hackers
Luke Lonergan wrote:
> Tom,
> > 
> > ... and instead, define some new behavior that will soon be considered
> > broken legacy code itself?
> 
> I'll not argue further about whether to have a separate LOAD command.
> That's not as important as fixing the performance issues in the data load
> path in PostgreSQL to me.
> 
> However, I find it compelling that the underlying problem confronting high
> performance data loading in the engine is the slow parse code in psql/copy,
> etc., and not the WAL or other issues.  Next on the list are huge gains from
> removing things like repetitive calls to strlen(TZ) in the attribute
> conversion code. 

Yep, we would _love_ those improvements.

> > There isn't any demand for changing the semantics of COPY, as far as
> > I've noticed.  If we can make it faster with the same semantics that's
> > great, but I'm not in favor of inventing an alternate that does almost
> > the same thing but (eg) breaks backslash handling in the name of speed.
> 
> A 540% increase in parsing speed while performing substantially the same
> logic is what the patch accomplishes.  The patch is now available on
> pgsql-patches.  In this case the speed did not come from breaking backslash
> handling, but rather from using faster processing for escape processing and
> delimiter parsing.  We *could* recreate identical escape semantics to COPY
> at nearly the same speed, but frankly we're puzzled as to why character
> sequences with "\" automatically imply escape processing.

I am confused why you are confused.  :-)

> We've found that there are many cases where more sophisticated escape
> processing options are necessary, including multi-byte delimiters and escape
> sequences, and the default use of "\" for escaping breaks many data load
> cases.

Uh, how do you do the escapes if you don't double the escape character
on input so you can distinguish a literal escape from one use to mark
special data like a literal delimiter or a null?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman(at)candle(dot)pha(dot)pa(dot)us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

In response to

Responses

pgsql-hackers by date

Next:From: Bruce MomjianDate: 2005-06-02 01:24:11
Subject: Re: [PATCHES] Adding \x escape processing to COPY, psql, backend
Previous:From: Bruce MomjianDate: 2005-06-02 01:01:56
Subject: Re: NOLOGGING option, or ?

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group