Re: [HACKERS] Inefficiencies in COPY command

From: Bruce Momjian <maillist(at)candle(dot)pha(dot)pa(dot)us>
To: Wayne Piekarski <wayne(at)senet(dot)com(dot)au>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, pgsql-hackers(at)postgreSQL(dot)org
Subject: Re: [HACKERS] Inefficiencies in COPY command
Date: 1999-08-24 04:13:17
Message-ID: 199908240413.AAA16086@candle.pha.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> Tom Lane wrote -
> > Wayne Piekarski <wayne(at)senet(dot)com(dot)au> writes:
> > > So I made some changes to CopyAttributeOut so that it escapes the string
> > > initially into a temporary buffer (allocated onto the stack) and then
> > > feeds the whole string to the CopySendData which is a lot more efficient
> > > because it can blast the whole string in one go, saving about 1/3 to 1/4
> > > the number of memcpy and so on.
> >
> > copy.c is pretty much of a hack job to start with, IMHO. If you can
> > speed it up without making it even uglier, have at it! However, it
> > also has to be portable, and what you've done here:
>
> Ok, well I will write up a proper patch for CopyAttributeOut so it is not
> such a hack (using all those #defines and stuff wasn't very "elegant") and
> then submit a proper patch for it.... This was pretty straight forward to
> fix up.

Great.

>
> > While formatting an int is pretty simple, formatting a float is not so
> > simple. I'd be leery of replacing sprintf with quick-hack float
> > conversion code. OTOH, if someone wanted to go to the trouble of doing
> > it *right*, using our own code would tend to yield more consistent
> > results across different OSes, which would be a Good Thing. I'm not
> > sure it'd be any faster than the typical sprintf, but it might be worth
> > doing anyway.
>
> I understand there are issues to do with not being able to use GPL code
> with Postgres, because its BSD license is not compatible, but would it be
> acceptable to extract code from BSD style code? If so, my FreeBSD here has
> libc code and includes the internals used by sprintf for rendering
> integers (and floats) and so we could include that code in, and should
> hopefully be portable at the same time as well.
>
> This would be a lot faster than going via sprintf and lots of other
> functions, and would make not just COPY, but I think any SELECT query runs
> faster as well (because they get rewritten to strings by the output
> functions don't they). I guess other advantages would be improvements in
> the regression tests maybe, for problem types like int8 which in the past
> have had trouble under some BSDs.

Does using the FreeBSD sprintf conversion functions really make it
faster than just calling sprintf? How?

--
Bruce Momjian | http://www.op.net/~candle
maillist(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message The Hermit Hacker 1999-08-24 05:20:52 Re: [HACKERS] Tangent ... you know what's scary?
Previous Message Bruce Momjian 1999-08-24 03:59:51 Re: [HACKERS] vacuum process size