From: | Bruce Momjian <maillist(at)candle(dot)pha(dot)pa(dot)us> |
---|---|
To: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
Cc: | Wayne Piekarski <wayne(at)senet(dot)com(dot)au>, pgsql-hackers(at)postgreSQL(dot)org |
Subject: | Re: [HACKERS] Inefficiencies in COPY command |
Date: | 1999-09-27 18:04:42 |
Message-ID: | 199909271804.OAA10131@candle.pha.pa.us |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Yes, I too would be interested in any code that would speed up COPY
without loosing modularity or portability.
Please let us know if you get a patch we can include in our source tree.
> Wayne Piekarski <wayne(at)senet(dot)com(dot)au> writes:
> > So I made some changes to CopyAttributeOut so that it escapes the string
> > initially into a temporary buffer (allocated onto the stack) and then
> > feeds the whole string to the CopySendData which is a lot more efficient
> > because it can blast the whole string in one go, saving about 1/3 to 1/4
> > the number of memcpy and so on.
>
> copy.c is pretty much of a hack job to start with, IMHO. If you can
> speed it up without making it even uglier, have at it! However, it
> also has to be portable, and what you've done here:
>
> > CopyAttributeOut(FILE *fp, char *server_string, char *delim, int is_array)
> > {
> > char *string;
> > char c;
> > + char __buffer [(strlen (server_string) * 2) + 4]; /* Use +4 for safety */
>
> is not portable --- variable-sized local arrays are a gcc-ism. (I use
> 'em a lot too, but not in code intended for public release...) Also,
> be careful that you don't introduce any assumptions about maximum
> field or tuple width; we want to get rid of those, not add more.
>
> > to also make changes to remove all use of sprintf when numbers
> > and floats are being converted into strings.
> ^^^^^^^^^^
>
> While formatting an int is pretty simple, formatting a float is not so
> simple. I'd be leery of replacing sprintf with quick-hack float
> conversion code. OTOH, if someone wanted to go to the trouble of doing
> it *right*, using our own code would tend to yield more consistent
> results across different OSes, which would be a Good Thing. I'm not
> sure it'd be any faster than the typical sprintf, but it might be worth
> doing anyway.
>
> (My idea of *right* is the guaranteed-inverse float<=>ASCII routines
> published a few years ago in some SIGPLAN proceedings ... I've got the
> proceedings, and I even know approximately where they are, but I don't
> feel like excavating for them right now...)
>
> regards, tom lane
>
>
--
Bruce Momjian | http://www.op.net/~candle
maillist(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026
From | Date | Subject | |
---|---|---|---|
Next Message | Bruce Momjian | 1999-09-27 18:05:29 | Re: [HACKERS] another DECIMAL problem |
Previous Message | Oleg Bartunov | 1999-09-27 17:59:50 | NULL as an argument in plpgsql functions |