How to add big amounts of data into database?

From: "Olonichev Sergei" <olonichev(at)scnsoft(dot)com>
To: "Pgsql-Hackers(at)Postgresql(dot) Org" <pgsql-hackers(at)postgresql(dot)org>
Subject: How to add big amounts of data into database?
Date: 2002-05-24 15:57:38
Message-ID: 01c701c2033b$ba64d7c0$f566a8c0@olonichev
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hello,

Can anyone help me how to add big amount of data into database?

I tried COPY, first the addition speed was good, but after each minute it
became worser and worser.
You can see the table illustrating this below.

There were ~10,000,000 records.
There were no indexes created on table.
Table was locked in exclusive mode.

With the best regards,
Sergei

Table:
First column is size of table in kilobytes.
Second - speed of COPY command.

Size, Kb | Speed, Kb
--------------------
137416
139856 2440
142260 2404
144624 2364
146936 2312
149220 2284
151460 2240
153672 2212
155840 2168
157988 2148
160108 2120
162176 2068
164248 2072
166284 2036
168300 2016
170312 2012
172296 1984
174260 1964
176196 1936
178112 1916
180000 1888
181868 1868
183708 1840
185532 1824
187336 1804
189128 1792
190900 1772
192668 1768
194432 1764
196152 1720
197884 1732
199580 1696
201260 1680
202952 1692
204608 1656
206276 1668
207916 1640
209564 1648
211192 1628
212800 1608
214412 1612
216004 1592
217596 1592
219160 1564
220736 1576
222312 1576
223860 1548
225412 1552
226928 1516
228496 1568
230000 1504
231500 1500
232964 1464
234276 1312
235112 836
236584 1472
238040 1456
239508 1468
240980 1472
242500 1520
243568 1068
244632 1064
245784 1152
246904 1120
248340 1436
249772 1432
251232 1460
252672 1440
254096 1424
255524 1428
256948 1424
258444 1496
259912 1468
261024 1112
262152 1128
263460 1308
264860 1400
266276 1416
267688 1412
269096 1408
270488 1392
271876 1388
273300 1424
274612 1312
275684 1072
276920 1236
278136 1216
279344 1208
280356 1012
281212 856
281964 752
282652 688
283756 1104
285104 1348
286448 1344
287784 1336
289132 1348
290460 1328
291860 1400
292832 972
293712 880
294464 752
295424 960
296708 1284
297956 1248
299156 1200
300156 1000
300944 788
302240 1296
303104 864
304296 1192
305540 1244
306812 1272
308068 1256
309048 980
309792 744

Browse pgsql-hackers by date

  From Date Subject
Next Message Joe Conway 2002-05-24 17:27:18 Re: Redhat 7.3 time manipulation bug
Previous Message Joel Burton 2002-05-24 15:40:15 Re: Exposed function to find table in schema search list?