| From: | "Tsunakawa, Takayuki" <tsunakawa(dot)takay(at)jp(dot)fujitsu(dot)com> |
|---|---|
| To: | 'Srinivas Karthik V' <skarthikv(dot)iitb(at)gmail(dot)com>, Don Seiler <don(at)seiler(dot)us> |
| Cc: | PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>, Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com> |
| Subject: | RE: Bulk Insert into PostgreSQL |
| Date: | 2018-07-02 00:19:34 |
| Message-ID: | 0A3221C70F24FB45833433255569204D1FA27379@G01JPEXMBYT05 |
| Views: | Raw Message | Whole Thread | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
From: Srinivas Karthik V [mailto:skarthikv(dot)iitb(at)gmail(dot)com]
> I was using copy command to load. Removing the primary key constraint on
> the table and then loading it helps a lot. In fact, a 400GB table was loaded
> and the primary constraint was added in around 15 hours. Thanks for the
> wonderful suggestions.
400 GB / 15 hours = 7.6 MB/s
That looks too slow. I experienced a similar slowness. While our user tried to INSERT (not COPY) a billion record, they reported INSERTs slowed down by 10 times or so after inserting about 500 million records. Periodic pstack runs on Linux showed that the backend was busy in btree operations. I didn't pursue the cause due to other businesses, but there might be something to be improved.
Regards
Takayuki Tsunakawa
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Thomas Munro | 2018-07-02 00:39:00 | Re: Synchronous replay take III |
| Previous Message | Thomas Munro | 2018-07-02 00:14:19 | Re: Commitfest 2018-07 is underway |