Bugs/slowness inserting and indexing cubes

From: Jay Levitt <jay(dot)levitt(at)gmail(dot)com>
To: pgsql-hackers(at)postgresql(dot)org
Subject: Bugs/slowness inserting and indexing cubes
Date: 2012-02-07 19:26:20
Message-ID: 4F317ADC.4060005@gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

[Posted at Andres's request]

TL;DR: Inserting and indexing cubes is slow and/or broken in various ways in
various builds.

NOTABLE PROBLEMS

1. In 9.1.2, inserting 10x rows takes 19x the time.
- 9.1-HEAD and 9.2 "fix" this; it now slows down linearly
- but: 10s > 8s > 5s!
- but: comparing Ubuntu binary w/vanilla source build on virtual disks,
might not be significant

2. In both 9.1 and 9.2, there is a long delay before CREATE INDEX realizes
it can't work on an unlogged table
3. In 9.2, creating the 10-million-row index always fails
4. 9.1-HEAD never successfully indexes 10 million rows ("never" = at least
20 minutes on two runs; I will follow up in a few hours)

DETAILS

Times are in seconds, single run.

+-------------------+---------+---------+----------+----------+
| Platform | 1m rows | 1m rows | 10m rows | 10m rows |
| | INSERT | CR NDX | INSERT | CR NDX |
+-------------------+---------+---------+----------+----------+
| 9.1.2 logged | 5 | 35 | 98 | 434 |
| 9.1.2 unlogged | 2 | 34[**] | 22 | 374[**] |
| 9.1-HEAD logged | 10 | 65 | 89 | [***] |
| 9.1-HEAD unlogged | 2 | 39 | 20 | 690[**] |
| 9.2 logged | 8 | 57 | 87 | 509[*] |
| 9.2 unlogged | 2 | 33[**] | 21 | 327[*] |
+-------------------+---------+---------+----------+----------+

[*] psql:slowcube.sql:20: ERROR: node buffer of page being split (121550)
does not exist
[**] psql:slowcube.sql:21: ERROR: unlogged GiST indexes are not supported
[***] never completed after 10-20 minutes; nothing in server.log at default
logging levels, postgres process consuming about 1 CPU in IOWAIT,
checkpoints every 7-8 seconds

VARIABILITY

A few runs in a row on 9.1-HEAD, 1 million rows, logged:

+--------+--------------+
| INSERT | CREATE INDEX |
+--------+--------------+
| 10 | 65 |
| 8 | 61 |
| 7 | 59 |
| 8 | 61 |
| 7 | 55 |
+--------+--------------+

SYSTEM SPECS

Amazon EC2, EBS-backed, m1.large
7.5GB RAM, 2 cores
Intel(R) Xeon(R) CPU E5645 @ 2.40GHz

shared_buffers = 1867MB
checkpoint_segments = 32
effective_cache_size = 3734MB

9.1.2: installed binaries from Ubuntu's oneiric repo
9.1-HEAD: REL9_1_STABLE, ef19c9dfaa99a2b78ed0f78aa4a44ed31636fdc4, built
with simple configure/make/make install
9.2: master, 1631598ea204a3b05104f25d008b510ff5a5c94a, built with simple
configure/make/make install

9.1.2 and 9.1-HEAD were run on different (but identically configured)
instances. 9.1-HEAD and 9.2 were run on the same instance, but EBS
performance is unpredictable. YMMV.

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Jay Levitt 2012-02-07 19:28:11 Re: Bugs/slowness inserting and indexing cubes
Previous Message Bruce Momjian 2012-02-07 17:51:45 Re: Setting -Werror in CFLAGS