Skip site navigation (1) Skip section navigation (2)

Re: Inserting 8MB bytea: just 25% of disk perf used?

From: "fkater(at)googlemail(dot)com" <fkater(at)googlemail(dot)com>
To: Hannu Krosing <hannu(at)2ndquadrant(dot)com>
Cc: pgsql-performance(at)postgresql(dot)org
Subject: Re: Inserting 8MB bytea: just 25% of disk perf used?
Date: 2010-01-18 13:08:38
Message-ID: 20100118130838.GC2913@comppasch2 (view raw or flat)
Thread:
Lists: pgsql-performance
Hannu Krosing:

> On Thu, 2010-01-14 at 21:14 +0100, fkater(at)googlemail(dot)com wrote:
> > Thanks a lot for your reply.
> > 
> > Hannu Krosing:
> > 
> > > > 4 Core CPU 3 Ghz, WinXP, 1 TB SATA disk.
> > > 
> > > try inserting the same data using 4 parallel connections or even 8
> > > parallel ones.
> > 
> > Interesting idea -- I forgot to mention though that 2-3
> > cores will be occupied soon with other tasks.
> 
> Even one core will probably be idling at the througput you mentioned, so
> the advice still stands, use more than one connection to get better
> throughput.

Thank You. Since connecting more than once would mean some
major changes in my db layer I fear considering it as a
solution.

BTW: I do not get the idea behind that. Since firstly, I
later will have just one core free for postgres processes,
and secondly neither the cpu nor the postgres processes seem
to be really busy yet. Do you mean a postgres process may be
programmed in a way that it waits for something unknown
which can be surrounded by feeding another postgres process
with work, even on the same CPU?

As a short check, this is what I did (see other postings
from today for further scenarios I tested):

Setting:

* About 11.1 GB data in the table "bin_table" on a
  separate "data disk" from the tests the last days (mostly
  rows of 80 MB bin data each)

* WAL/pg_xlog not symlinked to another disk anymore.

* created tables test + test2 "LIKE bin_table"

* 2x times pgAdminIII, running:
  INSERT INTO test SELECT * FROM bin_table;

  resp.

  INSERT INTO test2 SELECT * FROM bin_table;

Result:

* To copy those 11.1 GB into test + test2 in parallel it
  took 1699 s (13,17 MB/s)

This is what was to expect. It took quite exactly 2 times of
what 1 process needs for writing half of the data.


Thank You again.

 Felix



In response to

Responses

pgsql-performance by date

Next:From: fkater@googlemail.comDate: 2010-01-18 13:17:58
Subject: Re: Inserting 8MB bytea: just 25% of disk perf used?
Previous:From: fkater@googlemail.comDate: 2010-01-18 11:20:59
Subject: Re: Inserting 8MB bytea: just 25% of disk perf used?

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group