Re: COPY v. java performance comparison

On 4/3/2014 12:57 PM, Andy Colson wrote:
> On 4/3/2014 12:15 PM, Rob Sargent wrote:
>> Is the java app cpu bound?
>>>
>> > Also watch "vmstat 3" for a minute or two. The last two numbers (wa
>> > & id) (some vmstat's have a steal, ignore that) will tell you if you
>> > are io bound.
>> >
>> > -Andy
>> >
>> >
>> During COPY, with autovaccume off (server restarted, manual vacuum to
>> get things going). Immediately hit the "too frequently" gripe.
>> checkpoint_segments still at 12.
>>
>> Not cpu bound, so waiting for IO, but surely one expects that?
>>
>> vmstat 3
> <snip>
>
>
>
> Did you watch top? Is anything at 100%?
>
> How many cpus are you running? (well, cpus*cores I guess)
>
> -Andy
>
>
>

From Rob, off list:

Two quad-core
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz

So, 8 cores, so I'd bet one of your cores, probably the PG doing the
copy, is io bound.

>> Not cpu bound, so waiting for IO, but surely one expects that?

Not with good IO sub system. We've seen the COPY process be cpu bound
before.

Theses lines:
> 1 1 100148 3267020 763272 33778732 0 0 0 43125 10968 20848 5 2 90 3 0
> 0 1 100148 3250528 763272 33792804 0 0 0 63567 10467 20121 2 1 81 16 0
> 0 1 100148 3258584 763276 33784308 0 0 0 17121 10553 20146 1 1 86 12 0

Tell an interesting story. On the first line, one process is running,
the cpu's are 90% idle, and 3% io wait. On the next two lines (for the
next 6 seconds), there are no running processes, and one cpu is totally
in io wait. (100% / 8 cores = 12%)

Its also interesting that there are zero bi (blocks in), and lots of bo
(blocks out). I'm guessing the java app is reading completely from cache.

From your original post of vmstat the last 5 lines have no processes
running ... I think that means for at least 15 seconds (5 lines * 3
seconds), everything was blocked waiting for IO.

Have you tried the dd benchmark tests?

http://www.westnet.com/~gsmith/content/postgresql/pg-disktesting.htm

-Andy