Re: PG_DUMP very slow because of STDOUT ??

On 13/07/2010 3:53 PM, Andras Fabian wrote:
> Hi Scott,
>
> No, we didn't have a kernel update (it is still the stock Ubuntu 10.04 Server kernel ... 2.6.32.2). And in the meantime - this morning - I have discovered, that the rebooted server is again slowing down! It is not at the level of the not-rebooted-server (about 45 mins for the 3 Gig file)... it "only" needs 22 minutes, but it is already quite a bit away from the optimum of 3 minutes (or less). So, definitely, something is "deteriorating" in the system ...
>
> And I also did dome readings with iostat -xd 5 ... And the target drive to which the output of the STDOUT is directed is below 1% utilization (mostly around 0.2 - 0.4%) with rare "peaks around 2-3% when it does a little bit more. And this is maybe one of the interesting observations. It seems to periodically "flush" a bit more out, just to fall asleep again (with minimum write activity). The drive, from which the reads come (the one, where PG-s data files are ... it is the 8-disk RAID 10), has a little bit more activity (utilization 6-8%) but this data is also - concurrently - in use by some apps reading from the DB (just, normal traffic on the DB).

I don't think it'll be particularly helpful in this case, but you never
know, so: another information-collecting tool is "blktrace". This can
let you observe in detail exactly what's being done on a given block
device. It's helpful when you have weird I/O patterns and can't figure
out why, as it'll often reveal some process continually poking away at
some file it doesn't need to, thrashing away on a disk-backed mmap()ed
tempfile, or the like.

--
Craig Ringer