Skip site navigation (1) Skip section navigation (2)

Re: NDirectFileRead and Write

From: ITAGAKI Takahiro <itagaki(dot)takahiro(at)oss(dot)ntt(dot)co(dot)jp>
To: pgsql-hackers(at)postgresql(dot)org
Subject: Re: NDirectFileRead and Write
Date: 2008-08-05 06:26:08
Message-ID: 20080805150620.A197.52131E4D@oss.ntt.co.jp (view raw or flat)
Thread:
Lists: pgsql-hackers
Here is a patch to user NDirectFileRead/Write counters to get I/O counts
in BufFile module. We can see the counters when log_statement_stats is on.

The information is different from trace_sort; trace_sort shows used blocks
in external sort, and log_statement_stats shows how many I/Os are submitted
during sorts.

I wrote:
> I'd like to use NDirectFileRead and NDirectFileWrite statistics counters
> for counting reads and writes in BufFile. They are defined, but not used
> now. BufFile is used for tuple sorting or materializing, so we could use
> NDirectFileRead/Write to retrieve how many I/Os are done in temp tablespace.

=# SET client_min_messages = log;
=# SET trace_sort = on;
=# SET log_statement_stats = on;
=# EXPLAIN ANALYZE SELECT * FROM generate_series(1, 1000000) AS i ORDER BY i;
LOG:  begin tuple sort: nkeys = 1, workMem = 1024, randomAccess = f
LOG:  switching to external sort with 7 tapes: CPU 0.09s/0.26u sec elapsed 0.35 sec
LOG:  performsort starting: CPU 0.48s/1.68u sec elapsed 2.20 sec
LOG:  finished writing final run 1 to tape 0: CPU 0.48s/1.70u sec elapsed 2.21 sec
LOG:  performsort done: CPU 0.48s/1.70u sec elapsed 2.21 sec
LOG:  external sort ended, 2444 disk blocks used: CPU 0.79s/2.23u sec elapsed 3.06 sec
LOG:  QUERY STATISTICS
DETAIL:  ! system usage stats:
!       3.078000 elapsed 2.234375 user 0.812500 system sec
!       [3.328125 user 1.281250 sys total]
! buffer usage stats:
!       Shared blocks:          0 read,          0 written, buffer hit rate = 0.00%
!       Local  blocks:          0 read,          0 written, buffer hit rate = 0.00%
!       Direct blocks:       5375 read,       5374 written
                                                             QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------
 Sort  (cost=62.33..64.83 rows=1000 width=4) (actual time=2221.485..2743.831 rows=1000000 loops=1)
   Sort Key: i
   Sort Method:  external sort  Disk: 19552kB
   ->  Function Scan on generate_series i  (cost=0.00..12.50 rows=1000 width=4) (actual time=349.065..892.907 rows=1000000 loops=1)
 Total runtime: 3087.305 ms
(5 rows)

Regards,
---
ITAGAKI Takahiro
NTT Open Source Software Center


Attachment: NDirectFileReadWrite.patch
Description: application/octet-stream (3.0 KB)

In response to

Responses

pgsql-hackers by date

Next:From: Heikki LinnakangasDate: 2008-08-05 06:37:04
Subject: Re: Mini improvement: statement_cost_limit
Previous:From: David FetterDate: 2008-08-05 06:21:06
Subject: PL/LOLCODE [was Re: [PATCH] "\ef <function>" in psql]

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group