tuplestore potential performance problem

From: "Hitoshi Harada" <umi(dot)tanuki(at)gmail(dot)com>
To: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: tuplestore potential performance problem
Date: 2008-12-03 14:15:50
Message-ID: e08cc0400812030615rd5c6462we46ac362628c21c6@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

While attacking this issue(*1), I found that tuplestore that is on the
file status has potential performance problem.

The performance problem introduced by Heikki's new approach was caused
by BufFile's frequent flush out in such cases like you put a new row
into it and read middle row of it then put another row again, and so
on. When tuplestore switches its internal mode from TSS_WRITEFILE to
TSS_READFILE, underlying BufFile seeks to read pointer and flushes out
its dirty buffer if the reading pointer is not near the writing
pointer. Also, reading to writing switch avoids OS disk cache benefit.

This is not critical in TSS_INMEM.

So I decided to keep writing until finish if the tuplestore gets in
file mode from memory mode rather than switching reading and writing
randomly, which recovers the earlier performance almost. I am not sure
but am afraid that the nodeCtescan also uses similar logic. Doesn't
CTE have any problem for large data set?



Hitoshi Harada


Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2008-12-03 14:30:55 Re: tuplestore potential performance problem
Previous Message Tom Lane 2008-12-03 14:12:43 Re: snapshot leak and core dump with serializable transactions