Re: Query progress indication - an implementation

From: Simon Riggs <simon(at)2ndQuadrant(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Josh Berkus <josh(at)agliodbs(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Scara Maccai <m_lists(at)yahoo(dot)it>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: Query progress indication - an implementation
Date: 2009-06-30 08:02:42
Message-ID: 1246348962.27964.34.camel@dn-x300-willij
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Mon, 2009-06-29 at 18:49 -0400, Tom Lane wrote:
> Simon Riggs <simon(at)2ndQuadrant(dot)com> writes:
> > On Mon, 2009-06-29 at 14:07 -0400, Tom Lane wrote:
> >> I think this is pretty much nonsense --- most queries run all their plan
> >> nodes concurrently to some extent. You can't usefully say that a query
> >> is "on" some node, nor measure progress by whether some node is "done".
> > The requirement is not nonsense, even if the detail was slightly off.
> I was applying the word "nonsense" to the proposed implementation,
> not the desire to have query progress indications ...

Understood, just trying to limit the blast radius.

> > We can regard plans as acting in phases with each blocking node
> > separating the plan. We know which nodes those are, so we can report
> > that.
> [ shrug... ] You can regard them that way, but you won't get
> particularly helpful results for a large fraction of real queries.
> The system is generally set up to prefer "streaming" evaluation
> as much as it can. Even in nominally blocking nodes like Sort and Hash,
> there are operational modes that look more like streaming, or at least
> chunking.

It's not always useful, though many large queries do have multiple
phases. The concept and the name come from ETL tools and it is of real
practical use in those environments. We can put the phase number on the
EXPLAIN easily, and it is very simple to calculate the total number of
phases and the current phase - e.g. 2 of 5 phases complete.

Simon Riggs
PostgreSQL Training, Services and Support

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Lars Kanis 2009-06-30 09:07:03 Re: [PATCH] user mapping extension to pg_ident.conf
Previous Message Simon Riggs 2009-06-30 07:48:41 Re: Query progress indication - an implementation