Re: Regression testing for psql

From: Stephen Frost <sfrost(at)snowman(dot)net>
To: alvherre <alvherre(at)commandprompt(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Regression testing for psql
Date: 2010-05-26 23:57:26
Message-ID: 20100526235726.GF21875@tamriel.snowman.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

* alvherre (alvherre(at)commandprompt(dot)com) wrote:
> Excerpts from Stephen Frost's message of mié may 26 15:19:59 -0400 2010:
> > * Robert Haas (robertmhaas(at)gmail(dot)com) wrote:
> > > Then, too, there's the fact that many of these tests fail on my
> > > machine because my username is not sfrost,
> >
> > I've updated the patch to address this, it's again at:
> > http://snowman.net/~sfrost/psql-regress-help.patch
>
> Isn't this kind of test a pain to maintain? If somebody add a new SQL
> command, it will affect the entire \h output and she'll have to either
> apply the changes without checking them, or manually check the complete
> list. I have only to add a new function to make the test fail ...

Erm, last I checked, we already provide a diff output for the changes to
the regression output. That doesn't help when you add a new column to a
catalog, but that's a far cry from just adding some new SQL syntax or
adding a function.

> Also, having to exclude tests that mention the database owner means that
> you're only testing a fraction of the commands, so any possible problem
> has a large chance of going undetected. I mean, if we're going to test
> this kind of thing, shouldn't we be using something that allows us to
> ignore the db owner name? A simple wildcard in place of the owner name
> would suffice ... or do we need a regex for some other reason?

Yes, being able to use a regexp or something would be nice, but we don't
have those mechanics in the regression tests now (unless I missed
something).

> The \h output normally depends on terminal width. Have you handled that
> somehow?

I don't think that'll actually be an issue, but would love to hear from
people if it is.

> (And if we want something like this, I think we should not have a single
> huge file for the complete test, but a set of smaller files. I'd even
> put the bunch in src/bin/psql/regress rather than the main regress dir.)

The actual set of tests is rather small. The output is large, but
that's just because we have alot of things in the catalog.

Thanks,

Stephen

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2010-05-26 23:58:52 Re: Distclean does not remove gram.c
Previous Message Gurjeet Singh 2010-05-26 23:54:39 Distclean does not remove gram.c