Skip site navigation (1) Skip section navigation (2)

Re: import function's progress report

From: "Dave Page" <dpage(at)pgadmin(dot)org>
To: "Quan Zongliang" <quanzongliang(at)gmail(dot)com>
Cc: pgadmin-hackers(at)postgresql(dot)org
Subject: Re: import function's progress report
Date: 2008-12-31 10:24:56
Message-ID: 937d27e10812310224q2c4e2353t23495cdf8fb3c330@mail.gmail.com (view raw or flat)
Thread:
Lists: pgadmin-hackers
Hi

On Wed, Dec 31, 2008 at 7:14 AM, Quan Zongliang <quanzongliang(at)gmail(dot)com> wrote:
> Hi, all
>
> This is a progress report about import function.
> A screenshot with dialog test mode (/t) attached.
> Only the file page's code is finished.
>
> 1. Target page:
> Now it is blank.
> Reserved for standalone version to select target database and table.

OK. Note that the different pages should be on a wxNotebook - don't
use buttons to control the UI (we've - OK, I've - made that mistake
before).

> 2. Source pages: file / Source / ODBC
> Import from csv format file / other pgsql database / ODBC(for MS-Windows)
> First, csv file will be supported. Remainders only in plan.

The different sources should be on a single 'Source' tab. At the top
of that tab, use a radio button set to allow the user to select the
source type, and have that swap in/out framed sets of controls
underneath. For a very basic example of this, look at the Report form
which changes the stylesheet options based on whether you select HTML
or XML output.

> 3. Columns page:
> Set data format ( date & number), space trim and column's order
> I plan to support express, user can input a pgsql express to treat data.

Hmm, pgsql expressions could be slow as they can only be evaluated in
insert statements, which is really not good for such a tool. I would
suggest:

- Consider embedding Python (which is on the TODO anyway) and allowing
transforms to be scripted on the client.
- Convert the import data to a COPY compatible format so it can be fed
directly to the server without using INSERT which is *much* slower.

> 4. Options/Status/Others page:
> Only some ideas:
>  Skip empty rows
>  Import rows match regex
>  Don't import rows match regex
>  Commit per (n) rows.
>  Some log options
>  Test insert with (n) rows before insert to DB
>  Generate insert statement to a file.
>  Stop when (n) errors
> and so on.

Sounds good :-)

> Limited encoding can be Support, see attached file "encodinglist.txt".
> ASCII, Japanese EUCJP/Shift_JIS and Chinese GB2312/Big5 had been tested.

Cool :-)


-- 
Dave Page
EnterpriseDB UK:   http://www.enterprisedb.com

In response to

Responses

pgadmin-hackers by date

Next:From: Magnus HaganderDate: 2008-12-31 10:58:12
Subject: Re: import function's progress report
Previous:From: Dave PageDate: 2008-12-31 10:06:03
Subject: Re: Patch to handle connection limit for roles

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group