Re: Import from CSV - Questions

> > Umm. That would kind of assume it's written down in a
> comprehensible
> > way. Which it isn't, of course :-)
> >
> > The general ideas so far have been, off the top of my head:
> >
> > * Pluginnable set of "readers" and "writers". Originally I'd see
> > postgresql, odbc, xml and possibly csv. Pg driver would be
> optimised
> > to use COPY when available.
> >
> > * Pluginnable set of "transforms" that would operate on the
> rows. By
> > default things like copy and concatenate and maybe regexp. Future
> > enhancement would be a python extension, as Dave mentioned. (Or
> > really, anything else)
> >
> > * I was envisioning a split of say "package", "job", "step"
> (terms of
> > course subject to discussion). package basically a set of
> job, job a
> > set of steps. Things like connections would be defined at the "job"
> > level, along wiht parmaeters for transaction control etc.
> (So you can
> > use it to transfer 10 different tables within a single transaction,
> > something I need all the time).
> >
> > * I'd like to see the job format stored as XML with a well defined
> > schema, so different appliations can generate it - both manually
> > (GUI-wise from pgadmin and phppgadmin etc) and automatically.
> >
> > * The "engine" should be available both as a commandline
> tool (which
> > must not require X libraries etc, because it should be deployable
> > "everywhere") and as acommand inside pgadmin (like MS DTS)
> >
> >
> > Um. I think that's about it. I had some sketches of classes and
> > interfaces around (not complete, but an idea), but I can't
> find them
> > :(
> >
> >
> This sounds like an awful lot of work.
> A somewhat reduced version (IIRC Dave and me discussed
> something like the following briefly) a more raw import
> (maybe into temp tables) could be a big step, giving the
> admin the chance to create views on that tables that do the
> extractions he likes. PostgreSQL already has all functions
> you'd like, no need to reimplement them.

it is. That's why I wanted to do that first, but I wanted ot have some
sort of generic framework ready first so it could be expanded on later.
I'm definitly not saying it should have all that from the beginning :-)

//Magnus