raw data into table process

From: novice <user(dot)postgresql(at)gmail(dot)com>
To: pgsql-general(at)postgresql(dot)org, pgsql-sql(at)postgresql(dot)org
Subject: raw data into table process
Date: 2007-08-22 04:36:15
Message-ID: ddcb1c340708212136x1c3a5168ya884b252fa434ed7@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-sql

I am trying to record the following entries into a table. I'm curious
to know if there's an efficient/effective way of doing this? This
entries come from an ancient datalogger (note: separated by space and
uses YY/MM/DD format to record date)

Plain file sample.dat

3665 OK BS 07/08/16 07:28
3665 CC BS 07/08/16 07:29
3665 CS BS 07/08/16 07:29
3665 CS BS 07/08/16 07:29
4532 OK BS 07/08/16 07:34
4004 OK BS 07/08/16 07:51
3991 OK BS 07/08/16 07:54

This is the table that I'm adding the entries to

CREATE TABLE maintenance
(
maintenance_id SERIAL PRIMARY KEY,
meter_id integer,
status text,
inspector text,
inspection_date timestamp with time zone,
)

-- Begin SQL Script
-- First table to dump the records in
CREATE TABLE dataload1
(data text)

-- Dump records using \copy
\copy dataload1 FROM sample.dat

-- Second table to import unique records ONLY
CREATE TABLE dataload2 AS
SELECT DISTINCT
data FROM dataload1;

-- Now I update unique records into the maintenance table
-- maintenance_id is SERIAL so it will be populated automatically
INSERT INTO maintenance(meter_id, status, inspector, inspection_date)
SELECT substr("data", 1, 4)::int
, substr("data", 8, 3)
, substr("data", 21, 2)
, (20||substr("data", 24, 2) ||'-'|| substr("data", 27, 2) ||'-'||
substr("data", 30, 2)||' '||substr("data", 33, 5))::timestamp as
inspection_date
FROM dataload2
-- So the new records will also be in timestamp order
ORDER BY inspection_date ;

-- Some housekeeping
VACUUM FULL VERBOSE ANALYZE maintenance;

-- Finally, drop the temporary tables
DROP TABLE dataload1
DROP TABLE dataload2

-- End SQL script

Any thoughts and suggestions welcome.

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Trinath Somanchi 2007-08-22 04:59:59 Pgcluster 1.7 Fail safe !!!
Previous Message Scott Marlowe 2007-08-22 01:52:17 Re: Converting non-null unique idx to pkey

Browse pgsql-sql by date

  From Date Subject
Next Message Trinath Somanchi 2007-08-22 04:59:59 Pgcluster 1.7 Fail safe !!!
Previous Message Richard Broersma Jr 2007-08-22 03:10:04 Re: SELECT syntax synopsis: column_definition?