Re: Re: new set of psql patches for loading (saving) data from (to) text, binary files

From: Andres Freund <andres(at)anarazel(dot)de>
To: Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
Cc: Stephen Frost <sfrost(at)snowman(dot)net>, Jason O'Donnell <odonnelljp01(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Re: new set of psql patches for loading (saving) data from (to) text, binary files
Date: 2017-04-06 01:02:42
Message-ID: 20170406010242.gmbvfotggbcx45pr@alap3.anarazel.de
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 2017-03-18 17:51:48 +0100, Pavel Stehule wrote:
> What is done:
>
> create table foo foo(a bytea);
>
> -- import
> insert into foo values($1)
> \gloadfrom ~/xxx.jpg bytea
>
> -- export
> \pset format binary
> select a from foo
> \g ~/xxx2.jpg
>
> tested on import 55MB binary file
>
> Comments, notes?

I don't like the API here much. Loading requires knowledge of some
magic $1 value and allows only a single column, printing doesn't mean
much when there's multiple columns / rows.

I think the loading side of things should be redesigned into a more
general facility for providing query parameters. E.g. something like
\setparam $1 'whateva'
\setparamfromfile $2 'somefile'
\setparamfromprogram $3 cat /frakbar

which then would get used in the next query sent to the server. That'd
allow importing multiple columns, and it'd be useful for other purposes
than just loading binary data.

I don't yet have a good idea how to deal with moving individual cells
into files, so they can be loaded. One approach would be to to have
something like

\storequeryresult filename_template.%row.%column

which'd then print the current query buffer into the relevant file after
doing replacement on %row and %column.

I don't think we can find an API we agree upon in the next 48h...

- Andres

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message David Rowley 2017-04-06 01:05:24 Re: [COMMITTERS] pgsql: Collect and use multi-column dependency stats
Previous Message Tatsuo Ishii 2017-04-06 00:36:50 Re: pgbench - allow to store select results into variables