Re: [PATCH] COPY .. COMPRESSED

From: Stephen Frost <sfrost(at)snowman(dot)net>
To: Christopher Browne <cbbrowne(at)gmail(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: [PATCH] COPY .. COMPRESSED
Date: 2013-01-15 21:20:46
Message-ID: 20130115212046.GV16126@tamriel.snowman.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

* Christopher Browne (cbbrowne(at)gmail(dot)com) wrote:
> How about having a "pg_filters" table in pg_catalog which allows capturing
> labels and names of known-to-be-safe binary filters:

I was considering that (though I was thinking they'd be
"transformations" rather than filters; filter implies that you're
removing something, imv), but as I mentioned upthread, there are dangers
in that direction and having a default set of options strikes me as a
lot more challenging to provide.

> insert into pg_filters (label, location)
> values
> ('zcat', '/usr/bin/zcat'),
> ('bzip2', '/usr/bin/bzip2'),
> ('bunzip2', '/usr/bin/bunzip2');

We'd need to include which direction is supported also, I think.

> And then having some capability to grant permissions to roles to use
> these filters.

Yes, an additional ACL system, as I mentioned upthread, would be
required for this.

> That's not a "version 1" capability... Suppose we have, in 9.3, that there are
> direct references to "|/usr/bin/zcat" (and such), and then hope, in
> 9.4, to tease
> this out to be a non-superuser-capable facility via the above pg_filters?

It would be good to flush out what the syntax, etc, would look like for
this, if we're going to support it, before we go down a road that limits
us in what we can do. For example, if we implement the existing popen
call, and then later want to allow non-superusers to use certain
filters, how would the non-superuser specify the filter? I really don't
think we want to be taking the shell-like command provided by a
non-superuser and then try to match that against a list of commands in a
table..

> These filters should be useful for FDWs as well as for COPY.

I'm not sure I see how any FDW beyond file_fdw would really benefit from
this..? I don't think a MySQL FDW or Reddis FDW would gain anything...

Thanks,

Stephen

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Stephen Frost 2013-01-15 21:22:48 Re: [PATCH] COPY .. COMPRESSED
Previous Message Robert Haas 2013-01-15 20:56:04 Re: unlogged tables vs. GIST