Re: [PATCH] COPY .. COMPRESSED

On Wed, Jan 16, 2013 at 5:15 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> On Tue, Jan 15, 2013 at 3:37 PM, Christopher Browne <cbbrowne(at)gmail(dot)com> wrote:
>> That points towards a fix that involves having a set of non-arbitrary commands
>> that we allow plain users to use.
>>
>> Hmm. There's an interesting thought...
>>
>> How about having a "pg_filters" table in pg_catalog which allows capturing
>> labels and names of known-to-be-safe binary filters:
>>
>> insert into pg_filters (label, location)
>> values
>> ('zcat', '/usr/bin/zcat'),
>> ('bzip2', '/usr/bin/bzip2'),
>> ('bunzip2', '/usr/bin/bunzip2');
>>
>> And then having some capability to grant permissions to roles to use
>> these filters.
>
> I suspect that's going to be less efficient than using a compression
> library that's linked into the backend, because you have to copy all
> the data through the kernel to another process and back. And it's
> certainly a lot more complex.

More complex, certainly.

By spawning a separate process, we'd get benefit of multicore CPUs,
so I'm not sure I agree that it's necessarily slower.

> At any rate, I think it would be good to avoid letting our desire for
> infinite flexibility get in the way of doing something useful.

Oh, agreed. I was actively thinking of the cooler bits of this pointing more
towards 9.4 than 9.3.
--
When confronted by a difficult problem, solve it by reducing it to the
question, "How would the Lone Ranger handle this?"