Re: TABLESPACE and directory for Foreign tables?

On 05/05/2014 10:53 AM, Tom Lane wrote:
> Josh Berkus <josh(at)agliodbs(dot)com> writes:
>> I'm working with the cstore_fdw project, which has an interesting
>> property for an FDW: the FDW itself creates the files which make up the
>> database. This raises a couple of questions:
>
>> 1) Do we want to establish a standard directory for FDWs which create
>> files, such as $PGDATA/base/{database-oid}/fdw/ ? Or would we want to
>> leave it up to each FDW to decide?
>
> I think we ought to vigorously discourage FDWs from storing any files
> inside $PGDATA. This cannot lead to anything except grief. Just for
> starters, what will operations such as pg_basebackup do with them?

That was one advantage to putting them in PGDATA; you get a copy of the
files with pg_basebackup. Of course, they don't replicate after that,
but they potentially could, in the future, with Logical Streaming
Replication.

> A larger and more philosophical point is that such a direction of
> development could hardly be called a "foreign" data wrapper. People
> would expect Postgres to take full responsibility for such files,
> including data integrity considerations such as fsync-at-checkpoints
> and WAL support. Even if we wanted the FDW abstractions to allow
> for that, we're very far away from it. And frankly I'd maintain
> that FDW is the wrong abstraction.

Certainly pluggable storage would be a better abstraction; but we don't
have that yet. In the meantime, we have one FDW which creates files
*right now*, and we might have more in the future, so I'm trying to
establish some guidelines as to how such FDWs should behave. Regardless
of whether or not you think FDWs should be managing files, it's better
for users if all FDWs which manage files manage them in the same way.

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com