Re: pg_streamrecv for 9.1?

Magnus Hagander <magnus(at)hagander(dot)net> writes:
>>> Would people be interested in putting pg_streamrecv
>>> (http://github.com/mhagander/pg_streamrecv) in bin/ or contrib/ for
>>> 9.1? I think it would make sense to do so.

+1 for having that in core, only available for the roles WITH
REPLICATION I suppose?

>> I think that the base backup feature is more important than simple streaming
>> chunks of the WAL (SR already does this). Talking about the base backup over
>> libpq, it is something we should implement to fulfill people's desire that
>> claim an easy replication setup.
>
> Yes, definitely. But that also needs server side support.

Yeah, but it's already in core for 9.1, we have pg_read_binary_file()
there. We could propose a contrib module for previous version
implementing the function in C, that should be pretty easy to code.

The only reason I didn't do that for pg_basebackup is that I wanted a
self-contained python script, so that offering a public git repo is
all I needed as far as distributing the tool goes.

> Yeah, the WIP patch heikki posted is simliar, except it uses tar
> format and is implemented natively in the backend with no need for
> pl/pythonu to be installed.

As of HEAD the dependency on pl/whatever is easily removed.

The included C tool would need to have a parallel option from the get-go
if at all possible, but if you look at the pg_basebackup prototype, it
would be good to drop the wrong pg_xlog support in there and rely on a
proper archiving setup on the master.

Do you want to work on internal archive and restore commands over libpq
in the same effort too? I think this tool should be either a one time
client or a daemon with support for:

- running a base backup when receiving a signal
- continuous WAL streaming from a master
- accepting standby connections and streaming to them
- one-time libpq "streaming" of a WAL file, either way

Maybe we don't need to daemonize the tool from the get-go, but if you're
going parallel for the base-backup case you're almost there, aren't you?
Also having internal commands for archive and restore commands that rely
on this daemon running would be great too.

I'd offer more help if it wasn't for finishing the extension patches,
I'm currently working on 'alter extension ... upgrade', including how to
upgrade from pre-9.1 extensions. But if that flies quicker than I want,
count me in for more than only user specs :)

Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support