Support for pg_receivexlog --post-segment command

From: Feike Steenbergen <feikesteenbergen(at)gmail(dot)com>
To: PostgreSQL mailing lists <pgsql-hackers(at)postgresql(dot)org>
Cc: Magnus Hagander <magnus(at)hagander(dot)net>, David Steele <david(at)pgmasters(dot)net>
Subject: Support for pg_receivexlog --post-segment command
Date: 2017-01-06 12:45:09
Message-ID: CAK_s-G0noW9OLY2sJCbPB7W1qVp9F9QbWFGErfxpe_vp7zh9Og@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi all,

When reading through "Support for pg_receivexlog --format=plain|tar"[1], I
came across a notion from Magnus Hagander that has crossed my mind a few
times as well in the past years. As the feature proposed here is not
directly related to that thread, I thought it best to start a new thread to
discuss.

> I have been talking to David about it a couple of times, and he agreed
that it'd be useful to have a post-segment command. We haven't discussed it
in much detail though. I'll add him to direct-cc here to see if he has any
further input :)

I'm coming across a few usecases where this would seem very useful. I'm
looking to push finished segments to some api as soon as possible. Having
the post-segment command would allow me to get there. I've tried the
following approaches, none of them are very satisfying however:

- periodic checking of new files (cron)
- using inotify
- tailing verbose pg_receivexlog output to see when a segment was switched

> It could be that the best idea is to just notify some other process of
what's happening. But making it an external command would give that a lot
of flexibility. Of course, we need to be careful not to put ourselves back
in the position we are in with archive_command, in that it's very difficult
to write a good one.

A signal for would be good enough for my use case, I don't necessarily
need all the bookkeeping to ensure the post-segment command was finished
successfully. However I can see people expecting similar behaviour for the
post-segment command as for the archive_command. If we would use an
external command, does this also imply that we need some bookkeeping around
which segments are ready and done, similar to what is done on the server in
the archive_status directory?

Thanks,

Feike

[1]
https://www.postgresql.org/message-id/CAB7nPqTEVXjtH+fehcCbP791H71cfLN_p9rrd-h=yMJfshZQWQ@mail.gmail.com

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Robert Haas 2017-01-06 12:47:07 Re: [COMMITTERS] pgsql: Fix possible crash reading pg_stat_activity.
Previous Message Ashutosh Bapat 2017-01-06 12:26:20 Re: Potential data loss of 2PC files