Re: pg_dump --snapshot

* Andres Freund (andres(at)2ndquadrant(dot)com) wrote:
> > It helps in that once we have the lock, things aren't changing under us.
> > The closer we can keep that to when the transaction starts, the better..
>
> If you look at my example the timing where we take the snapshot isn't
> the problem. While we wait for a lock on one object the not-yet-locked
> objects can still change, get dropped et al. That window is so big that
> the timing around the snapshot acquiration and trying to get the first
> lock is insignificant.

I wasn't talking about just getting the first lock but rather all the
locks.. I agree that we can get stuck behind something else which is
already holding a lock and that's certainly a problem.

> Remember this is only a problem *if* there is
> concurrent DDL. And in that case we very, very likely will have access
> exlusive locks and thus will wait for them and have the described
> problem.

Yes, I understand that issue.

> I don't think thats entirely possible. Think e.g. of constraints,
> determination of HOTability, ... All those need to look at the database
> state as its valid now, not as it was valid back when our snapshot
> started.

There will certainly still need to be parts of the system which are
using SnapshotNow, yes. Catalog MVCC is a rather massive discussion
which isn't going to be solved here.

> In the logical decoding patches (submitted previously, will get
> resubmitted when 9.3 is getting somewhere), we export a snapshot once we
> have read enough WAL that all changes from that point henceforth can be
> streamed out. That snapshot is obviously very useful to get a new node
> up and running.

At that point, you take a snapshot and want to export it for pg_dump to
use to get the same view of the DB and then dump/restore it into a new
database where you'll then start applying changes from the logical
replication..? Interesting, but certainly also quite a specialized
use-case, no? How do you plan to handle the locking concerns which have
been raised during this discussion? Do you take locks out before
creating the snapshot to pass to pg_dump?

Thanks,

Stephen