Re: could not create directory "...": File exists

From: Stephen Frost <sfrost(at)snowman(dot)net>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: could not create directory "...": File exists
Date: 2013-01-17 17:37:00
Message-ID: 20130117173700.GS16126@tamriel.snowman.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

* Tom Lane (tgl(at)sss(dot)pgh(dot)pa(dot)us) wrote:
> Ugh. Still another problem with non-MVCC catalog scans.

Indeed.

> It seems that the only thing we actually use from each tuple is the OID.

Yes, that's true.

> So there are other ways to fix it, of which probably the minimum-change
> one is to keep a list of already-processed tablespace OIDs and skip a
> tuple if we get a match in the list. This would be O(N^2) in the number
> of tablespaces, but I rather doubt that's a problem.

I did consider this option also but it felt like a lot of additional
code to write and I wasn't entirely convinced it'd be worth it. It's
very frustrating that we can't simply get a list of *currently valid*
tablespaces with a guarantee that no one else is going to mess with that
list while we process it. That's what MVCC would give us, but we can't
rely on that yet..

If we agree that keeping a list and then skipping over anything on the
list already seen, I can go ahead and code that up.

> [ thinks for a bit... ] Ugh, no, because the *other* risk you've got
> here is not seeing a row at all, which would be really bad.

I'm not sure that I see how that could happen..? I agree that it'd be
really bad if it did though. Or are you thinking if we were to take a
snapshot and then walk the table?

> I'm not sure that transiently increasing the lock here is enough,
> either. The concurrent transactions you're worried about probably
> aren't holding their locks till commit, so you could get this lock
> and still see tuples with unstable committed-good states.

If there are other transactiosn which have open locks against the table,
wouldn't that prevent my being able to acquire ShareLock on it? Or put
another way, how would they not hold their locks till commit or
rollback? We do only look at tablespaces which exist for the database
we're copying, as I recall, so any newly created tablespaces shouldn't
impact this.

Thanks,

Stephen

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Fujii Masao 2013-01-17 18:05:47 Re: Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave
Previous Message Andres Freund 2013-01-17 17:35:20 Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave