Skip site navigation (1) Skip section navigation (2)

Re: concurrent snapshots

From: Robert Haas <robertmhaas(at)gmail(dot)com>
To: Ants Aasma <ants(dot)aasma(at)eesti(dot)ee>
Cc: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: concurrent snapshots
Date: 2011-09-08 15:46:36
Message-ID: CA+TgmoZ_Sn7Bcp37CFKQf=WKam0B0-9F9cUtoFpLXr-Gx-oduQ@mail.gmail.com (view raw or flat)
Thread:
Lists: pgsql-hackers
On Thu, Sep 8, 2011 at 11:33 AM, Ants Aasma <ants(dot)aasma(at)eesti(dot)ee> wrote:
> On Thu, Sep 8, 2011 at 5:28 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>> On Thu, Sep 8, 2011 at 9:26 AM, Ants Aasma <ants(dot)aasma(at)eesti(dot)ee> wrote:
>>> When go try to find the new csnmin
>>> and discover that a backend has a csnmin that is too old, we go through
>>> the snapshots of that backend and convert every snapshot under the
>>> desired csnmin to a traditional snapshot.
>>
>> I thought about something along these lines (though I didn't flesh out
>> the details as much as you have here), but rejected it because the
>> step described above would require all snapshots to be stored in
>> shared memory.  That's problematic for several reasons:
>>
>> 1. A backend can have lots of snapshots, potentially requiring an
>> unbounded amount of shared memory.  We can't accommodate that.
>
> If PostgreSQL gets POSIX shared memory capability at some point in the
> future, would it be enough to accommodate snapshots in shared memory?

By itself, that particular change would not help with this problem.

>> 2. You'd need to protect all of those snapshots with spinlocks or
>> something, which would be wicked expensive, because the owning process
>> would need to take and release that spinlock every time it touched the
>> snapshot.
>
> It seems to me that converting a transactions type can be done
> lock-free. The process that does the converting needs to ensure that
> the new transaction type flag is visible before releasing any xids.
> For visibility checking, the additional cost is a read barrier, two
> volatile reads (recheck snapshot type and dense horizon) and occasional
> retry after getting a visibility result.
>
> Maybe I'm missing something. I'll take a deeper look at the snapshot
> handling code tonight to see if anything else might have any
> implications.

I'm not convinced it's anywhere near that easy.  For one thing, on at
least one big server I'm playing with, memory latency on shared memory
is vastly higher (like >10x!) than on backend-local memory due to NUMA
effects.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

In response to

Responses

pgsql-hackers by date

Next:From: Kohei KaigaiDate: 2011-09-08 15:47:20
Subject: Re: force_not_null option support for file_fdw
Previous:From: Tom LaneDate: 2011-09-08 15:46:28
Subject: Re: concurrent snapshots

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group