Re: Proposal for CSN based snapshots

On 08/10/2016 05:51 PM, Tom Lane wrote:
> Heikki Linnakangas <hlinnaka(at)iki(dot)fi> writes:
>> On 08/10/2016 05:09 PM, Tom Lane wrote:
>>> Uh, what? That's not the semantics we have today, and I don't see why
>>> it's necessary or a good idea. Once the commit is in the WAL stream,
>>> any action taken on the basis of seeing the commit must be later in
>>> the WAL stream. So what's the problem?
>
>> I was talking about synchronous commits in the above. A synchronous
>> commit is not made visible to other transactions, until the commit WAL
>> record is flushed to disk.
>
> [ thinks for a bit... ] Oh, OK, that's because we don't treat a
> transaction as committed until its clog bit is set *and* it's not
> marked as running in the PGPROC array. And sync transactions will
> flush WAL in between.

Right.

> Still, having to invent CSNs seems like a huge loss for this design.
> Personally I'd give up async commit first. If we had only sync commit,
> the rule could be "xact LSN less than snapshot threshold and less than
> WAL flush position", and we'd not need CSNs. I know some people like
> async commit, but it's there because it was easy and cheap in our old
> design, not because it's the world's greatest feature and worth giving
> up performance for.

I don't think that's a very popular opinion (I disagree, for one).
Asynchronous commits are a huge performance boost for some applications.
The alternative is fsync=off, and I don't want to see more people doing
that. SSDs have made the penalty of an fsync much smaller, but it's
still there.

Hmm. There's one more possible way this could all work. Let's have CSN
== LSN, also for asynchronous commits. A snapshot is the current insert
position, but also make note of the current flush position, when you
take a snapshot. Now, when you use the snapshot, if you ever see an XID
that committed between the snapshot's insert position and the flush
position, wait for the WAL to be flushed up to the snapshot's insert
position at that point. With that scheme, an asynchronous commit could
return to the application without waiting for a flush, but if someone
actually looks at the changes the transaction made, then that
transaction would have to wait. Furthermore, we could probably skip that
waiting too, if the reading transaction is also using
synchronous_commit=off.

That's slightly different from the current behaviour. A transaction that
runs with synchronous_commit=on, and reads data that was modified by an
asynchronous transaction, would take a hit. But I think that would be
acceptable.

- Heikki