I had this flagged as needing a response, but it fell through the
cracks yesterday. Apologies for the delayed response.
Markus Wanner <markus(at)bluegap(dot)ch> wrote:
> I'm not clear if Kevin plans to go down to tuple level locking
> with granularity of the SIREAD thing.
Eventually, where possible, subject to escalation.
> (I don't like calling it a lock, because it actually isn't. Let's
> better call it a hint or a mark).
You have a point, but as I said in another email, I'm reluctant to
invent my own terminology which conflicts with the literature. So
I'll remain "conventional" on this.
> We certainly cannot store all of the SIREAD hint information
> within the tuple header, as there may be multiple transactions
> having marked the same tuple SIREAD, but we don't want to spend
> lots of bits for SSI there (rather no single bit).
Absolutely.
> It's easy to see that the SIREAD hint information doesn't fit in
> shared memory for large enough tables. The only way I can imagine
> tuple-level SIREAD locking half-ways working is by using that for
> transactions reading only a few tuples and fall back to some
> coarser grained locking strategy after a certain limit (per
> transaction).
Exactly.
> I'm not sure if I understand this correctly, but I don't think you
> need to "lock" tuples that don't exist, at least not with SIREAD.
> The Cahill paper covers this under "Detecting Phantoms" and
> proposes to use plain predicate locking to cover tuple level
> granularity AFAIUI.
>
> The proposed SIREAD hint seems to be an optimization that only
> works for existing tuples. I don't find it hard to believe that it
> performs better than a general purpose predicate locking strategy.
> (Especially for test cases with short transactions, i.e. only few
> SIREAD locks per txn).
When the Cahill paper talks about predicate locking, it *is* talking
about what to lock with SIREAD locks, at least as I read it. If you
see something to indicate otherwise, please share. (If I'm off base
on understanding any of this, I'd sure like to get straightened out
as soon as possible.)
> (It's interesting that with "database page" level granularity, he
> states that predicate locking would not be necessary. Instead any
> page can be locked at any time. For this to work, according to my
> reasoning, you'd have to know in advance on which page potentially
> accessible (but not yet visible) new tuples would end up. This is
> close to impossible for Postgres, however, it seems to work for
> Berkeley DB).
It was the way Sybase worked for ages, too. If you consider that
you're locking both the heap and index pages read, it becomes clear
that in order to add a row which conflicts with a predicate, you
have to modify a page which would have been read to check the
predicate.
> How about storing the SIREAD info in shared memory and using
> dynamic granularity based on the conflict rate and available
> memory? *duck*
Don't duck. That's the plan. I really wish I was better at
communicating these things. :-( See lock.c and the associated
README and see if you don't think they would fit the bill.
> As this seems to be an optimization of predicate locking, don't we
> need to implement that first?
Exactly. That's the plan. Implement it in the simplest form, then
optimize.
-Kevin