Re: Anyone working on better transaction locking?

On Wed, 9 Apr 2003, Ron Peacetree wrote:

> "Andrew Sullivan" <andrew(at)libertyrms(dot)info> wrote in message
> news:20030409170926(dot)GH2255(at)libertyrms(dot)info(dot)(dot)(dot)
> > On Wed, Apr 09, 2003 at 05:41:06AM +0000, Ron Peacetree wrote:
> > Nonsense. You explicitly made the MVCC comparison with Oracle, and
> > are asking for a "better" locking mechanism without providing any
> > evidence that PostgreSQL's is bad.
> >
> Just because someone else's is "better" does not mean PostgreSQL's is
> "bad", and I've never said such. As I've said, I'll get back to Tom
> and the list on this.

But you didn't identify HOW it was better. I think that's the point
being made.

> > > Please see my posts with regards to ...
> >
> > I think your other posts were similar to the one which started this
> > thread: full of mighty big pronouncements which turned out to depend
> > on a bunch of not-so-tenable assumptions.
> >
> Hmmm. Well, I don't think of algorithm analysis by the likes of
> Knuth, Sedgewick, Gonnet, and Baeza-Yates as being "not so tenable
> assumptions", but YMMV. As for "mighty pronouncements", that also
> seems a bit misleading since we are talking about quantifiable
> programming and computer science issues, not unquantifiable things
> like politics.

But the real truth is revealed when the rubber hits the pavement.
Remember that Linux Torvalds was roundly criticized for his choice of a
monolithic development model for his kernel, and was literally told that
his choice would restrict to "toy" status and that no commercial OS could
scale with a monolithic kernel.

There's no shortage of people with good ideas, just people with the skills
to implement those good ideas. If you've got a patch to apply that's been
tested to show something is faster EVERYONE here wants to see it.

If you've got a theory, no matter how well backed up by academic research,
it's still just a theory. Until someone writes to code to implement it,
the gains are theoretical, and many things that MIGHT help don't because
of the real world issues underlying your database, like I/O bandwidth or
CPU <-> memory bandwidth.

> > I'm sorry to be so cranky about this, but I get tired of having to
> > defend one of my employer's core technologies from accusations based
> > on half-truths and "everybody knows" assumptions. For instance,
> >
> Again, "accusations" is a bit strong. I thought the discussion was
> about the technical merits and costs of various features and various
> ways to implement them, particularly when this product must compete
> for installed base with other solutions. Being coldly realistic about
> what a product's strengths and weaknesses are is, again, just good
> business. Sun Tzu's comment about knowing the enemy and yourself
> seems appropriate here...

No, you're wrong. Postgresql doesn't have to compete. It doesn't have to
win. it doesn't need a marketing department. All those things are nice,
and I'm glad if it does them, but doesn't HAVE TO. Postgresql has to
work. It does that well.

Postgresql CAN compete if someone wants to put the effort into competing,
but it isn't a priority for me. Working is the priority, and if other
people aren't smart enough to test Postgresql to see if it works for them,
all the better, I keep my edge by having a near zero cost database engine,
while the competition spends money on MSSQL or Oracle.

Tom and Andrew ARE coldly realistic about the shortcomings of postgresql.
It has issues, and things that need to be fixed. It needs more coders.
It doesn't need every feature that Oracle or DB2 have. Heck some of their
"features" would be considered a mis-feature in the Postgresql world.

> > > I'll mention thread support in passing,
> >
> > there's actually a FAQ item about thread support, because in the
> > opinion of those who have looked at it, the cost is just not worth
> > the benefit. If you have evidence to the contrary (specific
> > evidence, please, for this application), and have already read all
> the
> > previous discussion of the topic, perhaps people would be interested
> in
> > opening that debate again (though I have my doubts).
> >
> Zeus had a performance ceiling roughly 3x that of Apache when Zeus
> supported threading as well as pre-forking and Apache only supported
> pre forking. The Apache folks now support both. DB2, Oracle, and SQL
> Server all use threads. Etc, etc.

Yes, and if you configured your apache server to have 20 or 30 spare
servers, in the real world, it was nearly neck and neck to Zeus, but since
Zeus cost like $3,000 a copy, it is still cheaper to just overwhelm it
with more servers running apache than to use zeus.

> That's an awful lot of very bright programmers and some serious $$
> voting that threads are worth it.

For THAT application. for what a web server does, threads can be very
useful, even useful enough to put up with the problems created by running
threads on multiple threading libs on different OSes.

Let me ask you, if Zeus scrams and crashes out, and it's installed
properly so it just comes right back up, how much data can you lose?

If Postgresql scrams and crashes out, how much data can you lost?

> Given all that, if PostgreSQL
> specific
> thread support is =not= showing itself to be a win that's an
> unexpected
> enough outcome that we should be asking hard questions as to why not.

There HAS been testing on threads in Postgresql. It has been covered to
death. The fact that you're still arguing proves you likely haven't read
the archive (google has it back to way back when, use that to look it up)
about this subject.

Threads COULD help on multi-sorted results, and a few other areas, but the
increase in performance really wasn't that great for 95% of all the cases,
and for the 5% it was, simple query planner improvements have provided far
greater performance increases.

The problem with threading is that we can either use the one process ->
many thread design, which I personally don't trust for something like a
database, or a process per backend connection which can run
multi-threaded. This scenario makes Postgresql just as stable and
reliable as it was as a multi-process app, but allows threaded performance
in certain areas of the backend that are parallelizable to run in parallel
on multi-CPU systems.

the gain, again, is minimal, and on a system with many users accessing it,
there is NO real world gain.

> At their core, threads are a context switching efficiency tweak.

Except that on the two OSes which Postgresql runs on the most, threads are
really no faster than processes. In the Linux kernel, the only real
difference is how the OS treats them, creation, destruction of threads
versus processes is virtually identical there.

> Certainly it's =possible= that threads have nothing to offer
> PostgreSQL, but IMHO it's not =probable=. Just another thing for me
> to add to my TODO heap for looking at...

It's been tested, it didn't help a lot, and it made it MUCH harder to
maintain, as threads in Linux are handled by a different lib than in say
Solaris, or Windows or any other OS. I.e. you can't guarantee the thread
lib you need will be there, and that there are no bugs. MySQL still has
thread bug issues pop up, most of which are in the thread libs themselves.