Re: One process per session lack of sharing

On Sun, Jul 17, 2016 at 9:28 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:

> On Sun, Jul 17, 2016 at 3:12 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> > Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> >> On Fri, Jul 15, 2016 at 4:28 AM, Craig Ringer <craig(at)2ndquadrant(dot)com>
> wrote:
> >>> I don't think anyone's considering moving from multi-processing to
> >>> multi-threading in PostgreSQL. I really, really like the protection
> that the
> >>> shared-nothing-by-default process model gives us, among other things.
> >
> >> We get some very important protection by having the postmaster in a
> >> separate address space from the user processes, but separating the
> >> other backends from each other has no value.
> >
> > I do not accept that proposition in the least. For one thing, debugging
> > becomes an order of magnitude harder when you've got multiple threads
> > in the same address space: you have essentially zero guarantees about
> > what one thread might have done to the supposedly-private state of
> > another one.
>
> Well, that's true, in theory. In practice, random memory clobbers are
> a pretty rare type of bug. The chances that thread A crashed because
> thread B overwrote its supposedly-private state are just not very
> high. Also, such bugs are extremely hard to troubleshoot even when
> there is only process and one thread involved, so it's not like things
> are a rose garden today. I don't buy the argument that it's worth
> giving up arbitrary amounts of performance and functionality for this.
>

The random memory clobbers are partially rare because they often aren't
triggered. Many of them are of the dangling pointer type and in a single
threaded process, there is less of a chance to allocate and overwrite the
free'd and then used memory.

But you are right, all of them are tricky to hunt and I remember using
hardware watch points and what not in the past. They make your day
though when you finally find them. So you will be sorry when the last
one is hunted down.

>
> >> ... enough other people have
> >> written complex, long-running multithreaded programs that I think it
> >> is probably possible to do so without unduly compromising reliability.
> >
> > I would bet that every single successful project of that sort has been
> > written with threading in mind from the get-go. Trying to retro-fit
> > threading onto thirty years' worth of single-threaded coding is a recipe
> > for breaking your project; even if you had control of all the code
> running
> > in the address space, which we assuredly do not.
>
> I admit that it is risky, but I think there are things that could be
> done to limit the risk. I don't believe we can indefinitely continue
> to ignore the potential performance benefits of making a switch like
> this. Breaking a thirty-year old code base irretrievably would be
> sad, but letting it fade into irrelevance because we're not willing to
> make the architecture changes that are needed to remain relevant would
> be sad, too.
>

I have to agree with Robert on that one. We have been "thinking" about
multi-threading some 16 years ago already. We were aware of the dangers
and yet at least considered doing it some day for things like a parallel
executor. And that would probably be our best bang for the buck still.

The risks of jamming all sessions into a single, multi-threaded process are
huge. Think of snapshot visibility together with catalog cache
invalidations.
I'd say no to that one as a first step.

But multi-threading the executor or even certain utility commands at first
should not be rejected purely on the notion that "we don't have
multithreading
today."

Regards, Jan

>
> --
> Robert Haas
> EnterpriseDB: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers
>

--
Jan Wieck
Senior Postgres Architect
http://pgblog.wi3ck.info