Re: multithreaded zstd backup compression for client and server

On Wed, Mar 23, 2022 at 04:34:04PM -0400, Robert Haas wrote:
> be, spawning threads inside the PostgreSQL backend. Short of cats and
> dogs living together, it's hard to think of anything more terrifying,
> because the PostgreSQL backend is very much not thread-safe. However,
> a lot of the things we usually worry about when people make noises
> about using threads in the backend don't apply here, because the
> threads are hidden away behind libzstd interfaces and can't execute
> any PostgreSQL code. Therefore, I think it might be safe to just ...
> turn this on. One reason I think that is that this whole approach was
> recommended to me by Andres ... but that's not to say that there
> couldn't be problems. I worry a bit that the mere presence of threads
> could in some way mess things up, but I don't know what the mechanism
> for that would be, and I don't want to postpone shipping useful
> features based on nebulous fears.

Note that the PGDG .RPMs and .DEBs are already linked with pthread, via
libxml => liblzma.

$ ldd /usr/pgsql-14/bin/postgres |grep xm
libxml2.so.2 => /lib64/libxml2.so.2 (0x00007faab984e000)
$ objdump -p /lib64/libxml2.so.2 |grep NEED
NEEDED libdl.so.2
NEEDED libz.so.1
NEEDED liblzma.so.5
NEEDED libm.so.6
NEEDED libc.so.6
VERNEED 0x0000000000019218
VERNEEDNUM 0x0000000000000005
$ objdump -p /lib64/liblzma.so.5 |grep NEED
NEEDED libpthread.so.0

Did you try this on windows at all ? It's probably no surprise that zstd
implements threading differently there.