Re: Bump soft open file limit (RLIMIT_NOFILE) to hard limit on startup

On Tue Feb 11, 2025 at 8:42 PM CET, Andres Freund wrote:
> Not sure that's quite the right thing to do for postmaster. What I'd start
> with is to increase the soft limit to
> "already used files" + max_files_per_process.

I needed to rebase this patch, and that made me finally take the time to
do the restoration of the file limit for subprocesses properly: In
previous versions of this patch it restored the limit before the call to
system() and it didn't restore it at all for popen. This latest version
the patch adds custom pg_system() and pg_popen() functions that restore
the limits in the child process right after the fork, but before the
exec.

There are two reasons to do this:
1. Any executables that still use select(2) will get clear "out of file
descriptors" errors instead of failing in mysterious ways.
2. Future looking when we'll have multi-threading (which this change is
needed for) it would be problematic to restore the original limit
temporarily in the postgres process tree. Some other thread might
want to open a file while the limit is too low. By only calling
setrlimit with the lower value in the child process there's not a
single moment where the original Postgres process has a too low limit.