Re: File descriptors inherited by restore_command

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: David Steele <david(at)pgmasters(dot)net>
Cc: Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: File descriptors inherited by restore_command
Date: 2019-06-21 13:45:32
Message-ID: 28722.1561124732@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

David Steele <david(at)pgmasters(dot)net> writes:
> While investigating "Too many open files" errors reported in our
> parallel restore_command I noticed that the restore_command can inherit
> quite a lot of fds from the recovery process. This limits the number of
> fds available in the restore_command depending on the setting of system
> nofile and Postgres max_files_per_process.

Hm. Presumably you could hit the same issue with things like COPY FROM
PROGRAM. And the only reason the archiver doesn't hit it is it never
opens many files to begin with.

> I was wondering if we should consider closing these fds before calling
> restore_command? It seems like we could do this by forking first or by
> setting FD_CLOEXEC using fcntl() or O_CLOEXEC on open() where available.

+1 for using O_CLOEXEC on machines that have it. I don't think I want to
jump through hoops for machines that don't have it --- POSIX has required
it for some time, so there should be few machines in that category.

regards, tom lane

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Dagfinn Ilmari Mannsåker 2019-06-21 13:45:47 Re: using explicit_bzero
Previous Message David Steele 2019-06-21 13:37:59 File descriptors inherited by restore_command