| From: | Andres Freund <andres(at)anarazel(dot)de> | 
|---|---|
| To: | pgsql-hackers(at)postgresql(dot)org, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, Antonin Houska <ah(at)cybertec(dot)at> | 
| Subject: | 2pc leaks fds | 
| Date: | 2020-04-06 02:56:51 | 
| Message-ID: | 20200406025651.fpzdb5yyb7qyhqko@alap3.anarazel.de | 
| Views: | Whole Thread | Raw Message | Download mbox | Resend email | 
| Thread: | |
| Lists: | pgsql-hackers | 
Hi,
Using 2PC with master very quickly leads to:
2020-04-05 19:42:18.368 PDT [2298126][5/2009:0] LOG:  out of file descriptors: Too many open files; release and retry
2020-04-05 19:42:18.368 PDT [2298126][5/2009:0] STATEMENT:  COMMIT PREPARED 'ptx_2';
This started with:
commit 0dc8ead46363fec6f621a12c7e1f889ba73b55a9 (HEAD -> master)
Author: Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>
Date:   2019-11-25 15:04:54 -0300
Refactor WAL file-reading code into WALRead()
I found this while trying to benchmark the effect of my snapshot changes
on 2pc. I just used the attached pgbench file.
andres(at)awork3:~/build/postgres/dev-assert/vpath$ pgbench -n -s 500 -c 4 -j 4 -T 100000 -P1 -f ~/tmp/pgbench-write-2pc.sql
progress: 1.0 s, 3723.8 tps, lat 1.068 ms stddev 0.305
client 2 script 0 aborted in command 8 query 0: ERROR:  could not seek to end of file "base/14036/16396": Too many open files
client 1 script 0 aborted in command 8 query 0: ERROR:  could not seek to end of file "base/14036/16396": Too many open files
client 3 script 0 aborted in command 8 query 0: ERROR:  could not seek to end of file "base/14036/16396": Too many open files
client 0 script 0 aborted in command 8 query 0: ERROR:  could not seek to end of file "base/14036/16396": Too many open files
transaction type: /home/andres/tmp/pgbench-write-2pc.sql
I've not yet reviewed the change sufficiently to pinpoint the issue.
It's a bit sad that nobody has hit this in the last few months :(.
Greetings,
Andres Freund
| Attachment | Content-Type | Size | 
|---|---|---|
| pgbench-write-2pc.sql | application/x-sql | 715 bytes | 
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Amit Kapila | 2020-04-06 03:25:01 | Re: WAL usage calculation patch | 
| Previous Message | Michael Paquier | 2020-04-06 02:47:57 | Re: ALTER tbl rewrite loses CLUSTER ON index |