[PATCH] Atomic pgrename on Windows

From: Alexander Korotkov <a(dot)korotkov(at)postgrespro(dot)ru>
To: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: [PATCH] Atomic pgrename on Windows
Date: 2017-11-27 06:28:35
Message-ID: CAPpHfds7dyuGZt+PF2GL9qSSVV0OZnjNwqiCPjN7mirDw882tA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi!

It's assumed in PostgreSQL codebase that pgrename atomically replaces
target file with source file even if target file is open and being read by
another process. And this assumption is true on Linux, but it's false on
Windows. MoveFileEx() triggers an error when target file is open (and
accordingly locked). Some our customers has been faced such errors while
operating heavily loaded PostgreSQL instance on Windows.

LOG could not rename temporary statistics file "pg_stat_tmp/global.tmp" to
"pg_stat_tmp/global.stat": Permission denied

I've managed to reproduce this situation. Reliable reproducing of this
issue required patch to PostgreSQL core. I've written
slow-read-statfiles.patch for artificial slowdown
of pgstat_read_statsfiles() – sleep 100 ms after each statfile entry. If
you run make-100-dbs.sql on patched version, and then few times execute

select pg_stat_get_tuples_inserted('t1'::regclass);

in psql, then you would likely get the error above on Windows.

Attached patch atomic-pgrename-windows-1.patch fixes this problem. It
appears to be possible to atomically replace file on Windows –
ReplaceFile() does that. ReplaceFiles() requires target file to exist,
this is why we still need to call MoveFileEx() when it doesn't exist.

This patch is based on work of Victor Spirin who was asked by Postgres Pro
to research this problem.

------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Attachment Content-Type Size
slow-read-statfiles.patch application/octet-stream 381 bytes
make-100-dbs.sql application/octet-stream 5.1 KB
atomic-pgrename-windows-1.patch application/octet-stream 1.5 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Kyotaro HORIGUCHI 2017-11-27 06:49:38 Re: [HACKERS] More stats about skipped vacuums
Previous Message Jing Wang 2017-11-27 05:39:53 Re: [HACKERS] [PATCH] Vacuum: Update FSM more frequently