Re: Avoiding deadlocks on mass delete / update

From: Роман Маширов <mrj(at)nandu(dot)ru>
To: Craig Ringer <craig(at)postnewspapers(dot)com(dot)au>
Cc: PostgreSQL General <pgsql-general(at)postgresql(dot)org>
Subject: Re: Avoiding deadlocks on mass delete / update
Date: 2010-03-26 10:15:23
Message-ID: 4BAC893B.4030301@nandu.ru
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Craig Ringer wrote:
<blockquote cite="mid:%3C4BA99567(dot)6070502(at)postnewspapers(dot)com(dot)au%3E"
type="cite">
<pre wrap="">Роман Маширов wrote:
</pre>
<blockquote type="cite">
<pre wrap="">I've got a simple 'spool' table, one process 'worker' reads and updates
this table, other 'stat' performs 'delete ... where ... returning *'.
Sometimes I've got dedlocks on delete operation in 'stat', seems like at
the moment of expiration of data by timeout some state changes arrived
from worker. So the question, is it possible to somehow set order of row
deletion in such bulk delete operation, to avoid deadlocks?
</pre>
</blockquote>
<pre wrap=""><!---->
OK, so for the sake of example, WORKER is UPDATEing rows that stat is
trying to DELETE at the same time, such that worker holds a lock on row
A and wants a lock on row B, but stat holds B and wants A?

In other words, the deadlock is an _interaction_ between 'stat' and
'worker'?
</pre>
</blockquote>
yes<br>
<blockquote cite="mid:%3C4BA99567(dot)6070502(at)postnewspapers(dot)com(dot)au%3E"
type="cite">
<pre wrap="">
Can you post the queries?
</pre>
</blockquote>
as dumb as possible :) <br>
<br>
worker parses several thousand events and do<br>
update queue set state=$1 where queue_id in (&lt;id list&gt;) and state
in (&lt;previous state list&gt;)<br>
for each target state, so it performs 1-4 update queries.<br>
<br>
stat do<br>
delete from queue where queue_stamp &lt; now()-'1day'::interval or
state in (&lt;terminal state list&gt;)<br>
returning *<br>
<br>
The main reason for such thing is to reduce amount of queries to
perform, since this queue could work at about 50 objects per second
with 4 state changes.<br>
<blockquote cite="mid:%3C4BA99567(dot)6070502(at)postnewspapers(dot)com(dot)au%3E"
type="cite">
<pre wrap="">One option is to SELECT ... FOR UPDATE NOWAIT before your UPDATE or DELETE.
</pre>
</blockquote>
Yep, thank you very much!<br>
<br>
But, it would be good feature to somehow allow to explicitly set order
of multi-row update / delete, <br>
or to 'delete but locked', meaning delete all rows by given query
parameters, except locked ones. <br>
<br>
--<br>
MRJ<br>
</body>
</html>

Attachment Content-Type Size
unknown_filename text/html 2.3 KB

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Paresh Masani 2010-03-26 10:29:43 Invitation to connect on LinkedIn
Previous Message Tom Lane 2010-03-26 06:38:10 Re: Problem with Memory Leak