Skip site navigation (1) Skip section navigation (2)

Re: INSERTing lots of data

From: Joachim Worringen <joachim(dot)worringen(at)iathh(dot)de>
To: pgsql-general(at)postgresql(dot)org
Subject: Re: INSERTing lots of data
Date: 2010-05-28 10:00:58
Message-ID: 4BFF945A.70302@iathh.de (view raw or flat)
Thread:
Lists: pgsql-general
On 05/28/2010 11:48 AM, Szymon Guz wrote:
> Remember about Python's GIL in some Python implementations so those
> threads could be serialized at the Python level.

My multi-threaded queries scale nicely with Python 2.6 on Linux, so this 
is not an issue here. But the queries do not perform concurrent write 
accesses on the same table.

> This is possible that those inserts will be faster. The speed depends on
> the table structure, some constraints and triggers and even database
> configuration. The best answer is: just check it on some test code, make
> a simple multithreaded aplication and try to do the inserts and check
> that out.

Sure, testing always shows something, but I wonder if something general 
can be said about the execution of concurrent write transaction on the 
same table (no triggers, some non-NULL constraints, one index).

http://www.postgresql.org/docs/8.4/interactive/mvcc-intro.html says 
about MVCC:
"
The main advantage of using the MVCC model of concurrency control rather 
than locking is that in MVCC locks acquired for querying (reading) data 
do not conflict with locks acquired for writing data, and so reading 
never blocks writing and writing never blocks reading.
"

It does not mention whether writing may block writing, or if it always 
does. 
http://bytes.com/topic/python/answers/728130-parallel-insert-postgresql-thread 
indicates it should not block - can this be confirmed by some Postgresql 
guru?

thanks, Joachim



In response to

pgsql-general by date

Next:From: Martin GaintyDate: 2010-05-28 10:14:10
Subject: Re: INSERTing lots of data
Previous:From: Szymon GuzDate: 2010-05-28 09:48:16
Subject: Re: INSERTing lots of data

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group