Re: Input and Output data traffic

From: Israel Ben Guilherme Fonseca <israel(dot)bgf(at)gmail(dot)com>
To: psycopg(at)postgresql(dot)org
Subject: Re: Input and Output data traffic
Date: 2011-05-12 01:16:08
Message-ID: BANLkTimpHE8x4RWRcmo5sLgsWMi+ufDCMw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: psycopg

Man.. what a twist of events. My java code was indeed implemented wrong,
while python/ruby were all normalized with same input values 20 character
strings for the columns, the java was with 2! LOL

Now the bizarre thing... In the and the JDBC driver is the one with a hight
traffic (almost the double), maybe I'm wrong AGAIN, but now i'm migrating to
the jdbc forum to ask about it.

Anyway, I've already killed the responsible of this outrage.

Thanks guys for the help and the tremendous amount of knowledge, sorry for
the trouble.

Just for curiosity:

# Python

conn = psycopg2.connect("dbname=tcc host=localhost port=%s user=postgres
password=postgres" % 4444)

for i in range(100):
c = conn.cursor()
c.execute("select * from pessoa")
c.fetchall()
c.close()

# Sent 8.3 Receive 9.6 (KB)

// Java

Properties p = new Properties();
p.setProperty("URL", "jdbc:postgresql://localhost:4444/tcc");
p.setProperty("user", "postgres");
p.setProperty("password", "postgres");

Connection con = DriverManager.getConnection(p.getProperty("URL"),
p);
con.setAutoCommit(false);

for(int i = 0; i < 100; i++){
con.createStatement().executeQuery("select * from
pessoa").next();
}

// Sent 6.1 Receive 31.1 (KB)

The postgre log difference (repeats 100 times):

Python LOG: comando: select * from pessoa
Java LOG: executar <unnamed>: select * from pessoa

comando -> command, executar -> execute

Should this mean something?

2011/5/11 Israel Ben Guilherme Fonseca <israel(dot)bgf(at)gmail(dot)com>

> Man now YOU really screwed my mind. AHUAHua
>
> Actually i'm doing tests for all operations (insert, select, delete,
> update), but here in mailist to keep things simple I'm trying to focus on
> the insert operation.
>
> Let's keep simple. The objective is (example):
>
> Java
> sent: 100 bytes
> received 30 bytes
>
> Python
> sent: 100 bytes
> received: 85 bytes
>
> I want to see the python driver getting the same result (forget about %
> differences maybe that turned everything more confuse). Did you get it?
>
> But anyway, I'll do more tests, and maybe even put some charts here, I'll
> recheck my implementations too. We are talking about this since the morning
> and I didn't have time to recheck the code (now with postgres logs like
> sugested). Maybe everything is an false alarm. :)
>
>
> 2011/5/11 Karsten Hilbert <Karsten(dot)Hilbert(at)gmx(dot)net>
>
>> On Wed, May 11, 2011 at 02:37:28PM -0300, Israel Ben Guilherme Fonseca
>> wrote:
>>
>> > With that extra clause on every commit, that could increase the size of
>> the
>> > sent data. JDBC doesn't do it, so maybe it could be the cause, at last
>> for
>> > the sent data not the received data.
>>
>> This is what you originally wrote:
>>
>> >> The volume of sent data (application -> database) and the response data
>> >> (database -> application) are basically the same.
>> >>
>> >> For simple inserts/updates/deletes i got a difference of about 15%
>> between
>> >> sent and received data.
>>
>> (later you said you were testing INSERTs only ...)
>>
>> >> I used a ruby driver and got the same results for the same instruction.
>> >>
>> >> Finally I used a Java JDBC driver (for postgres too), and the
>> difference was
>> >> huge, it was 70% (the received data was much smaller).
>>
>> What you've been wondering here is this:
>>
>> - psycopg2 sends, say, 100 bytes (+some) and receives 85 bytes (+some)
>> - JDBC sends, say, 100 bytes and receives 30 bytes
>>
>> Now, since psycopg2 supposedly *sends more* data (namely
>> setting the transaction isolation level) how is that
>> supposed to make the difference between sent and received
>> *smaller* ? This would only make sense if the "more" data
>> would provoke *a lot more* data to be received. Which
>> setting the transaction isolation certainly shouldn't.
>>
>> But maybe I'm royally screwing up my thinking :-)))
>>
>> Karsten
>> --
>> GPG key ID E4071346 @ gpg-keyserver.de
>> E167 67FD A291 2BEA 73BD 4537 78B9 A9F9 E407 1346
>>
>> --
>> Sent via psycopg mailing list (psycopg(at)postgresql(dot)org)
>> To make changes to your subscription:
>> http://www.postgresql.org/mailpref/psycopg
>>
>
>

In response to

Responses

Browse psycopg by date

  From Date Subject
Next Message Federico Di Gregorio 2011-05-12 08:01:18 Re: Transaction control overhauling
Previous Message Israel Ben Guilherme Fonseca 2011-05-11 23:11:39 Re: Transaction control overhauling