Re: Input and Output data traffic

From: Israel Ben Guilherme Fonseca <israel(dot)bgf(at)gmail(dot)com>
To: psycopg(at)postgresql(dot)org
Subject: Re: Input and Output data traffic
Date: 2011-05-11 14:38:40
Message-ID: BANLkTimq6gv3Sb=GgZoiiEeuYvg96sXMag@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: psycopg

Thanks for the insight Oswaldo, i'll give a look. Do you know the exactly
file path to enable the log? Does the pgAdmin3 have some support to show
this data?

About the cache theory:

I'm explicitly committing the insert instruction at every iteration, so I
don't think that's possible to cache all the inserts in one statement like
you did.

But I'll give a look anyway,

Thanks again,

2011/5/11 Oswaldo <listas(at)soft-com(dot)es>

> El 11/05/2011 15:04, Israel Ben Guilherme Fonseca escribió:
>
> No I didn't. I'm using the "noob-approach", and using all default, no
>> special optimization, because normally that's what the people around
>> here do. :)
>>
>> For ruby, python and java, I just got the drivers and used it in the
>> simplest way.
>>
>> The tests are all built on "synthetic" scenario, I cratead the base,
>> tables, tests. So it's unlike to have some unexpected behavior
>> happening, but I'll give a look anyway.
>>
>> About caching plan / binary protocol (and I know
>> very-little-almost-nothing about these), it can't be used with psycopg?
>> Let's say that the jdbc driver use these techniques, shouldn't psycopg
>> do the same? (let-me repeat that I may not know about what i'm talking
>> here. :) )
>>
>>
> Perhaps the jdbc driver are caching the querys, note that you are using
> only inserts whitout any return, then these calls are equivalent but the
> first is faster and uses less network traffic:
>
> # 1 single call with 1000 inserts
> cursor.execute("insert into foo (a,b,c) values(1,2,3);" * 1000)
>
> # 1000 calls, each time an insert
> for n in range(1000):
> cursor.execute("insert into foo (a,b,c) values(1,2,3)")
>
> I'm testing these technique to use it over low bandwith lines.
>
> I suggest:
>
> - Enable log_connections and log_ddl in the server an see what is receiving
> the server in each test and the time used in the server side
>
> - Check if the drivers are using ssl connections with compressed data.
>
> - Modify your test app and do selects between inserts.
>
>
> Regards
>
> --
> Oswaldo Hernández
>
> --
> Sent via psycopg mailing list (psycopg(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/psycopg
>

In response to

Responses

Browse psycopg by date

  From Date Subject
Next Message Daniele Varrazzo 2011-05-11 14:56:56 Re: Input and Output data traffic
Previous Message Oswaldo 2011-05-11 14:21:15 Re: Input and Output data traffic