Re: Jsonb transform for pl/python

From: Nikita Glukhov <n(dot)gluhov(at)postgrespro(dot)ru>
To: Alexander Korotkov <a(dot)korotkov(at)postgrespro(dot)ru>
Cc: Peter Eisentraut <peter(dot)eisentraut(at)2ndquadrant(dot)com>, Aleksander Alekseev <a(dot)alekseev(at)postgrespro(dot)ru>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Anthony Bykov <a(dot)bykov(at)postgrespro(dot)ru>
Subject: Re: Jsonb transform for pl/python
Date: 2018-06-22 23:44:29
Message-ID: 16593cd6-4200-e2ab-8d2e-58b456986d69@postgrespro.ru
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi!

We have found another performance problem in this transform -- very slow
conversion via I/O from PostgreSQL numerics (which are used for the
representation of jsonb numbers) to Python Decimals.

Attached patch with fix.

We are simply trying first to convert numeric to int64 if is does not have
digits after the decimal point, and then construct Python Int instead of
Decimal. Standard Python json.loads() does the same for exact integers.

A special function numeric_to_exact_int64() was added to numeric.c. Existing
numeric_int8() can't be used here because it rounds input numeric.

Performance results (see the second attached file jsonb_plplython_tests.sql
for the function definitions):

- calculating the length of the passed jsonb object (input transformation):

py_jsonb_length_trans opt 2761,873 ms
py_jsonb_length_trans 10419,230 ms
py_jsonb_length_json 8691,201 ms

- returning integer arrays (output transformation):

py_jsonb_ret_int_array_trans opt 3284,810 ms
py_jsonb_ret_int_array_trans 4540,063 ms
py_jsonb_ret_int_array_raw 5100,793 ms
py_jsonb_ret_int_array_json 9887,821 ms

- returning float arrays (output transformation):

py_jsonb_ret_float_array_trans opt 5699,360 ms
py_jsonb_ret_float_array_trans 5735,854 ms
py_jsonb_ret_float_array_raw 6516,514 ms
py_jsonb_ret_float_array_json 10869,213 ms

--
Nikita Glukhov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Attachment Content-Type Size
0001-Optimize-numeric-conversions-in-jsonb_plpython.patch text/x-patch 3.2 KB
jsonb_plpython_tests.sql application/sql 2.2 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2018-06-22 23:48:03 Re: Fix some error handling for read() and errno
Previous Message Nico Williams 2018-06-22 22:32:17 Re: Using JSONB directly from application