Rewriting PL/Python's typeio code

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: pgsql-hackers(at)postgreSQL(dot)org
Subject: Rewriting PL/Python's typeio code
Date: 2017-10-30 20:00:13
Message-ID: 24449.1509393613@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox
Thread:
Lists: pgsql-hackers

I started to work on teaching PL/Python about domains over composite,
and soon found that it was a can of worms. Aside from the difficulty
of shoehorning that in with a minimal patch, there were pre-existing
problems. I found that it didn't do arrays of domains right either
(ok, that's an oversight in my recent commit c12d570fa), and there
are assorted bugs that have been there much longer. For instance, if
you return a composite type containing a domain, it fails to enforce
domain constraints on the type's field. Also, if a transform function
is in use, it missed enforcing domain constraints on the result.
And in many places it missed checking domain constraints on null values,
because the plpy_typeio code simply wasn't called for Py_None.

Plus the code was really messy and duplicative, e.g. domain_check was
called in three different places ... which wasn't enough. It also did
a lot of repetitive catalog lookups.

So, I ended up rewriting/refactoring pretty heavily. The new idea
is to solve these problems by making heavier use of recursion between
plpy_typeio's conversion functions, and in particular to treat domains
as if they were containers. So now there's exactly one place to call
domain_check, in a conversion function that has first recursed to do
conversion of the base type. Nulls are treated more honestly, and
the SQL-to-Python functions are more careful about not leaking memory.
Also, I solved some of the repetitive catalog lookup problems by
making the code rely as much as possible on the typcache (which I think
didn't exist when this code originated). I added a couple of small
features to typcache to help with that.

This is a fairly large amount of code churn, and it could stand testing
by someone who's more Python-savvy than I am. So I'll stick it into
the upcoming commitfest as a separate item.

regards, tom lane

Attachment Content-Type Size
rewrite-plpy_typeio-1.patch text/x-diff 124.8 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2017-10-30 20:03:39 Re: Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM
Previous Message Nikita Glukhov 2017-10-30 19:53:38 Re: SQL/JSON in PostgreSQL