Skip site navigation (1) Skip section navigation (2)

Re: json api WIP patch

From: Andrew Dunstan <andrew(at)dunslane(dot)net>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: json api WIP patch
Date: 2013-01-15 22:45:46
Message-ID: 50F5DC1A.60008@dunslane.net (view raw or flat)
Thread:
Lists: pgsql-hackers
On 01/15/2013 11:31 AM, Andrew Dunstan wrote:
>
> On 01/14/2013 11:02 PM, Andrew Dunstan wrote:
>>
>> On 01/14/2013 12:52 PM, Andrew Dunstan wrote:
>>>
>>> On 01/14/2013 11:32 AM, Robert Haas wrote:
>>>>
>>>> So, how much performance does this lose on json_in() on a large
>>>> cstring, as compared with master?
>>>
>>> That's a good question. I'll try to devise a test.
>>>
>>>>
>>>> I can't shake the feeling that this is adding a LOT of unnecessary
>>>> data copying.  For one thing, instead of copying every single lexeme
>>>> (including the single-character ones?) out of the original object, we
>>>> could just store a pointer to the offset where the object starts and a
>>>> length, instead of copying it.
>>>
>>> In the pure pares case (json_in, json_reccv) nothing extra should be 
>>> copied. On checking this after reading the above I found that wasn't 
>>> quite the case, and some lexemes (scalars and field names, but not 
>>> punctuation) were being copied when not needed. I have made a fix 
>>> (see 
>>> <https://bitbucket.org/adunstan/pgdevel/commits/139043dba7e6b15f1f9f7675732bd9dae1fb6497>) 
>>> which I will include in the next version I publish.
>>>
>>> In the case of string lexemes, we are passing back a de-escaped 
>>> version, so just handing back pointers to the beginning and end in 
>>> the input string doesn't work.
>>
>>
>> After a couple of iterations, some performance enhancements to the 
>> json parser and lexer have ended up with a net performance 
>> improvement over git tip. On our test rig, the json parse test runs 
>> at just over 13s per 10000 parses on git tip and approx 12.55s per 
>> 10000 parses with the attached patch.
>>
>> Truth be told, I think the lexer changes have more than paid for the 
>> small cost of the switch to an RD parser. But since the result is a 
>> net performance win PLUS some enhanced functionality, I think we 
>> should be all good.
>>
>>
>
> Latest version of this patch, including some documentation, mainly 
> from Merlin Moncure but tweaked by me.
>

Now with more comments.

cheers

andrew

Attachment: jsonapi6.patch
Description: text/x-patch (136.5 KB)

In response to

Responses

pgsql-hackers by date

Next:From: Tom LaneDate: 2013-01-15 22:46:57
Subject: Re: [PATCH] COPY .. COMPRESSED
Previous:From: Bruce MomjianDate: 2013-01-15 22:45:15
Subject: Re: Parallel query execution

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group