Re: [proposal] de-TOAST'ing using a iterator

From: Binguo Bao <djydewang(at)gmail(dot)com>
To: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
Cc: Andrey Borodin <x4mmm(at)yandex-team(dot)ru>, Atri Sharma <atri(dot)jiit(at)gmail(dot)com>, Stephen Frost <sfrost(at)snowman(dot)net>, Владимир Лесков <vladimirlesk(at)yandex-team(dot)ru>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [proposal] de-TOAST'ing using a iterator
Date: 2019-07-10 16:39:05
Message-ID: CAL-OGkvQDgC3e8M9t09fgD=QV5XDi76z2UBjewXomQNgBtd4sQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

This is the patch that fix warnings.

Best Regards,
Binguo Bao

Binguo Bao <djydewang(at)gmail(dot)com> 于2019年7月10日周三 下午10:18写道:

> Hi Thomas,
> I've fixed the warnings.
>
> Thomas Munro <thomas(dot)munro(at)gmail(dot)com> 于2019年7月5日周五 下午12:21写道:
>
>> On Thu, Jun 20, 2019 at 1:51 AM Binguo Bao <djydewang(at)gmail(dot)com> wrote:
>> > Hi hackers!
>> > This proposal aims to provide the ability to de-TOAST a fully TOAST'd
>> and compressed field using an iterator and then update the appropriate
>> parts of the code to use the iterator where possible instead of
>> de-TOAST'ing and de-compressing the entire value. Examples where this can
>> be helpful include using position() from the beginning of the value, or
>> doing a pattern or substring match.
>> >
>> > de-TOAST iterator overview:
>> > 1. The caller requests the slice of the attribute value from the
>> de-TOAST iterator.
>> > 2. The de-TOAST iterator checks if there is a slice available in the
>> output buffer, if there is, return the result directly,
>> > otherwise goto the step3.
>> > 3. The de-TOAST iterator checks if there is the slice available in the
>> input buffer, if there is, goto step44. Otherwise,
>> > call fetch_datum_iterator to fetch datums from disk to input buffer.
>> > 4. If the data in the input buffer is compressed, extract some data
>> from the input buffer to the output buffer until the caller's
>> > needs are met.
>> >
>> > I've implemented the prototype and apply it to the position() function
>> to test performance.
>>
>> Hi Binguo,
>>
>> Interesting work, and nice performance improvements so far. Just by
>> the way, the patch currently generates warnings:
>>
>> https://travis-ci.org/postgresql-cfbot/postgresql/builds/554345719
>>
>> --
>> Thomas Munro
>> https://enterprisedb.com
>>
>

Attachment Content-Type Size
0001-de-TOASTing-using-a-iterator-3.patch text/x-patch 22.0 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2019-07-10 16:57:08 buildfarm's typedefs list has gone completely nutso
Previous Message Robert Haas 2019-07-10 16:36:14 Re: POC: Cleaning up orphaned files using undo logs