Re: VOPS-2.0

On 28.11.2018 16:45, Alvaro Herrera wrote:
> On 2018-Nov-28, Bruce Momjian wrote:
>
>> On Wed, Nov 28, 2018 at 01:01:03PM +0300, Konstantin Knizhnik wrote:
>>> Hi,
>>>
>>> I want to introduce new version of VOPS extension for Postgres (vectorized
>>> operations) providing new, more convenient way of usage:
>>> auto-substitution of projections.
>> [Announce post moved to hackers.]
>>
>> I remember the good performance numbers from this feature. How does
>> this interact with the JIT executor feature, which is also designed to
>> speed up the executor? Is it something that can be combined with JIT?
> ISTM that VOPS is a temporary hack that we'll need to include in some
> form eventually, but that basing it on top of pluggable storage would be
> a better way forward than the current proposal.
>
I am not sure that pluggable storage API will solve all problems.
Right now there are the following challenges with VOPS:
1. Provide convenient way of defining projections for user: not doable
now at extension level.
2. Vector executor: in principle it is possible to reimplement all (or
most) nodes to use vector operations and somehow make optimizer to
generate them. But it requires huge amount of work. And I am not sure
that all can be done et extension level just by defining new custom nodes.
3. Update projections. It can be efficiently done only using bulk
inserts, but it means that projections are not up-to-date with original
table. Hiding internals inside pluggable storage doesn't automatically
solve this problem.

Actually VOPS approach has least of all problems with storage layer,
because VOPS projections are stored as normal Postgres tables.
The main challenge is incorporating vector operations in executor.

--
Konstantin Knizhnik
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company