Re: Why the index is not used ?

From: Vladimir Ryabtsev <greatvovan(at)gmail(dot)com>
To: didier(dot)ros(at)edf(dot)fr
Cc: pgsql-sql(at)lists(dot)postgresql(dot)org, pgsql-performance(at)lists(dot)postgresql(dot)org, pgsql-general(at)lists(dot)postgresql(dot)org
Subject: Re: Why the index is not used ?
Date: 2018-10-06 16:51:24
Message-ID: CAMqTPqkEpPgm+vB8MyYTDnQ8q5fdK5VKz-g2zjACNk3Y=469kA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-performance pgsql-sql

Hello Didier,

(3), (5) to find the match, you decrypt the whole table, apparently this
take quite a long time.
Index cannot help here because indexes work on exact match of type and
value, but you compare mapped value, not indexed. Functional index should
help, but like it was said, it against the idea of encrypted storage.

(6) I never used pgp_sym_encrypt() but I see that in INSERT INTO you
supplied additional parameter 'compress-algo=2, cipher-algo=aes256' while
in (6) you did not. Probably this is the reason.

In general matching indexed bytea column should use index, you can ensure
in this populating the column unencrypted and using 'test value 32'::bytea
for match.
In you case I believe pgp_sym_encrypt() is not marked as STABLE or
IMMUTABLE that's why it will be evaluated for each row (very inefficient)
and cannot use index. From documentation:

"Since an index scan will evaluate the comparison value only once, not once
at each row, it is not valid to use a VOLATILE function in an index scan
condition."
https://www.postgresql.org/docs/10/static/xfunc-volatility.html

If you cannot add STABLE/IMMUTABLE to pgp_sym_encrypt() (which apparently
should be there), you can encrypt searched value as a separate operation
and then search in the table using basic value match.

Vlad

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Phil Endecott 2018-10-06 21:56:15 Text-indexing UTF-8 bytea, convert_from() immutability, null bytes...
Previous Message Pavel Stehule 2018-10-06 10:13:31 Re: Why the index is not used ?

Browse pgsql-performance by date

  From Date Subject
Next Message Paul McGarry 2018-10-07 02:20:53 Re: Why the index is not used ?
Previous Message Pavel Stehule 2018-10-06 10:13:31 Re: Why the index is not used ?

Browse pgsql-sql by date

  From Date Subject
Next Message Paul McGarry 2018-10-07 02:20:53 Re: Why the index is not used ?
Previous Message Pavel Stehule 2018-10-06 10:13:31 Re: Why the index is not used ?