Re: BUG #14654: With high statistics targets on ts_vector, unexpectedly high memory use & OOM are triggered

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Heikki Linnakangas <hlinnaka(at)iki(dot)fi>
Cc: james+postgres(at)carbocation(dot)com, pgsql-bugs(at)postgresql(dot)org
Subject: Re: BUG #14654: With high statistics targets on ts_vector, unexpectedly high memory use & OOM are triggered
Date: 2017-07-12 15:30:59
Message-ID: 4427.1499873459@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

Heikki Linnakangas <hlinnaka(at)iki(dot)fi> writes:
> Yes, I can see that happening here too. The problem seems to be that the
> analyze-function detoasts every row in the sample. Tsvectors can be very
> large, so it adds up.

> That's pretty easy to fix, the analyze function needs to free the
> detoasted copies as it goes. But in order to do that, it needs to make
> copies of all the lexemes stored in the hash table, instead of pointing
> directly to the detoasted copies.

> Patch attached. I think this counts as a bug, and we should backport this.

+1. I didn't test the patch, but it looks sane to the eyeball.

regards, tom lane

In response to

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message Tom Lane 2017-07-12 15:42:04 Re: BUG #14722: Segfault in tuplesort_heap_siftup, 32 bit overflow
Previous Message Heikki Linnakangas 2017-07-12 13:15:10 Re: BUG #14722: Segfault in tuplesort_heap_siftup, 32 bit overflow