Re: Loading table with indexed jsonb field is stalling

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Will Hartung <willhartung(at)gmail(dot)com>
Cc: Adrian Klaver <adrian(dot)klaver(at)aklaver(dot)com>, Ron <ronljohnsonjr(at)gmail(dot)com>, pgsql-general(at)lists(dot)postgresql(dot)org
Subject: Re: Loading table with indexed jsonb field is stalling
Date: 2019-05-21 00:31:47
Message-ID: 22234.1558398707@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Will Hartung <willhartung(at)gmail(dot)com> writes:
>> On May 20, 2019, at 5:14 PM, Adrian Klaver <adrian(dot)klaver(at)aklaver(dot)com> wrote:
>> Well looks like you are down to Tom's suggestion of creating a test case. Given that it seems confined to the jsonb field and corresponding index, I would think that is all that is needed for the test case. Start with some smaller subset, say 10,000 rows and work up till you start seeing an issue.

> This will take quite some work, and I wouldn’t attempt it with less than 5M rows to load.

Well, you're the only one who's seen this problem, and none of the
rest of us have any idea how to reproduce it. So if you want something
to get done in a timely fashion, it's up to you to show us a test case.

My guess is that it wouldn't be that hard to anonymize your data to
the point where it'd be OK to show to someone else. It's unlikely
that the problem depends on the *exact* data you've got --- though it
might depend on string lengths and the number/locations of duplicates.
But you should be able to substitute random strings for the original
values while preserving that.

regards, tom lane

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Adrian Klaver 2019-05-21 00:33:52 Re: Loading table with indexed jsonb field is stalling
Previous Message Will Hartung 2019-05-21 00:22:36 Re: Loading table with indexed jsonb field is stalling