Re: Fdw batch insert error out when set batch_size > 65535

On 6/9/21 4:05 PM, Tomas Vondra wrote:
> On 6/9/21 3:28 PM, Tom Lane wrote:
>> Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com> writes:
>>> Note that the problem here is [1] - we're creating a lot of slots
>>> referencing the same tuple descriptor, which inflates the duration.
>>> There's a fix in the other thread, which eliminates ~99% of the
>>> overhead. I plan to push that fix soon (a day or two).
>>
>> Oh, okay, as long as there's a plan to bring the time back down.
>>
>
> Yeah. Sorry for not mentioning this in the original message about the
> new regression test.
>

I've pushed a fix addressing the performance issue.

There's one caveat, though - for regular builds the slowdown is pretty
much eliminated. But with valgrind it's still considerably slower. For
postgres_fdw the "make check" used to take ~5 minutes for me, now it
takes >1h. And yes, this is entirely due to the new test case which is
generating / inserting 70k rows. So maybe the test case is not worth it
after all, and we should get rid of it.

regards

--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company