Re: pg_bulkload slowing down

From: Álvaro Herrera <alvherre(at)kurilemu(dot)de>
To: Sbob <sbob(at)quadratum-braccas(dot)com>
Cc: pgsql-admin(at)lists(dot)postgresql(dot)org
Subject: Re: pg_bulkload slowing down
Date: 2025-07-28 13:23:33
Message-ID: 202507281323.55loakn7tfyx@alvherre.pgsql
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-admin

On 2025-Jul-28, Sbob wrote:

> All;
>
>
> We are loading a series of files into a single table, each file has approx
> 500million rows, the first file took ~ 1.25 hours, now on file number 8 it
> has been running for more than 10 hours, there is only a PK index on the
> table.

I bet it's trying to find unused OIDs for toast pointers -- there are
only 2^32-1 of those available in any one toast table, and by the eighth
file of 500 million rows, you'd run out.

You could workaround this by partitioning the table.

--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/

In response to

Responses

Browse pgsql-admin by date

  From Date Subject
Next Message Ron Johnson 2025-07-28 13:52:59 Re: pg_bulkload slowing down
Previous Message Ron Johnson 2025-07-28 13:17:32 Re: pg_bulkload slowing down