Re: ERROR: invalid memory alloc request size when committing transaction

Michael Harris <michael(dot)harris(at)ericsson(dot)com> writes:
> - Next we run a migration script on each database in the cluster. The migration script
> converts a large number of tables from inheritance based partitioning to declarative
> partitioning. Unfortunately I am not at liberty to share the migration script.

How many is "a large number"?

> The migration is performed inside a transaction, and during the process of committing the
> transaction the following error is thrown:
> [2021-08-11 11:27:50 CEST] aue_75(at)218006 218015(at)[local] db_vrqv1 ERROR: invalid memory alloc request size 1073741824
> [2021-08-11 11:27:50 CEST] aue_75(at)218006 218015(at)[local] db_vrqv1 STATEMENT: commit
> [2021-08-11 11:27:50 CEST] aue_75(at)218006 218015(at)[local] db_vrqv1 WARNING: AbortTransaction while in COMMIT state

It's impossible to do more than speculate with this amount of information.
However, a plausible speculation is that you're processing so many objects
in one transaction as to overrun the limit on how large a WAL commit record,
or something related to that, can be. Can you adjust your migration
script to break it into multiple transactions that process fewer tables
per transaction?

regards, tom lane