RE: POC: postgres_fdw insert batching

From: "tsunakawa(dot)takay(at)fujitsu(dot)com" <tsunakawa(dot)takay(at)fujitsu(dot)com>
To: 'Amit Langote' <amitlangote09(at)gmail(dot)com>, Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>
Cc: "Andrey V(dot) Lepikhov" <a(dot)lepikhov(at)postgrespro(dot)ru>, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: RE: POC: postgres_fdw insert batching
Date: 2021-01-15 14:59:55
Message-ID: TYAPR01MB2990BDB626DB2630E6CEA78AFEA70@TYAPR01MB2990.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

From: Amit Langote <amitlangote09(at)gmail(dot)com>
> Okay, so maybe not moving the whole logic into the FDW's
> BeginForeignModify(), but at least if we move this...
>
> @@ -441,6 +449,72 @@ ExecInsert(ModifyTableState *mtstate,
> + /*
> + * Determine if the FDW supports batch insert and determine the
> batch
> + * size (a FDW may support batching, but it may be disabled for the
> + * server/table). Do this only once, at the beginning - we don't want
> + * the batch size to change during execution.
> + */
> + if (resultRelInfo->ri_FdwRoutine->GetForeignModifyBatchSize &&
> + resultRelInfo->ri_FdwRoutine->ExecForeignBatchInsert &&
> + resultRelInfo->ri_BatchSize == 0)
> + resultRelInfo->ri_BatchSize =
> +
> resultRelInfo->ri_FdwRoutine->GetForeignModifyBatchSize(resultRelInfo);
>
> ...into ExecInitModifyTable(), ExecInsert() only needs the following block:

Does ExecInitModifyTable() know all leaf partitions where the tuples produced by VALUES or SELECT go? ExecInsert() doesn't find the target leaf partition for the first time through the call to ExecPrepareTupleRouting()? Leaf partitions can have different batch_size settings.

Regards
Takayuki Tsunakawa

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Langote 2021-01-15 15:34:09 Re: POC: postgres_fdw insert batching
Previous Message Álvaro Herrera 2021-01-15 14:29:26 Re: {CREATE INDEX, REINDEX} CONCURRENTLY improvements