Re: speedup COPY TO for partitioned table.

From: jian he <jian(dot)universality(at)gmail(dot)com>
To: Álvaro Herrera <alvherre(at)kurilemu(dot)de>
Cc: torikoshia <torikoshia(at)oss(dot)nttdata(dot)com>, Kirill Reshke <reshkekirill(at)gmail(dot)com>, vignesh C <vignesh21(at)gmail(dot)com>, David Rowley <dgrowleyml(at)gmail(dot)com>, Melih Mutlu <m(dot)melihmutlu(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: speedup COPY TO for partitioned table.
Date: 2025-07-15 03:31:58
Message-ID: CACJufxG2O2Nk2DSQZOH8E0grv+t4Nizaqn1wpdt_gDpEA+ZX1w@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, Jul 14, 2025 at 10:02 PM Álvaro Herrera <alvherre(at)kurilemu(dot)de> wrote:
>
> On 2025-Jul-02, jian he wrote:
>
> > @@ -673,11 +680,34 @@ BeginCopyTo(ParseState *pstate,
> > errmsg("cannot copy from sequence \"%s\"",
> > RelationGetRelationName(rel))));
> > else if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
> > - ereport(ERROR,
> > - (errcode(ERRCODE_WRONG_OBJECT_TYPE),
> > - errmsg("cannot copy from partitioned table \"%s\"",
> > - RelationGetRelationName(rel)),
> > - errhint("Try the COPY (SELECT ...) TO variant.")));
> > + {
> > + children = find_all_inheritors(RelationGetRelid(rel),
> > + AccessShareLock,
> > + NULL);
> > +
> > + foreach_oid(childreloid, children)
> > + {
> > + char relkind = get_rel_relkind(childreloid);
> > +
> > + if (relkind == RELKIND_FOREIGN_TABLE)
> > + {
> > + char *relation_name;
> > +
> > + relation_name = get_rel_name(childreloid);
> > + ereport(ERROR,
> > + errcode(ERRCODE_WRONG_OBJECT_TYPE),
> > + errmsg("cannot copy from foreign table \"%s\"", relation_name),
> > + errdetail("Partition \"%s\" is a foreign table in the partitioned table \"%s\"",
> > + relation_name, RelationGetRelationName(rel)),
> > + errhint("Try the COPY (SELECT ...) TO variant."));
> > + }
>
> This code looks like it's duplicating what you could obtain by using
> RelationGetPartitionDesc and then observe the ->isleaf bits. Maybe have
> a look at the function RelationHasForeignPartition() in the patch at
> https://postgr.es/m/CANhcyEW_s2LD6RiDSMHtWQnpYB67EWXqf7N8mn7dOrnaKMfROg@mail.gmail.com
> which looks very similar to what you need here. I think that would also
> have the (maybe dubious) advantage that the rows will be output in
> partition bound order rather than breadth-first (partition hierarchy)
> OID order.
>
hi.

else if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
{
PartitionDesc pd = RelationGetPartitionDesc(rel, true);
for (int i = 0; i < pd->nparts; i++)
{
Relation partRel;
if (!pd->is_leaf[i])
continue;
partRel = table_open(pd->oids[i], AccessShareLock);
if (partRel->rd_rel->relkind == RELKIND_FOREIGN_TABLE)
ereport(ERROR,
errcode(ERRCODE_WRONG_OBJECT_TYPE),
errmsg("cannot copy from foreign table
\"%s\"", RelationGetRelationName(partRel)),
errdetail("Partition \"%s\" is a foreign
table in the partitioned table \"%s\"",

RelationGetRelationName(partRel), RelationGetRelationName(rel)),
errhint("Try the COPY (SELECT ...) TO
variant."));
table_close(partRel, NoLock);
scan_oids = lappend_oid(scan_oids, RelationGetRelid(partRel));
}
}

I tried the above code, but it doesn't work because RelationGetPartitionDesc
only retrieves the immediate partition descriptor of a partitioned relation, it
doesn't recurse to the lowest level.

Actually Melih Mutlu raised this question at
https://postgr.es/m/CAGPVpCQou3hWQYUqXNTLKdcuO6envsWJYSJqbZZQnRCjZA6nkQ%40mail.gmail.com
I kind of ignored it...
I guess we have to stick with find_all_inheritors here?

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2025-07-15 03:37:10 Re: ScanKeys passed to table_beginscan in SeqNext
Previous Message Michael Paquier 2025-07-15 03:31:06 Re: pg_logical_slot_get_changes waits continously for a partial WAL record spanning across 2 pages