From: | jian he <jian(dot)universality(at)gmail(dot)com> |
---|---|
To: | Kirill Reshke <reshkekirill(at)gmail(dot)com> |
Cc: | vignesh C <vignesh21(at)gmail(dot)com>, David Rowley <dgrowleyml(at)gmail(dot)com>, Melih Mutlu <m(dot)melihmutlu(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
Subject: | Re: speedup COPY TO for partitioned table. |
Date: | 2025-06-05 00:45:10 |
Message-ID: | CACJufxEEsFUHNdSHyO7Zy3yjp3pia2yZVfjTv9b7ZxkTKdQuKQ@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
hi.
In the V10 patch, there will be some regression if the partition column
ordering is different from the root partitioned table.
because in V10 CopyThisRelTo
+ while (table_scan_getnextslot(scandesc, ForwardScanDirection, slot))
+ {
+ if (map != NULL)
+ {
+ original_slot = slot;
+ root_slot = MakeSingleTupleTableSlot(rootdesc, &TTSOpsBufferHeapTuple);
+ slot = execute_attr_map_slot(map, slot, root_slot);
+ }
+ else
+ slot_getallattrs(slot);
+
+ if (original_slot != NULL)
+ ExecDropSingleTupleTableSlot(original_slot);
+}
as you can see, for each slot in the partition, i called
MakeSingleTupleTableSlot to get the dumpy root_slot
and ExecDropSingleTupleTableSlot too.
that will cause overhead.
we can call produce root_slot before the main while loop.
like the following:
+ if (root_rel != NULL)
+ {
+ rootdesc = RelationGetDescr(root_rel);
+ root_slot = table_slot_create(root_rel, NULL);
+ map = build_attrmap_by_name_if_req(rootdesc, tupdesc, false);
+ }
....
+ while (table_scan_getnextslot(scandesc, ForwardScanDirection, slot))
+ {
+ TupleTableSlot *copyslot;
+ if (map != NULL)
+ copyslot = execute_attr_map_slot(map, slot, root_slot);
+ else
+ {
+ slot_getallattrs(slot);
+ copyslot = slot;
+ }
+
please check CopyThisRelTo in v11. so, with v11, there is no
regression for case when
partition column ordering differs from partitioned.
I have tested 30 partitions, 10 columns, all the column ordering
is different with the root partitioned table.
copy pp to '/tmp/2.txt'
is still faster than
copy (select * from pp) to '/tmp/1.txt';
(359.463 ms versus 376.371 ms)
I am using -Dbuildtype=release
PostgreSQL 18beta1_release_build on x86_64-linux, compiled by gcc-14.1.0, 64-bit
you may see the attached test file.
Attachment | Content-Type | Size |
---|---|---|
copy_par_regress_test-7.sql | application/sql | 9.1 KB |
v11-0001-support-COPY-partitioned_table-TO.patch | text/x-patch | 10.6 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Michael Paquier | 2025-06-05 00:52:40 | Re: Encapsulate io_uring process count calculation |
Previous Message | Renan Alves Fonseca | 2025-06-05 00:33:09 | Re: PoC: Compute a histogram of prune_xid to support autovacuum improvements |