From: | Alexander Korotkov <aekorotkov(at)gmail(dot)com> |
---|---|
To: | Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com> |
Cc: | Alexander Pyhalov <a(dot)pyhalov(at)postgrespro(dot)ru>, Andrei Lepikhov <a(dot)lepikhov(at)postgrespro(dot)ru>, Richard Guo <guofenglinux(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: Assert failure on 'list_member_ptr(rel->joininfo, restrictinfo)' |
Date: | 2023-12-25 01:04:36 |
Message-ID: | CAPpHfduFcZrcMPY8wdZv+X67HQT4QLyMs4CAz7BugiPG8RB5oA@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Mon, Dec 25, 2023 at 2:56 AM Ashutosh Bapat
<ashutosh(dot)bapat(dot)oss(at)gmail(dot)com> wrote:
> On Sun, Dec 24, 2023 at 5:32 PM Alexander Korotkov <aekorotkov(at)gmail(dot)com> wrote:
> >
> >
> > Thank you Ashutosh for your work on this matter. With a large number of partitions, it definitely makes sense to reduce both Bitmapset's size as well as the number of Bitmapsets.
> >
> > I've checked the patchset [1] with your test suite to check the memory consumption. The results are in the table below.
> >
> > query | no patch | patch | no self-join removal
> > ----------------------------------------------------------------------------------
> > 2-way join, non partitioned | 14792 | 15208 | 29152
> > 2-way join, no partitionwise join | 19519576 | 19519576 | 19519576
> > 2-way join, partitionwise join | 40851968 | 40851968 | 40851968
> > 3-way join, non partitioned | 20632 | 21784 | 79376
> > 3-way join, no partitionwise join | 45227224 | 45227224 | 45227224
> > 3-way join, partitionwise join | 151655144 | 151655144 | 151655144
> > 4-way join, non partitioned | 25816 | 27736 | 209128
> > 4-way join, no partitionwise join | 83540712 | 83540712 | 83540712
> > 4-way join, partitionwise join | 463960088 | 463960088 | 463960088
> > 5-way join, non partitioned | 31000 | 33720 | 562552
> > 5-way join, no partitionwise join | 149284376 | 149284376 | 149284376
> > 5-way join, partitionwise join | 1663896608 | 1663896608 | 1663896608
> >
> >
> > The most noticeable thing for me is that self-join removal doesn't work with partitioned tables. I think this is the direction for future work on this subject. In non-partitioned cases, patchset gives a small memory overhead. However, the memory consumption is still much less than it is without the self-join removal. So, removing the join still lowers memory consumption even if it copies some Bitmapsets. Given that patchset [1] is required for the correctness of memory manipulations in Bitmapsets during join removals, I'm going to push it if there are no objections.
>
> I am missing the link between this work and the self join work. Can
> you please provide me relevant pointers?
This thread was started from the bug in self-join removal [1]. The
fix under consideration [2] makes replace_relid() leave the argument
unmodified. I've used your test set [3] to check the memory overhead
of this solution.
Links.
1. https://www.postgresql.org/message-id/CAMbWs4_wJthNtYBL%2BSsebpgF-5L2r5zFFk6xYbS0A78GKOTFHw%40mail.gmail.com
2. https://www.postgresql.org/message-id/CAPpHfdtLgCryACcrmLv%3DKoq9rAB3%3Dtr5y9D84dGgvUhSCvjzjg%40mail.gmail.com
3. https://www.postgresql.org/message-id/CAExHW5stmOUobE55pMt83r8UxvfCph%2BPvo5dNpdrVCsBgXEzDQ%40mail.gmail.com
------
Regards,
Alexander Korotkov
From | Date | Subject | |
---|---|---|---|
Next Message | Junwang Zhao | 2023-12-25 02:17:50 | Re: Transaction timeout |
Previous Message | Ashutosh Bapat | 2023-12-25 00:56:20 | Re: Assert failure on 'list_member_ptr(rel->joininfo, restrictinfo)' |