| From: | Pengzhou Tang <ptang(at)pivotal(dot)io> |
|---|---|
| To: | Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com> |
| Cc: | Richard Guo <guofenglinux(at)gmail(dot)com>, Jesse Zhang <sbjesse(at)gmail(dot)com>, Richard Guo <riguo(at)pivotal(dot)io>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Michael Paquier <michael(at)paquier(dot)xyz>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, Andres Freund <andres(at)anarazel(dot)de>, Andrew Gierth <andrew(at)tao11(dot)riddles(dot)org(dot)uk> |
| Subject: | Re: Parallel grouping sets |
| Date: | 2020-03-25 14:35:32 |
| Message-ID: | CAG4reAR+xM1fpZbw7uYUm-3LO0pkhcP6W0_pO4k9F4+fr5R_yQ@mail.gmail.com |
| Views: | Raw Message | Whole Thread | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
>
>
> The memory context stats from a running process before it gets killed by
> OOM look like this
>
> TopMemoryContext: 101560 total in 6 blocks; 7336 free (6 chunks); 94224
> used
> TopTransactionContext: 73816 total in 4 blocks; 11624 free (0
> chunks); 62192 used
> ExecutorState: 1375731712 total in 174 blocks; 5391392 free (382
> chunks); 1370340320 used
> HashAgg meta context: 315784 total in 10 blocks; 15400 free (2
> chunks); 300384 used
> ExprContext: 8192 total in 1 blocks; 7928 free (0 chunks); 264
> used
> ExprContext: 8192 total in 1 blocks; 7928 free (0 chunks); 264
> used
> ExprContext: 8192 total in 1 blocks; 7928 free (0 chunks); 264
> used
> ...
>
> That's 1.3GB allocated in ExecutorState - that doesn't seem right.
>
> FWIW there are only very few groups (each attribute has fewer than 30
> distinct values, so there's only about ~1000 groups. On master it works
> just fine, of course.
>
>
Thanks a lot, the patch has a memory leak in the lookup_hash_entries, it
uses a list_concat there
and causes a 64-byte leak for every tuple, has fixed that.
Also, resolved conflicts and rebased the code.
Thanks,
Pengzhou
| Attachment | Content-Type | Size |
|---|---|---|
| 0003-fix-a-numtrans-bug.patch | application/octet-stream | 3.4 KB |
| 0001-All-grouping-sets-do-their-own-sorting.patch | application/octet-stream | 35.9 KB |
| 0002-fixes.patch | application/octet-stream | 6.8 KB |
| v1-0004-Reorganise-the-aggregate-phases.patch | application/octet-stream | 89.8 KB |
| 0005-Parallel-grouping-sets.patch | application/octet-stream | 63.6 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Ibrar Ahmed | 2020-03-25 14:37:22 | Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits |
| Previous Message | Konstantin Knizhnik | 2020-03-25 14:29:59 | Re: weird hash plan cost, starting with pg10 |