Re: [HACKERS] Block level parallel vacuum

From: Mahendra Singh <mahi6run(at)gmail(dot)com>
To: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>, Robert Haas <robertmhaas(at)gmail(dot)com>, Haribabu Kommi <kommi(dot)haribabu(at)gmail(dot)com>, Michael Paquier <michael(dot)paquier(at)gmail(dot)com>, Amit Langote <Langote_Amit_f8(at)lab(dot)ntt(dot)co(dot)jp>, Thomas Munro <thomas(dot)munro(at)enterprisedb(dot)com>, David Steele <david(at)pgmasters(dot)net>, Claudio Freire <klaussfreire(at)gmail(dot)com>, Simon Riggs <simon(at)2ndquadrant(dot)com>, Pavan Deolasee <pavan(dot)deolasee(at)gmail(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [HACKERS] Block level parallel vacuum
Date: 2019-10-12 09:15:09
Message-ID: CAKYtNAqmG37pZWnA+6VWDABRVYRB9N545tmZgpGfJFpPiNwdXw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Thanks Amit for patch.

Crash is fixed by this patch.

Thanks and Regards
Mahendra Thalor

On Sat, Oct 12, 2019, 09:03 Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:

> On Fri, Oct 11, 2019 at 4:47 PM Mahendra Singh <mahi6run(at)gmail(dot)com> wrote:
> >
> >
> > I did some analysis and found that we are trying to free some already
> freed memory. Or we are freeing palloced memory in vac_update_relstats.
> > for (i = 0; i < nindexes; i++)
> > {
> > if (stats[i] == NULL || stats[i]->estimated_count)
> > continue;
> >
> > /* Update index statistics */
> > vac_update_relstats(Irel[i],
> > stats[i]->num_pages,
> > stats[i]->num_index_tuples,
> > 0,
> > false,
> > InvalidTransactionId,
> > InvalidMultiXactId,
> > false);
> > pfree(stats[i]);
> > }
> >
> > As my table have 2 indexes, so we have to free both stats. When i = 0,
> it is freeing propery but when i = 1, then vac_update_relstats is freeing
> memory.
> >>
> >> (gdb) p *stats[i]
> >> $1 = {num_pages = 218, pages_removed = 0, estimated_count = false,
> num_index_tuples = 30000, tuples_removed = 30000, pages_deleted = 102,
> pages_free = 0}
> >> (gdb) p *stats[i]
> >> $2 = {num_pages = 0, pages_removed = 65536, estimated_count = false,
> num_index_tuples = 0, tuples_removed = 0, pages_deleted = 0, pages_free = 0}
> >> (gdb)
> >
> >
> > From above data, it looks like, somewhere inside vac_update_relstats, we
> are freeing all palloced memory. I don't know, why is it.
> >
>
> I don't think the problem is in vac_update_relstats as we are not even
> passing stats to it, so it won't be able to free it. I think the real
> problem is in the way we copy the stats from shared memory to local
> memory in the function end_parallel_vacuum(). Basically, it allocates
> the memory for all the index stats together and then in function
> update_index_statistics, it is trying to free memory of individual
> array elements, that won't work. I have tried to fix the allocation
> in end_parallel_vacuum, see if this fixes the problem for you. You
> need to apply the attached patch atop
> v28-0001-Add-parallel-option-to-VACUUM-command posted above by
> Sawada-San.
>
> --
> With Regards,
> Amit Kapila.
> EnterpriseDB: http://www.enterprisedb.com
>

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Masahiko Sawada 2019-10-12 10:19:05 Re: Transparent Data Encryption (TDE) and encrypted files
Previous Message Michael Paquier 2019-10-12 06:08:41 Re: dropping column prevented due to inherited index