Skip site navigation (1) Skip section navigation (2)

Improper usage of MemoryContext in nodeSubplan.c for buildSubPlanHash() function. This maybe causes allocate memory failed.

From: "Tao Ma" <feng_eden(at)163(dot)com>
To: pgsql-hackers(at)postgresql(dot)org
Subject: Improper usage of MemoryContext in nodeSubplan.c for buildSubPlanHash() function. This maybe causes allocate memory failed.
Date: 2010-07-26 10:17:21
Message-ID: i2jnbo$1lcj$1@news.hub.org (view raw or flat)
Thread:
Lists: pgsql-hackers
Hi all,

This is a potential memory error in nodeSubplan.c or execGrouping.c
Using select '1'::TEXT IN ((SELECT '1'::NAME) UNION ALL SELECT '1'::NAME);
to reproduce this bug.

You may see the memory content that slot1's tts_values[0] point to before 
and after the statement : MemoryContextReset(evalContext) of 
execTuplesMatch() in PG version with --enable-cassert switch on.  You will 
find that the memory is set to 0x7f7f7f7f.

Here are my code analysis:
For the executor node SubPlanState, The node has a ExprContext named 
'innerecontext' that is created by itself in function ExecInitSubPlan(). 
Also, the 'innerecontext' is used to build a project info, 'projRight', of 
SubPlanState node.

When the SubPlanState node is executed, a hash table of subplan will be 
built, so buildSubPlanHash() will be called. In buildSubPlanHash function, 
'hashtable' of SubPlanState will be initialized by calling 
BuildTupleHashTable(), and the code passes the
'ecxt_per_tuple_memory' of 'innerecontext' to BuildTupleHashTable() as the 
'hashtable''s tempcxt.

At this point, we can conclude that the 'projRight' and 'hashtable''s 
'tempcxt' of SubPlanState node are all using the same memory context, that 
is 'innerecontext' in SubPlanState node. So:
1) The memory of all the fetched tuples from 'projRight' will be located in 
'innerecontext' of SubPlanState node.
2) All other intermediate result that is located in 'hashtable''s tempcxt, 
also in fact, will reside in the 'innerecontext' of SubPlanState node.

Now next:
In buildSubPlanHash(), we will fetch tuple from  'projRight'  to fill the 
hash table of SubPlanState node. As we known, all the fetched tuples are 
located in the 'innerecontext'. When filling the tuple hash table, if the 
tuple already exists in the hash table of SubPlanState node, the match 
function execTuplesMatch() will be called by tuples matching, but in this 
function, the statement:
MemoryContextReset(evalContext);
will be reset 'evalContext', to free some memory usage. In fact, the 
'evalContext' is equal to 'innerecontext' that the fetched tuples are 
located in, and actually this statement will reset the 'innerecontext'. So 
the fetched tuple's memory are all lost. That's the problem.

In fact, this error only in debug version, but not in release version. In 
debug using --enable-cassert to configure, the memory will be set to 
0x7f7f7f7f, but not for release. For this error memory usage, the pg does 
not always report error because the 0x7f7f7f in testing Macro 
VARATT_IS_COMPRESSED() and VARATT_IS_EXTERNAL() are always false in 
!WORDS_BIGENDIAN platform such as i386 GNU Linux and in WORDS_BIGENDIAN, the 
testing Macro VARATT_IS_COMPRESSED() will be true and error may be reported 
such as AIX(Power 6 AIX V6.1).

To fix this problem, we can use another memory context to passin 
BuildTupleHashTable() as the hashtable's tempcxt, or use other memory 
context as hash table's tempcxt or other ways.


Any questions, please contact me.
Regards




Responses

pgsql-hackers by date

Next:From: Artur DabrowskiDate: 2010-07-26 10:20:16
Subject: Re: Incorrect FTS result with GIN index
Previous:From: Yeb HavingaDate: 2010-07-26 09:36:19
Subject: Re: Synchronous replication

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group