Re: Values list-of-targetlists patch for comments (was Re:

Joe Conway <mail(at)joeconway(dot)com> writes:
> Tom Lane wrote:
>> I wonder whether there is any reasonable way to determine which data
>> structures are responsible for how much space ... in my test I'm seeing

> I was doing it by sprinkling MemoryContextStats() in various places.

Yeah, I've just been doing that and some hand analysis too. What I get
(on a 64-bit machine) is that essentially all the space goes into

lists of A_Const lists: 320000000
lists of Const lists: 320000000
transformInsertRow extra lists: 144000000

I think we could safely list_free the input list in transformInsertRow
as your patch suggests, which would buy back the 144M part. But I don't
believe it's safe at all to free the raw_parser output --- the grammar
sometimes makes multiple links to the same subtree, eg in BETWEEN.
In any case the patch as proposed wouldn't catch all the detritus for
any case more complicated than a simple integer constant.

The way that the list memory usage works (again, 64-bit machine) is

sizeof(List) = 24
sizeof(ListCell) = 16
sizeof(A_Const) = 32

Each of these nodes will have 16 bytes palloc overhead, and the List
header will be rounded up to 32 bytes as well, so we have total space
for a 3-element integer list of
32+16 + (16+16 + 32+16) * 3
Add in 16+16 for the associated ListCell of the top list-of-lists,
and you come to 320 bytes per sublist. Const happens to also be
32 bytes so the transformed lists are the same size.

It's interesting to reflect on the fact that this comes to 184 bytes
of useful data and 136 bytes of palloc overhead per row ... not sure
if we can do much about the overhead though.

regards, tom lane