Re: Reducing Memory Consumption (aset and generation)

Em seg., 6 de jun. de 2022 às 21:14, Ranier Vilela <ranier(dot)vf(at)gmail(dot)com>
escreveu:

> Em seg., 6 de jun. de 2022 às 20:37, David Rowley <dgrowleyml(at)gmail(dot)com>
> escreveu:
>
>> On Mon, 6 Jun 2022 at 07:28, Ranier Vilela <ranier(dot)vf(at)gmail(dot)com> wrote:
>> > 4) 004-generation-reduces-memory-consumption-BUG.patch
>> > Same to the (2), but with BUG.
>> > It only takes a few tweaks to completely remove the field block.
>>
>> > This fails with make check.
>> > I couldn't figure out why it doesn't work with 16 bits (struct
>> GenerationChunk).
>>
>> Hi David, thanks for taking a look at this.
>
>
>> I think you're misunderstanding how blocks and chunks work here. A
>> block can have many chunks. You can't find the block that a chunk is
>> on by subtracting Generation_BLOCKHDRSZ from the pointer given to
>> GenerationFree(). That would only work if the chunk happened to be the
>> first chunk on a block. If it's anything apart from that then you'll
>> be making adjustments to the memory of some prior chunk on the block.
>> I imagine this is the reason you can't get the tests to pass.
>>
> Ok, I am still learning about this.
> Can you explain why subtracting Generation_BLOCKHDRSZ from the pointer,
> works for sizeof(struct GenerationChunk) = 24 bits,
> When all references for the block field have been removed.
> This pass check-world.
>
>
>>
>> Can you also explain why you think moving code around randomly or
>> adding unlikely() macros helps reduce the memory consumption overheads
>> of generation contexts?
>
> Of course, those changes do not reduce memory consumption.
> But, IMO, I think those changes improve the access to memory regions,
> because of the locality of the data.
>
> About "unlikely macros", this helps the branchs prediction, when most of
> the time,
> malloc and related functions, will not fail.
>
>
>> I imagine you think that's helping to further
>> improve performance, but you've not offered any evidence of that
>> separately from the other changes you've made. If you think those are
>> useful changes then I recommend you run individual benchmarks and
>> offer those as proof that those changes are worthwhile.
>>
> Ok, I can understand, are changes unrelated.
>
Let's restart this, to simplify the review and commit work.

Regarding the patches now, we have:
1) v1-001-aset-reduces-memory-consumption.patch
Reduces memory used by struct AllocBlockData by minus 8 bits,
reducing the total size to 32 bits, which leads to "fitting" two structs in
a 64bit cache.

Remove tests elog(ERROR, "could not find block containing chunk %p" and
elog(ERROR, "could not find block containing chunk %p", moving them to
MEMORY_CONTEXT_CHECKING context.

Since 8.2 versions, nobody complains about these tests.

But if is not acceptable, have the option (3)
v1-003-aset-reduces-memory-consumption.patch

2) v1-002-generation-reduces-memory-consumption.patch
Reduces memory used by struct GenerationBlock, by minus 8 bits,
reducing the total size to 32 bits, which leads to "fitting" two structs in
a 64bit cache.

3) v1-003-aset-reduces-memory-consumption.patch
Same to the (1), but without remove the tests:
elog(ERROR, "could not find block containing chunk %p" and
elog(ERROR, "could not find block containing chunk %p",
But at the cost of removing a one tiny part of the tests and
moving them to MEMORY_CONTEXT_CHECKING context.

Since 8.2 versions, nobody complains about these tests.

regards,
Ranier Vilela