Re: Zedstore - compressed in-core columnar storage

From: Taylor Vesely <tvesely(at)pivotal(dot)io>
To: Ashutosh Sharma <ashu(dot)coek88(at)gmail(dot)com>
Cc: Alexandra Wang <lewang(at)pivotal(dot)io>, Heikki Linnakangas <hlinnaka(at)iki(dot)fi>, Ashwin Agrawal <aagrawal(at)pivotal(dot)io>, DEV_OPS <devops(at)ww-it(dot)cn>, PostgreSQL mailing lists <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Zedstore - compressed in-core columnar storage
Date: 2019-09-20 00:18:19
Message-ID: CAFaX_4LAHx1H_GMs2d80DtYB4t8mJ_oU6hN3PmG_QjJ6tEzCQQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> When doing update operation, for each tuple being modified,
> *tuplebuffers_insert()* says that there is no entry for the relation
> being modified in the hash table although it was already added when
> the first tuple in the table was updated. Why is it so?

Currently, when doing an update, it will actually flush the tuple
buffers every time we update a tuple. As a result, we only ever spool
up one tuple at a time. This is a good place to put in an optimization
like was implemented for insert, but I haven't gotten around to
looking into that yet.

The memory leak is actually happening because it isn't freeing the
attbuffers after flushing. Alexandra Wang and I have a working
branch[1] where we tried to plug the leak by freeing the attbuffers,
but it has exposed an issue with triggers that I need to understand
before I push the fix into the main zedstore branch.

I don't like our solution of freeing the buffers either, because they
could easily be reused. I'm going to take a stab at making that better
before merging in the fix.

[1] https://github.com/l-wang/postgres-1/tree/zedstore-fix-memory-issues

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Jeff Davis 2019-09-20 00:40:15 Re: Add "password_protocol" connection parameter to libpq
Previous Message Michael Paquier 2019-09-20 00:16:58 Re: [PATCH] src/test/modules/dummy_index -- way to test reloptions from inside of access method