Re: MMAP Buffers

On Fri, 15 Apr 2011 14:33:37 +0300, Heikki Linnakangas wrote:
> On 15.04.2011 13:32, Radosław Smogura wrote:
>> If I may, I want to share some concept to use mmap in PG. It's far,
>> far
>> away from perfect, but it's keeps WAL before data. As well I crated
>> table, with index, inserted few values, and I done vacuum full on
>> this
>> table. Db inits welcome from orginal sources.
>>
>> Performance of read (if backend is loaded) is really good, query
>> time
>> goes down from 450ms to about 410ms. Update may be slower - but work
>> is
>> in progress (I will start with write, as I went to point when simple
>> updates may be performed). Even that I didn't covered all aspects
>> off
>> updating, it's simple to do it, just to call PreopareBufferToUpdate
>> before modifing buffer, ofc some ideas of increasing this are still
>> in
>> my head.
>>
>> Any comments, suggestions welcome.
>
> The patch is quite hard to read because of random whitespace changes
> and other stylistic issues, but I have a couple of high-level
> questions on the design:
Yes, but, hmm... in Netbeans I had really long gaps (probably 8 spaces,
from tabs), so deeper "ifs", comments at the and of variables, went of
out my screen. I really wanted to not format this, but sometimes I
needed.

> * Does each process have its own mmappings, or are the mmapping done
> to shared buffers?
Those are MAP_SHARED mappings, but each process has it's own pointer to
this.

> * How do you handle locking?
I do not do locking... I do different thing (worst)... When buffer
should be updated, it gets shared buffer, content is copied (so
situation almost same like fread), and depending on situation content is
used directly or pages between mmaping and shared (updatable) regions
are swapped - it keeps tuple pointers, etc. I really would be happy if
such method (lock flushing to file) could exists.

> Do you still need to allocate a shared buffer for each mmapped page?
Currently each mmaped page has additional shared buffer, but it's
almost ready to use independent pool of shared buffers. This will be
good, as mmaped buffers could cover whole system cache, keeping maybe
10%-20% of this size for write in SHMEM.

I think about MAP_PRIVATE, but those has some pluses and minuses, e.g.
MAP_SHARED may be, for less critical systems, simplier equipped with GUC
mmap_direct_write=true.

Regards,
Radek