From: | Robert Haas <robertmhaas(at)gmail(dot)com> |
---|---|
To: | Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com> |
Cc: | Jim Nasby <Jim(dot)Nasby(at)bluetreble(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>, Amit Langote <Langote_Amit_f8(at)lab(dot)ntt(dot)co(dot)jp> |
Subject: | Re: Way to check whether a particular block is on the shared_buffer? |
Date: | 2016-02-13 04:46:14 |
Message-ID: | CA+TgmoaHvQm4H405m3YpZvsdLJMpeGBj4tpAzBhXctXyQHgh5Q@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Thu, Feb 11, 2016 at 9:05 PM, Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com> wrote:
> Hmm. In my experience, it is often not a productive discussion whether
> a feature is niche or commodity. So, let me change the viewpoint.
>
> We may utilize OS-level locking mechanism here.
>
> Even though it depends on filesystem implementation under the VFS,
> we may use inode->i_mutex lock that shall be acquired during the buffer
> copy from user to kernel, at least, on a few major filesystems; ext4,
> xfs and btrfs in my research. As well, the modified NVMe SSD driver can
> acquire the inode->i_mutex lock during P2P DMA transfer.
>
> Once we can consider the OS buffer is updated atomically by the lock,
> we don't need to worry about corrupted pages, but still needs to pay
> attention to the scenario when updated buffer page is moved to GPU.
>
> In this case, PD_ALL_VISIBLE may give us a hint. GPU side has no MVCC
> infrastructure, so I intend to move all-visible pages only.
> If someone updates the buffer concurrently, then write out the page
> including invisible tuples, PD_ALL_VISIBLE flag shall be cleared because
> updated tuples should not be visible to the transaction which issued
> P2P DMA.
>
> Once GPU met a page with !PD_ALL_VISIBLE, it can return an error status
> that indicates CPU to retry this page again. In this case, this page is
> likely loaded to the shared buffer already, so retry penalty is not so
> much.
>
> I'll try to investigate the implementation in this way.
> Please correct me, if I misunderstand something (especially, treatment
> of PD_ALL_VISIBLE).
I suppose there's no theoretical reason why the buffer couldn't go
from all-visible to not-all-visible and back to all-visible again all
during the time you are copying it.
Honestly, I think trying to access buffers without going through
shared_buffers is likely to be very hard to make correct and probably
a loser. Copying the data into shared_buffers and then to the GPU is,
doubtless, at least somewhat slower. But I kind of doubt that it's
enough slower to make up for all of the problems you're going to have
with the approach you've chosen.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
From | Date | Subject | |
---|---|---|---|
Next Message | Robert Haas | 2016-02-13 04:47:05 | Re: Small PATCH: check of 2 Perl modules |
Previous Message | Robert Haas | 2016-02-13 04:40:19 | Re: Speed up Clog Access by increasing CLOG buffers |