Re: Streaming I/O, vectored I/O (WIP)

From: Nazir Bilal Yavuz <byavuz81(at)gmail(dot)com>
To: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
Cc: Heikki Linnakangas <hlinnaka(at)iki(dot)fi>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>, Andres Freund <andres(at)anarazel(dot)de>, Melanie Plageman <melanieplageman(at)gmail(dot)com>
Subject: Re: Streaming I/O, vectored I/O (WIP)
Date: 2024-04-07 21:01:26
Message-ID: CAN55FZ1k1oQVF++UFGruKqna_hw8bTNPkbx6DVpANqs1=1qLqg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

On Sun, 7 Apr 2024 at 20:33, Nazir Bilal Yavuz <byavuz81(at)gmail(dot)com> wrote:
>
> Hi,
>
> On Tue, 2 Apr 2024 at 11:40, Thomas Munro <thomas(dot)munro(at)gmail(dot)com> wrote:
> >
> > I had been planning to commit v14 this morning but got cold feet with
> > the BMR-based interface. Heikki didn't like it much, and in the end,
> > neither did I. I have now removed it, and it seems much better. No
> > other significant changes, just parameter types and inlining details.
> > For example:
> >
> > * read_stream_begin_relation() now takes a Relation, likes its name says
> > * StartReadBuffers()'s operation takes smgr and optional rel
> > * ReadBuffer_common() takes smgr and optional rel
>
> Read stream objects can be created only using Relations now. There
> could be read stream users which do not have a Relation but
> SMgrRelations. So, I created another constructor for the read streams
> which use SMgrRelations instead of Relations. Related patch is
> attached.

After sending this, I realized that I forgot to add persistence value
to the new constructor. While working on it I also realized that
current code sets persistence in PinBufferForBlock() function and this
function is called for each block, which can be costly. So, I moved
setting persistence to the out of PinBufferForBlock() function.

Setting persistence outside of the PinBufferForBlock() function (0001)
and creating the new constructor that uses SMgrRelations (0002) are
attached.

--
Regards,
Nazir Bilal Yavuz
Microsoft

Attachment Content-Type Size
0001-Refactor-PinBufferForBlock-to-remove-if-checks-about.patch text/x-patch 3.6 KB
0002-Add-a-way-to-create-read-stream-object-by-using-SMgr.patch text/x-patch 5.6 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Andres Freund 2024-04-07 21:09:24 Re: Add bump memory context type and use it for tuplesorts
Previous Message Melanie Plageman 2024-04-07 20:59:26 Re: Use streaming read API in ANALYZE