| From: | Xuneng Zhou <xunengzhou(at)gmail(dot)com> |
|---|---|
| To: | Nazir Bilal Yavuz <byavuz81(at)gmail(dot)com> |
| Cc: | pgsql-hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
| Subject: | Re: Streamify more code paths |
| Date: | 2025-12-30 02:43:17 |
| Message-ID: | CABPTF7W-f_zPN442FCp4Xaopi721oDmGYimq=VhAk=F7jwYZDQ@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hi,
On Tue, Dec 30, 2025 at 9:51 AM Xuneng Zhou <xunengzhou(at)gmail(dot)com> wrote:
>
> Hi,
>
> Thanks for looking into this.
>
> On Mon, Dec 29, 2025 at 6:58 PM Nazir Bilal Yavuz <byavuz81(at)gmail(dot)com> wrote:
> >
> > Hi,
> >
> > On Sun, 28 Dec 2025 at 14:46, Xuneng Zhou <xunengzhou(at)gmail(dot)com> wrote:
> > >
> > > Hi,
> > > >
> > > > Two more to go:
> > > > patch 5: Streamify log_newpage_range() WAL logging path
> > > > patch 6: Streamify hash index VACUUM primary bucket page reads
> > > >
> > > > Benchmarks will be conducted soon.
> > > >
> > >
> > > v6 in the last message has a problem and has not been updated. Attach
> > > the right one again. Sorry for the noise.
> >
> > 0003 and 0006:
> >
> > You need to add 'StatApproxReadStreamPrivate' and
> > 'HashBulkDeleteStreamPrivate' to the typedefs.list.
>
> Done.
>
> > 0005:
> >
> > @@ -1321,8 +1341,10 @@ log_newpage_range(Relation rel, ForkNumber forknum,
> > nbufs = 0;
> > while (nbufs < XLR_MAX_BLOCK_ID && blkno < endblk)
> > {
> > - Buffer buf = ReadBufferExtended(rel, forknum, blkno,
> > - RBM_NORMAL, NULL);
> > + Buffer buf = read_stream_next_buffer(stream, NULL);
> > +
> > + if (!BufferIsValid(buf))
> > + break;
> >
> > We are loosening a check here, there should not be a invalid buffer in
> > the stream until the endblk. I think you can remove this
> > BufferIsValid() check, then we can learn if something goes wrong.
>
> My concern before for not adding assert at the end of streaming is the
> potential early break in here:
>
> /* Nothing more to do if all remaining blocks were empty. */
> if (nbufs == 0)
> break;
>
> After looking more closely, it turns out to be a misunderstanding of the logic.
>
> > 0006:
> >
> > You can use read_stream_reset() instead of read_stream_end(), then you
> > can use the same stream with different variables, I believe this is
> > the preferred way.
> >
> > Rest LGTM!
> >
>
> Yeah, reset seems a more proper way here.
>
Run pgindent using the updated typedefs.list.
--
Best,
Xuneng
| Attachment | Content-Type | Size |
|---|---|---|
| v4-0001-Switch-Bloom-scan-paths-to-streaming-read.-n-nRep.patch | application/octet-stream | 2.4 KB |
| v4-0005-Streamify-log_newpage_range-WAL-logging-path.patch | application/octet-stream | 2.3 KB |
| v4-0004-Replace-synchronous-ReadBufferExtended-loop-with-.patch | application/octet-stream | 2.5 KB |
| v4-0002-Streamify-Bloom-VACUUM-paths.-n-nUse-streaming-re.patch | application/octet-stream | 4.2 KB |
| v4-0006-Streamify-hash-index-VACUUM-primary-bucket-page-r.patch | application/octet-stream | 5.5 KB |
| v4-0003-Streamify-heap-bloat-estimation-scan.-Introduce-a.patch | application/octet-stream | 6.4 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Naga Appani | 2025-12-30 02:57:11 | Re: [Proposal] Expose internal MultiXact member count function for efficient monitoring |
| Previous Message | Xuneng Zhou | 2025-12-30 02:42:09 | Re: Implement waiting for wal lsn replay: reloaded |