Re: BitmapHeapScan streaming read user and prelim refactoring

From: Melanie Plageman <melanieplageman(at)gmail(dot)com>
To: Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>
Cc: Andres Freund <andres(at)anarazel(dot)de>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org>, Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, Heikki Linnakangas <hlinnaka(at)iki(dot)fi>, Nazir Bilal Yavuz <byavuz81(at)gmail(dot)com>
Subject: Re: BitmapHeapScan streaming read user and prelim refactoring
Date: 2024-02-28 14:38:54
Message-ID: CAAKRu_ar_C+RCMV_LrCL10ccWA0n-0rZcxfhL67hXFqREpPRfg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Wed, Feb 28, 2024 at 8:22 AM Tomas Vondra
<tomas(dot)vondra(at)enterprisedb(dot)com> wrote:
>
> Hi,
>
> I haven't looked at the code very closely yet, but I decided to do some
> basic benchmarks to see if/how this refactoring affects behavior.
>
> Attached is a simple .sh script that
>
> 1) creates a table with one of a couple basic data distributions
> (uniform, linear, ...), with an index on top
>
> 2) runs a simple query with a where condition matching a known fraction
> of the table (0 - 100%), and measures duration
>
> 3) the query is forced to use bitmapscan by disabling other options
>
> 4) there's a couple parameters the script varies (work_mem, parallel
> workers, ...), the script drops caches etc.
>
> 5) I only have results for table with 1M rows, which is ~320MB, so not
> huge. I'm running this for larger data set, but that will take time.
>
>
> I did this on my two "usual" machines - i5 and xeon. Both have flash
> storage, although i5 is SATA and xeon has NVMe. I won't share the raw
> results, because the CSV is like 5MB - ping me off-list if you need the
> file, ofc.

I haven't looked at your results in detail yet. I plan to dig into
this more later today. But, I was wondering if it was easy for you to
run the shorter tests on just the commits before the last
https://github.com/melanieplageman/postgres/tree/bhs_pgsr
i.e. patches 0001-0013. Patch 0014 implements the streaming read user
and removes all of the existing prefetch code. I would be interested
to know if the behavior with just the preliminary refactoring differs
at all.

- Melanie

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tomas Vondra 2024-02-28 14:56:06 Re: BitmapHeapScan streaming read user and prelim refactoring
Previous Message Jacob Champion 2024-02-28 14:05:52 Re: [PoC] Federated Authn/z with OAUTHBEARER