Re: Proposal for enabling auto-vectorization for checksum calculations

From: Andrew Kim <tenistarkim(at)gmail(dot)com>
To: John Naylor <johncnaylorls(at)gmail(dot)com>
Cc: pgsql-hackers(at)lists(dot)postgresql(dot)org, Oleg Tselebrovskiy <o(dot)tselebrovskiy(at)postgrespro(dot)ru>
Subject: Re: Proposal for enabling auto-vectorization for checksum calculations
Date: 2025-11-05 23:49:47
Message-ID: CAK64mnc6jbehHv5AHc84tVFRJg4zeMiFuvPX9xZkRpq0210MFA@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi John,

Thank you for reviewing and bringing this up regarding checksum architecture.

On Tue, Oct 28, 2025 at 7:50 PM John Naylor <johncnaylorls(at)gmail(dot)com> wrote:
>
> On Fri, Oct 24, 2025 at 2:49 PM Andrew Kim <tenistarkim(at)gmail(dot)com> wrote:
> > The function signatures (pg_checksum_block, pg_checksum_page) remain
> > identical, and checksum_impl.h still contains the complete
> > implementation that external programs can include. The runtime
> > dispatch only affects internal PostgreSQL usage.
>
> I don't quite understand the architecture here -- all
> platform-specific definitions were put in the "checksum_impl.h"
> header. My thinking was that checksum.c would have all that, with thin
> wrappers around the functions included from that header.

The v9 patch series is attached.
I've implemented the architecture as you described.
checksum_impl.h
-No platform-specific code (removed all AVX2, CPUID, intrinsics)
-External programs get a clean, portable standalone implementation
-Uses #ifndef PG_CHECKSUM_INTERNAL guard to prevent conflicts

checksum.c (full implementation for checksum):
-Includes checksum_impl.h for the basic implementation and common definitions
-Contains all platform-specific code (AVX2, CPUID detection, runtime dispatch)
-Implements thin wrapper functions that provide the public interface
-Uses #define PG_CHECKSUM_INTERNAL before including the header

>
> --
> John Naylor
> Amazon Web Services

Thanks
Andrew

Attachment Content-Type Size
v9-0001-Move-checksum-functions-from-backend-storage-to-port.patch application/octet-stream 8.8 KB
v9-0002-Move-platform-specific-checksum-code-from-header-to-.patch application/octet-stream 14.8 KB
v9-0003-Benchmark-code-for-postgres-checksums.patch application/octet-stream 5.0 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Chao Li 2025-11-05 23:59:18 Re: doc: Improve description of io_combine_limit and io_max_combine_limit GUCs
Previous Message Chao Li 2025-11-05 23:37:37 Re: Suggestion to add --continue-client-on-abort option to pgbench