Re: COPY TO BLACKHOLE / pg_dump -j -Fb

From: Hannu Krosing <hannuk(at)google(dot)com>
To: Jakub Wartak <jakub(dot)wartak(at)enterprisedb(dot)com>
Cc: ahmed <gouda0x(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: COPY TO BLACKHOLE / pg_dump -j -Fb
Date: 2026-07-30 11:12:57
Message-ID: CAMT0RQR5_CQ3NsVvmHePMqF2etQoJSeFrTiMcCvXVkMHT8rh=w@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

To force full construction of tuples I usually use

SELECT sum(hashtext(t::text) FROM <tablename> AS t;

For "BLACKHOLE" you can use '/dev/null' most of the time, so

COPY tablename TO '/dev/null';

On Thu, Jul 30, 2026 at 9:00 AM Jakub Wartak <jakub(dot)wartak(at)enterprisedb(dot)com>
wrote:

> On Wed, Jul 29, 2026 at 4:09 PM ahmed <gouda0x(at)gmail(dot)com> wrote:
> >
> > Hi Jakub, The use case that this feature addresses is interesting, but
> the proposed implementation feels a bit ad-hoc to us:
> >
> > 1. `COPY ... TO ...` implies moving data from one place to another.
> Having a `BLACKHOLE` destination whose purpose is to check for data
> corruption seems counter-intuitive.
>
> > 2. The argument to `pg_dump --format=...` specifies an archive format. A
> `blackhole` archive format and the `archBlackhole` entry in the
> `ArchiveFormat` enum in the `pg_dump` code also seem ad-hoc and counter-
> intuitive, because nothing is actually being archived.
>
> > For those reasons, I don't think `COPY` or `pg_dump` is the right place
> to support this use case. Perhaps this needs a new command, but that might
> overcomplicate things. In that case, I would prefer to leave things as they
> are.
>
> Hi Ahmed & Milosz,
>
> thanks for reminding me of this old thread :)
>
> I partially agree. Let me explain, this was based on minimal changes
> approach
> to have it in and re-use parallel scheduling for this SQL syntax in pg_dump
> (the primary idea is to have one simple one-liner command that people could
> use to logically check their databases somewhat similiar to pg_amcheck, but
> using more-rigorious "COPY" than amcheck functions do itself; to this day
> people are using pg_dump > /dev/null for this but it really writes and
> sadly
> cannot use --jobs).
>
> Perhaps we should have some other command or even amcheck function to do
> the
> same, but that would re-implementation of everything that actually COPY
> does,
> so maybe instead it would be good idea to have COPY .. TO BLACKHOLE (or
> COPY
> VERIFY %tablename%?), but support launching many jobs using pg_amcheck
> --jobs
> rather than having pg_dump --jobs (and as You mention this is not
> archiving),
> as it would be more suited for logical database verification. I'm pretty
> open
> for speciifc naming proposals if anyone has those, before writing new
> version
> of the patch.
>
> -J.
>
>
>

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Hannu Krosing 2026-07-30 11:16:36 Re: Support for 8-byte TOAST values, round two
Previous Message Shlok Kyal 2026-07-30 11:06:42 Re: Bug in ALTER SUBSCRIPTION ... SERVER / ... CONNECTION with broken old server