Re: COPY TO BLACKHOLE / pg_dump -j -Fb

From: Jakub Wartak <jakub(dot)wartak(at)enterprisedb(dot)com>
To: ahmed <gouda0x(at)gmail(dot)com>
Cc: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: COPY TO BLACKHOLE / pg_dump -j -Fb
Date: 2026-07-30 06:59:56
Message-ID: CAKZiRmz6aWmf6Kg8y91sZBgzB455dAhfLx8w=QtkdcQpN2eNbQ@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Wed, Jul 29, 2026 at 4:09 PM ahmed <gouda0x(at)gmail(dot)com> wrote:
>
> Hi Jakub, The use case that this feature addresses is interesting, but the proposed implementation feels a bit ad-hoc to us:
>
> 1. `COPY ... TO ...` implies moving data from one place to another. Having a `BLACKHOLE` destination whose purpose is to check for data corruption seems counter-intuitive.

> 2. The argument to `pg_dump --format=...` specifies an archive format. A `blackhole` archive format and the `archBlackhole` entry in the `ArchiveFormat` enum in the `pg_dump` code also seem ad-hoc and counter- intuitive, because nothing is actually being archived.

> For those reasons, I don't think `COPY` or `pg_dump` is the right place to support this use case. Perhaps this needs a new command, but that might overcomplicate things. In that case, I would prefer to leave things as they are.

Hi Ahmed & Milosz,

thanks for reminding me of this old thread :)

I partially agree. Let me explain, this was based on minimal changes approach
to have it in and re-use parallel scheduling for this SQL syntax in pg_dump
(the primary idea is to have one simple one-liner command that people could
use to logically check their databases somewhat similiar to pg_amcheck, but
using more-rigorious "COPY" than amcheck functions do itself; to this day
people are using pg_dump > /dev/null for this but it really writes and sadly
cannot use --jobs).

Perhaps we should have some other command or even amcheck function to do the
same, but that would re-implementation of everything that actually COPY does,
so maybe instead it would be good idea to have COPY .. TO BLACKHOLE (or COPY
VERIFY %tablename%?), but support launching many jobs using pg_amcheck --jobs
rather than having pg_dump --jobs (and as You mention this is not archiving),
as it would be more suited for logical database verification. I'm pretty open
for speciifc naming proposals if anyone has those, before writing new version
of the patch.

-J.

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Masahiko Sawada 2026-07-30 07:06:29 Re: Fix "unexpected logical decoding status change" error; from concurrent logical decoding activation
Previous Message Tristan Partin 2026-07-30 06:51:32 Fix hashchar() and hashcharextended() to not depend on char signedness