| From: | Hannu Krosing <hannuk(at)google(dot)com> |
|---|---|
| To: | Jakub Wartak <jakub(dot)wartak(at)enterprisedb(dot)com> |
| Cc: | ahmed <gouda0x(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
| Subject: | Re: COPY TO BLACKHOLE / pg_dump -j -Fb |
| Date: | 2026-07-30 11:12:57 |
| Message-ID: | CAMT0RQR5_CQ3NsVvmHePMqF2etQoJSeFrTiMcCvXVkMHT8rh=w@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
To force full construction of tuples I usually use
SELECT sum(hashtext(t::text) FROM <tablename> AS t;
For "BLACKHOLE" you can use '/dev/null' most of the time, so
COPY tablename TO '/dev/null';
On Thu, Jul 30, 2026 at 9:00 AM Jakub Wartak <jakub(dot)wartak(at)enterprisedb(dot)com>
wrote:
> On Wed, Jul 29, 2026 at 4:09 PM ahmed <gouda0x(at)gmail(dot)com> wrote:
> >
> > Hi Jakub, The use case that this feature addresses is interesting, but
> the proposed implementation feels a bit ad-hoc to us:
> >
> > 1. `COPY ... TO ...` implies moving data from one place to another.
> Having a `BLACKHOLE` destination whose purpose is to check for data
> corruption seems counter-intuitive.
>
> > 2. The argument to `pg_dump --format=...` specifies an archive format. A
> `blackhole` archive format and the `archBlackhole` entry in the
> `ArchiveFormat` enum in the `pg_dump` code also seem ad-hoc and counter-
> intuitive, because nothing is actually being archived.
>
> > For those reasons, I don't think `COPY` or `pg_dump` is the right place
> to support this use case. Perhaps this needs a new command, but that might
> overcomplicate things. In that case, I would prefer to leave things as they
> are.
>
> Hi Ahmed & Milosz,
>
> thanks for reminding me of this old thread :)
>
> I partially agree. Let me explain, this was based on minimal changes
> approach
> to have it in and re-use parallel scheduling for this SQL syntax in pg_dump
> (the primary idea is to have one simple one-liner command that people could
> use to logically check their databases somewhat similiar to pg_amcheck, but
> using more-rigorious "COPY" than amcheck functions do itself; to this day
> people are using pg_dump > /dev/null for this but it really writes and
> sadly
> cannot use --jobs).
>
> Perhaps we should have some other command or even amcheck function to do
> the
> same, but that would re-implementation of everything that actually COPY
> does,
> so maybe instead it would be good idea to have COPY .. TO BLACKHOLE (or
> COPY
> VERIFY %tablename%?), but support launching many jobs using pg_amcheck
> --jobs
> rather than having pg_dump --jobs (and as You mention this is not
> archiving),
> as it would be more suited for logical database verification. I'm pretty
> open
> for speciifc naming proposals if anyone has those, before writing new
> version
> of the patch.
>
> -J.
>
>
>
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Hannu Krosing | 2026-07-30 11:16:36 | Re: Support for 8-byte TOAST values, round two |
| Previous Message | Shlok Kyal | 2026-07-30 11:06:42 | Re: Bug in ALTER SUBSCRIPTION ... SERVER / ... CONNECTION with broken old server |