Re: Request For Feature: pg_dump

Am 22.05.26 um 19:20 schrieb Ron Johnson:
> On Fri, May 22, 2026 at 12:53 PM Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>
> Ron Johnson <ronljohnsonjr(at)gmail(dot)com> writes:
> > In --format=directory mode, remove .dat files with zero data
> records, and
> > mark that table's toc.dat entry that it's an empty table.
>
> > Justification: *lots* of empty tables means *lots* of teeny-tiny
> files in
> > the DB's dump directory.  That unnecessarily bloats the fs, and
> makes "du
> > -c" really really slow.
>
> Evidence please?  Most file systems that I've looked at optimize
> zero-size files pretty well.
>
>
> They aren't zero bytes.
> It's those pesky 5 (or 14 or whatever size that gzip and lz4 produces)
> byte files.  66 thousand tiny files plus 8 thousand files with data in
> them makes for a 2.4MB directory.  That's big and slow.
>
> $ find . -size 14c | wc
>   66180   66180 1191240
>
> $ zstd -dk 2115841.dat.zst
> 2115841.dat.zst     : 5 bytes
>
> $ cat 2115841.dat
> \.
>
> $ dir | grep " 14 " | head -n20
> -rw-r--r-- 1 postgres postgres         14 2026-05-22 00:50:30
> 2115841.dat.zst
> -rw-r--r-- 1 postgres postgres         14 2026-05-22 00:50:30
> 2115842.dat.zst
> -rw-r--r-- 1 postgres postgres         14 2026-05-22 00:50:30
> 2115843.dat.zst
> -rw-r--r-- 1 postgres postgres         14 2026-05-22 00:50:30
> 2115844.dat.zst
> -rw-r--r-- 1 postgres postgres         14 2026-05-22 00:50:30
> 2115845.dat.zst
> -rw-r--r-- 1 postgres postgres         14 2026-05-22 00:50:30
> 2115851.dat.zst
> -rw-r--r-- 1 postgres postgres         14 2026-05-22 00:50:30
> 2115899.dat.zst
> -rw-r--r-- 1 postgres postgres         14 2026-05-22 00:50:30
> 2115901.dat.zst
> -rw-r--r-- 1 postgres postgres         14 2026-05-22 00:50:30
> 2115902.dat.zst
> -rw-r--r-- 1 postgres postgres         14 2026-05-22 00:50:30
> 2115903.dat.zst
> -rw-r--r-- 1 postgres postgres         14 2026-05-22 00:50:30
> 2115905.dat.zst
> -rw-r--r-- 1 postgres postgres         14 2026-05-22 00:50:30
> 2115907.dat.zst
> -rw-r--r-- 1 postgres postgres         14 2026-05-22 00:50:30
> 2115909.dat.zst
> -rw-r--r-- 1 postgres postgres         14 2026-05-22 00:50:30
> 2115913.dat.zst
> -rw-r--r-- 1 postgres postgres         14 2026-05-22 00:50:30
> 2115915.dat.zst
> -rw-r--r-- 1 postgres postgres         14 2026-05-22 00:50:30
> 2115917.dat.zst
> -rw-r--r-- 1 postgres postgres         14 2026-05-22 00:50:30
> 2115919.dat.zst
> -rw-r--r-- 1 postgres postgres         14 2026-05-22 00:50:30
> 2115923.dat.zst
> -rw-r--r-- 1 postgres postgres         14 2026-05-22 00:50:30
> 2115926.dat.zst
> -rw-r--r-- 1 postgres postgres         14 2026-05-22 00:50:30
> 2115931.dat.zst
> --
> Death to <Redacted>, and butter sauce.
> Don't boil me, I'm still alive.
> <Redacted> lobster!

Maybe just avoiding to compress empty files would already do the job. I
think any file below a certain size isn't worth compressing.

Regards,

Holger

--

Holger Jakobs