| From: | Bráulio Oliveira <brauliobo(at)gmail(dot)com> |
|---|---|
| To: | Dimitrios Apostolou <jimis(at)gmx(dot)net>, Thomas Munro <thomas(dot)munro(at)gmail(dot)com> |
| Cc: | Andres Freund <andres(at)anarazel(dot)de>, pgsql-hackers(at)lists(dot)postgresql(dot)org |
| Subject: | Re: fallocate() causes btrfs to never compress postgresql files |
| Date: | 2025-11-17 03:23:09 |
| Message-ID: | CAJri6_syvJsMe2kfsWXHxdAn7WuYbwK86Ecbj=B=0mhVUv9HLg@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
On Sun, Nov 16, 2025 at 11:38 PM Dimitrios Apostolou <jimis(at)gmx(dot)net> wrote:
>
> Hi,
>
> this is just a heads-up about files being generated by PostgreSQL 17 not
> being compressed by Btrfs, even when mounted with the force-compress mount
> option. I have this occuring aggressively when restoring a database via
> pg_restore. I think this is caused mdzeroextend() calling FileFallocate(),
> which in turn invokes posix_fallocate().
>
> I also verified that turning off the use of fallocate causes the database
> to write compressed files again, like it did in older versions.
> Unfortunately the only way I found was to configure with a "hack" so that
> autoconf thinks the feature is not available:
>
> ./configure ac_cv_func_posix_fallocate=no
Hi,
I'm rebuilding the official Ubuntu package with this configuration option as
after migrating to PostgreSQL to a BTRFS partition with Snapper & Compression
is eating space really fast (around 50-100gb per hour).
Sorry for not using the [PING] thread, I couldn't get the email resent
as I just joined the mailing list.
Are there any plans to merge the file_extend_method soon? BTRFS
compressed the original
database to 36% of the original size, but now with the DB running it is using
a lot of space crazily, not sure if running VACUUM and ANALYSE is
making things worse.
Thankfully,
Bráulio
>
> There have been discussions on the btrfs mailing list about why it does
> that, the summary is that it is very difficult to guarantee that
> compressed writes will not fail with ENOSPACE on a CoW filesystem, thus
> files with fallocate()d ranges are treated as being marked NOCOW,
> effectively disabling compression.
>
> Should PostgreSQL provide a setting to avoid the use of fallocate()? Or is
> it the filesystem at fault for not returning EOPNOTSUPP, in which case
> postgres would use its fallback code?
>
> BTW even in the last case, PostgreSQL would not notice the lack of
> fallocate() support as glibc implements a userspace fallback in
> posix_fallocate(). That fallback has its own issues that hopefully will
> not affect postgres (see CAVEATS in man 3 posix_fallocate).
>
> Regards,
> Dimitris
>
>
>
>
>
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Thomas Munro | 2025-11-17 03:35:52 | Re: fallocate() causes btrfs to never compress postgresql files |
| Previous Message | jian he | 2025-11-17 03:04:15 | Re: PartitionKeyData->partattrs, refactor some 0 to InvalidAttrNumber |