Re: PostgreSQL on S3-backed Block Storage with Near-Local Performance

Hi all,

Circling back on this thread, ZeroFS now supports placing its WAL on local storage (or something like S3 Express One Zone). ZeroFS wal is sub-gigabyte and just there to handle frequent syncs, it doesn't act as writeback caching.

Here are pgbench results with synchronous_commit = on, WAL on local NVMe, on a 6-core / 32GB RAM machine with a 4 Gb/s pipe:

$ pgbench -c 100 -T 100 --protocol=prepared

transaction type: <builtin: TPC-B (sort of)>
scaling factor: 100
query mode: prepared
number of clients: 100
number of threads: 1
duration: 100 s
number of transactions actually processed: 1,578,675
number of failed transactions: 0 (0.000%)
latency average = 6.312 ms
tps = 15,843 (without initial connection time)

Best,
Pierre

On Fri, Jul 25, 2025, at 00:03, Jeff Ross wrote:
> On 7/24/25 13:50, Pierre Barre wrote:
>
>> It’s not “safe” or “unsafe”, there’s mountains of valid workloads which don’t require synchronous_commit. Synchronous_commit don’t make your system automatically safe either, and if that’s a requirement, there’s many workarounds, as you suggested, it certainly doesn’t make the setup useless.
>>
>> Best,
>> Pierre
>>
>> On Thu, Jul 24, 2025, at 21:44, Nico Williams wrote:
>>> On Fri, Jul 18, 2025 at 12:57:39PM +0200, Pierre Barre wrote:
>>>> - Postgres configured accordingly memory-wise as well as with
>>>> synchronous_commit = off, wal_init_zero = off and wal_recycle = off.
>>> Bingo. That's why it's fast (synchronous_commit = off). It's also why
>>> it's not safe _unless_ you have a local, fast, persistent ZIL device
>>> (which I assume you don't).
>>>
>>> Nico
>>> --
> This then begs the obvious question of how fast is this with
> synchronous_commit = on?