Re: Untrusted relations and WAL

On 08/11/2026 8:53 PM, Andrey Borodin wrote:
> Hi hackers,
>
> I know that everybody who can do anything about vulnerabilities is busy
> with the workload LLMs generate for pgsql-security these days. Still,
> the list does not forbid ideas for fun, and this one has been bothering
> me. There is no patch attached, and there will not be one: I understand
> the problem is beyond what I can solve alone, and current AI will not
> lift me to the required level either. I would be glad to hear thoughts.
>
> Our threat model is clear and well served by the security team: "a
> security vulnerability in PostgreSQL is an issue that allows a user to
> gain access to privileges or data that they do not have permission to
> use, or allows a user to execute arbitrary code through a PostgreSQL
> process" [0]. It covers a lot: the wire protocol, SQL constructs,
> database objects, ACLs, type input functions. And yet a typical exploit
> is just an SQL query. Frankly, a bit boring. Let's add attack surface in
> the form of new bytes!
>
> Today, letting a friend run your $PGDATA on their machine is handing them
> a Trojan horse, no weaker than an .exe or an ELF. No surprise: we have
> functions, which are small programs, i.e. code, i.e. executable code,
> and running somebody else's code is what security people call RCE.
>
> Cleaning the catalog is a tractable problem. A sufficiently patched
> pg_upgrade can drop every give_me_superuser() SECURITY DEFINER, recreate
> extensions from local packages, throw away remaining C functions, and so
> on, until the catalog is fresh. I have prototyped this far. Making
> relations and WAL safe is where my imagination ends.
>
> Why would anyone give PGDATA to a friend who does not trust its bytes?
>
> * A managed PostgreSQL service could onboard an on-premise database
> without downtime: make the cluster a physical replica, then switch
> the load. Much faster than pg_dump/pg_restore, and much simpler than
> setting up logical replication for an arbitrary schema.
> Just a small GUC dance around shared_buffers\max_connections needed.
>
> * Compute/storage separation systems like Neon or Aurora. Storage
> processes WAL from many tenants, so attacking it with a zero-day
> freshly published by our security team (there is no shortage) is
> worth somebody's time. I believe, and this is worth checking, that
> they confine the WAL parser in a container and hand out page images
> that nothing interprets further: pages go to compute as they are.
> As soon as we want to do anything interesting with pages in a place
> shared by tenants (put them into Iceberg?), the page contents can
> attack us. In the managed case a per-database container is also not
> a perfect answer, but I do not want to describe control plane attacks
> on the list.

In Neon we from the very beginning do not consider compute as trusted
source of data.
This is why applying WAL at page server is performed by separate process
with restricted set of system calls.
So even if some sequence of WAL records allows  a hacker to get full
control on this process, it still can not do anything harmful.

We thought about rewriting all redo handler in Rust which can make such
cracks much more difficult.
But we refused this idea because it will greatly complicate integration
with upstream changes.

Actually we didn't encountered any attempts to crack page server by
generating invalid WAL records.
But even legal WAL records can affect page server. For example "CRATE
DATABASE ... STRATEGY FILE_COPY" can generate single arbitrary large WAL
record.
Which can just cause OOM at PS. Also arbitrary large WAL records can be
generated with wal_level=logical. But we filter such data and do not
send them to page server.