TDE: Benchmarking WAL encryption approaches

Hello!

During the TDE discussions at pgconf.dev, I promised some benchmarks
about performance characteristics regarding different WAL encryption
options, to provide solid data before deciding on an approach.

I ended up benchmarking and optimizing different variants more than I
initially planned, but now I have some conclusions that I think are
interesting to share.

I will only focus on two variants that make sense from a performance
perspective, per page or per record AES-CTR-256. Other encryption
modes / integrations all result in significantly decreased performance
(or no visible benefit for added complexity). For example AES-GCM can
be implemented in both versions, but because GCM is slower, the end
result will be slower. I can also share numbers about GCM/XTS and
other possibilities if people are interested, but for a "simple
implementation in the core", I think it should be safe to assume that
we want to go with CTR.

So the two approaches are:

1. ctr_pagelevel: when we flush or read a wal page, we encrypt/decrypt
it. This is simple, encrypts absolutely everything, but does
everything in the critical section holding the WAL lock.
2. ctr_perrecord: encryption happens when we write a WAL record. Each
record has to hold an additional IV, since at this point we can't rely
on the LSN. It parallelizes encryption at the cost of increased
complexity in the backend. Only the record data is encrypted, headers
stay plaintext, which slightly simplifies command-line tools.

I attached (one version) of both patches. All patches I used for
testing are minimal: encryption is hardcoded, uses a hardcoded key
baked into the source, there are no options to disable it or change
settings, only the minimum required for performance testing and
validation (making sure that we indeed encrypt the WAL).

The naive implementation of perrecord is clearly slower than
plaintext/pagelevel, however, after optimizing both versions, we can
see the advantages of it: pagelevel performance degrades in scenarios
with many clients and high WAL throughput, as the lock contention
increases.

In my tests, I focused on write performance. Read performance
(recovery, replication) is a different topic, and my implementations
for both could be optimized better - I didn't try to measure and
improve them yet.

I used 3 test scenarios:

* sysbench oltp_read_write
* sysbench oltp_write_only
* A scenario in which each session repeats the following:
1. select 100 random IDs (locally, no SQL queries, as ids are serial
and continuous)
2. execute an UPDATE SET c=c+1 WHERE id IN (id list);

For testing, I used an AMD Threadripper 3970X (32 cores) and an Intel
Optane P5800X for storage, which doesn't have the same throttling
limitations as consumer SSDs, the tests weren't limited by IO.

In the read_write scenario, there's no difference between the 2
encryption implementation and unencrypted master at all. There's some
noise, but we can't notice the effect of encryption at all.

In the write_only scenario, there's still no difference between master
and ctr_perrecord, but ctr_pagelevel starts to fall behind at 32/64
threads. The difference disappears at 128 threads, as at that point
master performance also degrades in my test setup.

In the update_inlist scenario which increases WAL throughput more
quickly, it already falls behind the other two at 8 threads, and
remains there for all following runs: everywhere except in the single
threaded case we can see a clear 20% performance drop. Similarly to
the write_only scenario, there's no measurable difference between
master and ctr_perrecord.

This is in line with my earlier observations during less detailed
benchmarking, that if we want to guarantee good WAL encryption
performance, we have to go with per record encryption. The results
also depend on the exact hardware used for the tests, scenario, and so
on, but I think the general point of it is clear: the more work we do
in the critical section, the worse the performance loss can be in
certain configurations. Going with a choice that adds no additional
workload there seems like a generally safer choice to me.

Please share any feedback / improvement ideas for the patches, or
different approaches we could test, I can continue benchmarking more
scenarios / prototypes if anybody has ideas. Unless there's a
different conclusion, I'll proceed with sharing a first prototype of a
TDE proposal using the per record approach in the following weeks.

I also attached the actual results of a comparison run as a CSV file.Hello!

During the TDE discussions at pgconf.dev, I promised some benchmarks
about performance characteristics regarding different WAL encryption
options, to provide solid data before deciding on an approach.

I ended up benchmarking and optimizing different variants a lot more
than I initially planned, but now I have some conclusions that I think
are interesting to share.

I will only focus on two variants that make sense from a performance
perspective, per page or per record AES-CTR-256. Other encryption
modes / integrations all result in significantly decreased performance
(or no visible benefit for added complexity). For example AES-GCM can
be implemented in both versions, but because GCM is slower, the end
result will be slower. I can also share numbers about GCM/XTR and
other possibilities if people are interested, but for a "simple
implementation in the core", I think it should be safe to assume that
we want to go with CTR.

So the two approaches are:

1. ctr_pagelevel: when we flush or read a wal page, we encrypt/decrypt
it. This is simple, encrypts absolutely everything, but does
everything in the critical section holding the WAL lock.
2. ctr_perrecord: encryption happens when we write a WAL record. Each
record has to hold an additional IV, since at this point we can't rely
on the LSN. Parallelizes encryption, at the cost of increasing
complexity. Only the record data is encrypted, headers stay plaintext.

I attached (one version) of both patches. All patches I used for
testing are minimal: encryption is hardcoded, uses a hardcoded key
baked into the source, there are no options to disable it or change
settings, only the minimum required for performance testing and
validation (making sure that we indeed encrypt the WAL).

The naive implementation of perrecord is clearly slower than
plaintext/pagelevel, however, after optimizing both versions, we can
see the advantages of it: pagelevel performance degrades in scenarios
with many clients and high WAL throughput, as the lock contention
increases.

In my tests, I focused on write performance. Read performance
(recovery, replication) is a different topic, and my implementations
for both could be optimized better - I didn't try to measure and
improve them yet.
I used 3 test scenarios:

* sysbench oltp_read_write
* sysbench oltp_write_only
* A scenario in which each session repeats the following:
1. select 100 random IDs (locally, no SQL queries, as ids are serial
and continuous)
2. execute an UPDATE SET c=c+1 WHERE id IN (id list);

For testing, I used an AMD Threadripper 3970X (32 cores) and an Intel
Optane P5800X for storage, which doesn't have the same throttling
limitations as consumer SSDs, the tests weren't limited by IO.

In the read_write scenario, there's no difference between the 2
encryption implementation and unencrypted master at all. There's some
noise, but we can't notice the effect of encryption at all.

In the write_only scenario, there's still no difference between master
and ctr_perrecord, but ctr_pagelevel starts to fall behind at 32/64
threads. The difference disappears at 128 threads, as at that point
master performance also degrades in my test setup.

In the update_inlist scenario which increases WAL throughput more
quickly, it already falls behind the other two at 8 threads, and
remains there for all following runs: everywhere except in the single
threaded case we can see a clear 20% performance drop. Similarly to
the write_only scenario, there's no measurable difference between
master and ctr_perrecord.

This is in line with my earlier observations during less detailed
benchmarking, that if we want to guarantee good WAL encryption
performance, we have to go with per record encryption. The results
also depend on the exact hardware used for the tests, scenario, and so
on, but I think the general point of it is clear: the more work we do
in the critical section, the worse the performance loss can be in
certain configurations. Going with a choice that adds no additional
workload there seems like a generally safer choice to me.

Please share any feedback / improvement ideas for the patches, or
different approaches we could test, I can continue benchmarking more
scenarios / prototypes if anybody has ideas. Unless there's a
different conclusion, I'll proceed with sharing a first prototype of a
TDE proposal using the per record approach in the following weeks.

I also attached the actual results of a comparison run as a CSV file.