Re: BUG #17760: SCRAM authentication fails with "modern" (rsassaPss signature) server certificate

On 25/01/2023 13:32, PG Bug reporting form wrote:
> The following bug has been logged on the website:
>
> Bug reference: 17760
> Logged by: Gunnar "Nick" Bluth
> Email address: gunnar(dot)bluth(at)pro-open(dot)de
> PostgreSQL version: 13.8
> Operating system: Ubuntu 20.04
> Description:
>
> My client recently started rolling out new server certificates, which, when
> dumped with "openssl x509 [...]", show slightly different information
> regarding signature, hash etc.
>
> Old:
> Signature Algorithm: sha256WithRSAEncryption
> New:
> Signature Algorithm: rsassaPss
> Hash Algorithm: sha512
> Mask Algorithm: mgf1 with sha512
> Salt Length: 0x40
> Trailer Field: 0xBC (default)
>
> When trying to authenticate on a server using such a certificate using
> scram-sha-256, we receive an error message: "could not find digest for NID
> UNDEF"

I was able to create a certificate like that with these commands:

openssl genpkey -algorithm rsa-pss -pkeyopt rsa_keygen_bits:2048
-pkeyopt rsa_keygen_pubexp:65537 -out CA.key

openssl req -new -x509 -days 365 -nodes -text -out server.crt -key
CA.key -keyout server.key -subj "/CN=rsapss.localtest.me"

And was able to reproduce the problem with that.

As a workaround, you can disable channel binding with the
"channel_binding=disable" libpq option.

> I tried to boil this down a bit. The error comes from
> src/interfaces/libpq/fe-secure-openssl.c, pgtls_get_peer_certificate_hash(),
> line 440+.
> The comment there says "If something else is used, the same hash as the
> signature algorithm is used."
> And obviously, "EVP_get_digestbynid(NID_rsassaPss)" doesn't return a
> result.
>
> Now, openssl's "crypto/objects/obj_xref.txt" contains this:
> # OID cross reference table.
> # Links signatures OIDs to their corresponding public key algorithms
> # and digests.
> <snip>
> sha256WithRSAEncryption sha256 rsaEncryption
> <snip>
> # For PSS the digest algorithm can vary and depends on the included
> # AlgorithmIdentifier. The digest "undef" indicates the public key
> # method should handle this explicitly.
> rsassaPss undef rsassaPss
>
>
> That explains the "UNDEF" in the error message (i.e., I *think* that's where
> it's coming from).
>
> I have to say that I'm not all too deep into crypto stuff. I couldn't even
> tell how to create one such certificate, let alone what that PSS stuff is
> all about ;-/
>
> Maybe this is even fixed with recent OpenSSL versions (client has 1.1.1f,
> Ubuntu 20.04)? Though that line was introduced in 2010...

Thanks for the investigation!

> I do think however that this is an oversight on our side and has to be
> addressed. If not in code, the docs should point out that certain server
> certificate types (PSS) may not work with SCRAM auth (or libpq needs to be
> compiled against a minimum version of OpenSSL, if that's the root cause).

There are a few things we change in PostgreSQL for this:

1. If the server cannot compute a hash of the certificate, because it
cannot unambiguously determine the hash algorithm to use, it should not
offer the SCRAM-SHA-256-PLUS authentication method to the client. In
other words, if the server cannot do channel binding with it's
certificate, it should say so, instead of erroring out later.

2. Similarly in the client: if libpq cannot determinate the hash
algorithm to use with the server's certificate, it should not attempt
channel binding.

I believe this is OK from a security point of view. If the server is
using a certificate that cannot be used with channel binding, and the
client doesn't require channel binding, it's OK to not do it. A man in
the middle can present a certificate to the client that cannot be used
with channel binding, but if they can do that they could also just not
offer the SCRAM-SHA-256-PLUS option to the client. So I don't see risk
of downgrade attacks here.

3. Add support for channel binding with RSA-PSS. The problem is that
be_tls_get_certificate_hash() doesn't know which digest algorithm to
use. As you noted, OBJ_find_sigid_algs() returns "undef" (NID_undef) for
rsassaPss certificates. I did some googling: when certificate uses
RSASSA-PSS as the signature algorithm, there is a separate field,
RSASSA-PSS-params that specifies the hash algorithm
(https://www.rfc-editor.org/rfc/rfc4055#section-3.1). If you ask me,
OBJ_find_sigid_algs() should look into RSASSA-PSS-params and figure it
out, but I'm no expert in the OpenSSL codebase so maybe that would be
the wrong place to do it.

OpenSSL 3.0 actually has a function called X509_digest_sig(), which
parses the RSASSA-PSS-params field. It also has a fallback mechanism to
use SHA512 for ED25519, SHAKE256 for ED448, and SHA256 for anything
else. We could use that function, perhaps without accepting the
fallbacks, though. Or call lower level OpenSSL functions to get the hash
algorithm from the RSASSA-PSS-params field ourselves.

I'll write a patch for 1 and 2, at least, and maybe 3 if it's not too
complicated. Please poke me if you don't hear from me in a few days.

- Heikki