Re: libpq: Process buffered SSL read bytes to support records >8kB on async API

On 01/08/2025 00:48, Jacob Champion wrote:
> On Fri, Jul 18, 2025 at 11:11 AM Jacob Champion
> <jacob(dot)champion(at)enterprisedb(dot)com> wrote:
>> The attached still needs some documentation work
>
> v2 does a bunch of commit message work, but I imagine it needs a good
> bit of copy-editing for clarity.
>
> I'm not in any hurry to smash this in. I think we still need
> - independent verification of the architectural issue, to make sure
> it's not any deeper or shallower than pqReadData()
> - independent verification that this fixes the bugs that have been described
> - measurement of the performance characteristics of the new code
> - verification of the maximum amount of additional buffer memory that
> can be consumed during the drain
> - consensus that we want to maintain this new behavior
> - discussion of what we want this code to look like going forward
>
> Andres, does this patch help clarify my thoughts upthread? Ideally the
> additional code isn't getting in the way of any future
> rearchitectures, since it only pins the new requirement in the code
> that needs it.

A customer just ran into this issue and it took the team and I a few
days to debug until I remembered this thread. We're running PostgreSQL
with no changes to the networking parts, but there's a proxy in between
that decrypts and re-encrypts the TLS traffic. So I'm now motivated to
get this fixed :-).

I'll start reviewing the patch, but in the meanwhile, Jacob, could you
share the reproducer and any other testing scripts you have that might
be useful here?

- Heikki