Improve readability of json_lex()

I've been working in json_lex() for a little bit to add support for
JSONC[0] (patches to come later). In my endeavors to do so, I found
json_lex() to be quite beefy. Part of that is due to it being the entry
point for lexing when using both the incremental and non-incremental
JSON parsers. This leads to large sections of branching that seem to
hurt readability.

I've separated each refactor into its own commit such that we could take
some, all, or none of them. I know refactoring can be very opinionated.
Based on my experience with the upcoming JSONC patches, I have found
these changes to allow my future changes to be much more reviewable.
Ideally, these changes also make it easier for future readers to
understand and contribute in the JSON lexing code. History points to
this code being pretty static with the largest change coming when Andrew
added the non-recursive (incremental) parser.

Maybe others have thought of other ways to make this function a bit more
approachable.

[0]: https://jsonc.org

--
Tristan Partin
PostgreSQL Contributors Team
AWS (https://aws.amazon.com)