Re: documenting the backup manifest file format

On 4/14/20 3:03 PM, Andrew Dunstan wrote:
>
> On 4/14/20 1:33 PM, David Steele wrote:
>> On 4/14/20 1:27 PM, Alvaro Herrera wrote:
>>> On 2020-Apr-14, David Steele wrote:
>>>
>>>> On 4/14/20 12:56 PM, Robert Haas wrote:
>>>>
>>>>> Hmm, did David suggest that before? I don't recall for sure. I think
>>>>> he had some suggestion, but I'm not sure if it was the same one.
>>>>
>>>> "I'm also partial to using epoch time in the manifest because it is
>>>> generally easier for programs to work with.  But, human-readable
>>>> doesn't
>>>> suck, either."
>>>
>>> Ugh.  If you go down that road, why write human-readable contents at
>>> all?  You may as well just use a binary format.  But that's a very
>>> slippery slope and you won't like to be in the bottom -- I don't see
>>> what that gains you.  It's not like it's a lot of work to parse a
>>> timestamp in a non-internationalized well-defined human-readable format.
>>
>> Well, times are a special case because they are so easy to mess up.
>> Try converting ISO-8601 to epoch time using the standard C functions
>> on a system where TZ != UTC. Fun times.
>
> Even if it's a zulu time? That would be pretty damn sad.
ZULU/GMT/UTC are all fine. But if the server timezone is EDT for example
(not that I recommend this) you are likely to get the wrong result.
Results vary based on your platform. For instance, we found MacOS was
more likely to work the way you would expect and Linux was hopeless.

There are all kinds of fun tricks to get around this (sort of). One is
to temporarily set TZ=UTC which sucks if an error happens before it gets
set back. There are some hacks to try to determine your offset which
have inherent race conditions around DST changes.

After some experimentation we just used the Posix definition for epoch
time and used that to do our conversions:

https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap04.html#tag_04_16

Regards,
--
-David
david(at)pgmasters(dot)net