Re: documenting the backup manifest file format

On 4/14/20 3:19 PM, David Steele wrote:
> On 4/14/20 3:03 PM, Andrew Dunstan wrote:
>>
>> On 4/14/20 1:33 PM, David Steele wrote:
>>> On 4/14/20 1:27 PM, Alvaro Herrera wrote:
>>>> On 2020-Apr-14, David Steele wrote:
>>>>
>>>>> On 4/14/20 12:56 PM, Robert Haas wrote:
>>>>>
>>>>>> Hmm, did David suggest that before? I don't recall for sure. I think
>>>>>> he had some suggestion, but I'm not sure if it was the same one.
>>>>>
>>>>> "I'm also partial to using epoch time in the manifest because it is
>>>>> generally easier for programs to work with.  But, human-readable
>>>>> doesn't
>>>>> suck, either."
>>>>
>>>> Ugh.  If you go down that road, why write human-readable contents at
>>>> all?  You may as well just use a binary format.  But that's a very
>>>> slippery slope and you won't like to be in the bottom -- I don't see
>>>> what that gains you.  It's not like it's a lot of work to parse a
>>>> timestamp in a non-internationalized well-defined human-readable
>>>> format.
>>>
>>> Well, times are a special case because they are so easy to mess up.
>>> Try converting ISO-8601 to epoch time using the standard C functions
>>> on a system where TZ != UTC. Fun times.
>>
>> Even if it's a zulu time? That would be pretty damn sad.
> ZULU/GMT/UTC are all fine. But if the server timezone is EDT for
> example (not that I recommend this) you are likely to get the wrong
> result. Results vary based on your platform. For instance, we found
> MacOS was more likely to work the way you would expect and Linux was
> hopeless.
>
> There are all kinds of fun tricks to get around this (sort of). One is
> to temporarily set TZ=UTC which sucks if an error happens before it
> gets set back. There are some hacks to try to determine your offset
> which have inherent race conditions around DST changes.
>
> After some experimentation we just used the Posix definition for epoch
> time and used that to do our conversions:
>
> https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap04.html#tag_04_16
>
>
>

OK, but I think if we're putting a timestamp string in ISO-8601 format
in the manifest it should be in UTC / Zulu time, precisely to avoid
these issues. If that's too much trouble then yes an epoch time will
probably do.

cheers

andrew

--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services