Re: Patch: incorrect array offset in backend replication tar header

Brian Weaver <cmdrclueless(at)gmail(dot)com> writes:
> While researching the way streaming replication works I was examining
> the construction of the tar file header. By comparing documentation on
> the tar header format from various sources I certain the following
> patch should be applied to so the group identifier is put into thee
> header properly.

Yeah, this is definitely wrong.

> While I realize that wikipedia isn't always the best source of
> information, the header offsets seem to match the other documentation
> I've found. The format is just easier to read on wikipedia

The authoritative specification can be found in the "pax" page in the
POSIX spec, which is available here:
http://pubs.opengroup.org/onlinepubs/9699919799/

I agree that the 117 number is bogus, and also that the magic "ustar"
string is written incorrectly. What's more, it appears that the latter
error has been copied from pg_dump (but the 117 seems to be just a new
bug in pg_basebackup). I wonder what else might be wrong hereabouts :-(
Will sit down and take a closer look.

I believe what we need to do about this is:

1. fix pg_dump and pg_basebackup output to conform to spec.

2. make sure pg_restore will accept both conformant and
previous-generation files.

Am I right in believing that we don't have any code that's expected to
read pg_basebackup output? We just feed it to "tar", no?

I'm a bit concerned about backwards compatibility issues. It looks to
me like existing versions of pg_restore will flat out reject files that
have a spec-compliant "ustar\0" MAGIC field. Is it going to be
sufficient if we fix this in minor-version updates, or are we going to
need to have a switch that tells pg_dump to emit the incorrect old
format? (Ick.)

regards, tom lane