Re: Re: [ADMIN] problems using pg_start_backup/pg_stop_backup and pg_basebackup at same time

On Fri, Jul 1, 2016 at 5:00 PM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
>
>
> On Mon, Jun 27, 2016 at 8:27 PM, Alex Malek <magicagent(at)gmail(dot)com> wrote:
>>
>> Should I file a bug?
>
>
> Oh, sorry. I did not spot that this was posted to pgsql-admin, which has a
> lot fewer readers. I've moved it over to -hackers where more people will see
> it.
>
> (And also, please don't top-post on these lists, as it makes the discussion
> much harder to follow)
>
>
>>
>>
>> Does anyone know of a workaround?
>>
>> I could try running pg_basebackup w/ --xlog-method=fetch and see if that
>> helps but I do like knowing that the base backup will have all needed WAL
>> files regardless of WAL retention policy on master.
>>
>> Thanks.
>> Alex
>>
>> Recap of what I *think* is going on:
>>
>> 1) pg_basebackup w/ --xlog-method=stream is started
>> 2) archive_command is not executed while pg_basebackup() is running
>
>
> There is no reason why archive_command should not run during a pg_basebackup
> run. Do you get anything at all in the logfile indicating that this would be
> the case? If you look at the ps output, does the archiver process indicate
> what it's doing, and does it have a sub-process for your rsync command?
>
> Is there any chance that for example the network or other part of the system
> gets saturated and the archive_command is simply runniung too slow to keep
> up?
>
>
>> 3) while pg_basebackup() is running pg_start_backup() is executed
>> 4) Later pg_stop_backup() is executed.
>> 5) pg_stop_backup() hangs related to the archive_command not running
>> 6) hours later when pg_basebackup finishes, back logged archive commands
>> get executed then pg_stop_backup() finishes
>
>
> This would be consistent with (2), but there is no reason why (2) would
> happen because of pg_basebackup unless there's an actual bug there.
>

I think there is one dependency of archiver (2) to finish which is
that corresponding .ready file should be generated. Can it be
possible, that due to some reason the .ready file is not generated
corresponding to stoppoint determined by pg_stop_backup()? Is it
possible to verify if the required .ready files are present?

--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com