Re: Req. for some help with resolving the error "pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment"

The only way to increase the segment size is to adjust the setting in the source code and recompile the server. There are no runtime adjustments available to do this.

Sent from my iPad

> On Dec 5, 2016, at 3:41 PM, Keith <keith(at)keithf4(dot)com> wrote:
>
>
>
>> On Mon, Dec 5, 2016 at 3:27 PM, Tharmarajah, Sam <sambavan(dot)tharmarajah(at)usask(dot)ca> wrote:
>> Hi John,
>>
>>
>>
>> Thank you for the feed back. Presently it is taking around 1hr to finish the backups. We have the wal_keep_segments set to 50. Is there way to adjust the segment size to be more than 16MB? At times I notice in the pg_xlog, there are 10-15 segments on an invocation point.
>>
>>
>>
>>
>>
>> Thanx,
>>
>> Sam
>>
>>
>>
>> From: John Scalia [mailto:jayknowsunix(at)gmail(dot)com]
>> Sent: Monday, December 05, 2016 1:52 PM
>> To: Tharmarajah, Sam <sambavan(dot)tharmarajah(at)usask(dot)ca>
>> Cc: pgsql-admin(at)postgresql(dot)org
>> Subject: Re: [ADMIN] Req. for some help with resolving the error "pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment"
>>
>>
>>
>> More than likely your keep_wal_segments setting is too low. How long is the pg_basebackup taking? Remember that segments are generated about every 5 minutes, so if the backup takes an hour, you need at least 12 segments stored. At 2 hours, you need 24 etc., I'd set the value to about 12.2 segments/hour of backup.
>>
>> --
>>
>> Jay
>>
>>
>>
>> Sent from my iPad
>>
>>
>> On Dec 5, 2016, at 1:57 PM, Tharmarajah, Sam <sambavan(dot)tharmarajah(at)usask(dot)ca> wrote:
>>
>> Hi
>>
>>
>>
>> I am a newbie to postgres. We are on postgres 9.1 running on RedHat 6. When we do the postgres backups we are noticing the following errors
>>
>>
>>
>> pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment 000000010000020B00000040 has already been removed
>>
>>
>>
>> But when I look into pg_xlog, I notice
>>
>>
>>
>> 000000010000020B00000040.00000118.backup
>>
>>
>>
>>
>>
>> May I know how to resolve this error? Are the backups done with these errors, recoverable?
>>
>>
>>
>>
>>
>> Thanks in advance for your support.
>>
>>
>>
>> Thanx,
>>
>> Sam
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
> If you're using pgbasebackup just for creating consistent backups, use the --xlog-method=strean (-Xs) to ensure you get all the WAL files necessary for a consistent backup. This avoids having to set wal_keep_segments high enough to account for the time length of your entire backup. Note you will need an extra WAL stream for this setting, so ensure max_wal_senders is set high enough on the master.
>
> Now if you're going to be using this backup to create a slave, you must get that slave up and running before the master recycles all the WAL files necessary for replication to be able to catch up from when the backup stopped. Otherwise you will see that error you shared in your original post. That's when you may need to adjust wal_keep_segments. Either that or you should set up an archive_command on the master to keep a secondary copy of all WAL files.
>
> Keith