Skip site navigation (1) Skip section navigation (2)

Zen and art if WAL and dump file mainteance

From: dx k9 <bitsandbytes88(at)hotmail(dot)com>
To: posgres support <pgsql-admin(at)postgresql(dot)org>
Subject: Zen and art if WAL and dump file mainteance
Date: 2008-01-28 20:37:58
Message-ID: BAY116-W326F5FEA10C498D47792CDD1340@phx.gbl (view raw, whole thread or download thread mbox)
Lists: pgsql-admin

We are trying to confirm our understanding of WAL.  Thank you in advance for any comments and help.  We have version 8.24.

We noticed that when our nightly maintenance of the WAL folders kicked off    *** - see below.  Production processing was "KILLED".  We were in the process of rekeying  MANY existing tables.
We were doing heavy processing(which we normally don't do at the time of maintenance -  adding fields, table creation and filling tables going on.  During this time, archiving was enabled.

## Our data is set up like this
A /sprj/sqldumps_Linux.pgsql_serverX is our storage for the WAL
B /sqldata_Linux.pgsql_serverX is another computer/folder where the actual data is (pgdata).

We are trying to figure out why maintenance on A (zipping up & removing files) would have an affect on production processing (killed it) on pgdata - B.

We are thinking we should have turned off the nightly maintenance at this point, but we are wondering why and trying to improve our process logic.  We were thinking that our zipping and deleting
activity on A would not have an affect on B.  And it never has before, although we never had heavy processing at that time before.
Since, the archives don't happen too often, until the 36 logfile segments locally gets filled up, then one gets pushed out.  However, our clean up process is running from 11:30 - 12:32 in this case.
In our case, the first script wasn't finished zipping causing the second remove script to fail.

#This is what happened on A - nightly maintenance of dumps folder ***  Maybe there is a better way to do this and we are missing something here
30 19,23 * * * find /maintenance/NFSmntdd.dump001/sqldumps_Linux.pgsql_* -type f -mmin +10 ! -name \*.gz -size +0c -exec gzip {} \; 
30 0 * * * find /maintenance/NFSmntdd.dump001/sqldumps_Linux.pgsql_* -type f -mmin +20 -exec rm {} \

# This is what was happening on B

# - Settings -

#fsync = on                             # turns forced synchronization on or off
#wal_sync_method = fsync                # the default is the first option
                                        # supported by the operating system:
                                        #   open_datasync
                                        #   fdatasync
                                        #   fsync
                                        #   fsync_writethrough
                                        #   open_sync
#full_page_writes = on                  # recover from partial page writes
#wal_buffers = 64kB                     # min 32kB
wal_buffers = 128kB                     # min 32kB
                                        # (change requires restart)
#commit_delay = 0                       # range 0-100000, in microseconds
#commit_siblings = 5                    # range 1-1000

# - Checkpoints -

#checkpoint_segments = 3                # in logfile segments, min 1, 16MB each
checkpoint_segments = 36                # in logfile segments, min 1, 16MB each
#checkpoint_timeout = 5min              # range 30s-1h
#checkpoint_warning = 30s               # 0 is off

# - Archiving -

#archive_command = ''           # command to use to archive a logfile segment
archive_command = 'cp "%p" /sprj/sqldumps_Linux.pgsql_serverx/"%f"'         # command to use to archive a logfile segment
archive_timeout = 0             # force a logfile segment switch after this
                                # many seconds; 0 is off

Shed those extra pounds with MSN and The Biggest Loser!


pgsql-admin by date

Next:From: Tom LaneDate: 2008-01-28 20:43:36
Subject: Re: production error
Previous:From: Scott MarloweDate: 2008-01-28 18:41:12
Subject: Re: Search string without tsearch

Privacy Policy | About PostgreSQL
Copyright © 1996-2017 The PostgreSQL Global Development Group