From: | Wells Oliver <wells(dot)oliver(at)gmail(dot)com> |
---|---|
To: | Laurenz Albe <laurenz(dot)albe(at)cybertec(dot)at> |
Cc: | pgsql-admin <pgsql-admin(at)postgresql(dot)org> |
Subject: | Re: RDS restore failed due to WAL log and disk space-- any tidy fixes? |
Date: | 2024-11-17 17:12:05 |
Message-ID: | CAOC+FBX1L6hx4DHzH34eaWyhH6NK9ywYSJMMcEJH3rHZfW86+A@mail.gmail.com |
Views: | Whole Thread | Raw Message | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-admin |
Interesting. I am migrating a pg_dump archive to a new server, in a single
go. Does it make sense to disable (or speed up?) WAL archiving during the
restore, then reenable it after the restore so a future replica could work?
What would be the steps here? Would disabling or "speeding up" be faster?
max_slot_wal_keep_size is -1 at the moment so I think that's why it kept a
ton of WAL and ran out of space.
On Sun, Nov 17, 2024 at 7:41 AM Laurenz Albe <laurenz(dot)albe(at)cybertec(dot)at>
wrote:
> On Sat, 2024-11-16 at 16:33 -0800, Wells Oliver wrote:
> > I provisioned an RDS instance with 2500GB space and began the restore of
> a database I know to be about 1750 GB using 16 jobs.
> >
> > Unfortunately, it died very near the end when it ran out of disk space
> due to WAL log usage. Lots of:
> >
> > 2024-11-17 00:07:09 UTC::@:[19861]:PANIC: could not write to file
> "pg_wal/xlogtemp.19861": No space left on device
> >
> >
> > And then kaboom.
> >
> > I'm wondering what my course of action should be. Can I disable/reduce
> WAL during a restore?
> > wal_level is set to replica, can this temporarily be set to minimal?
> Should I just eat the extra
> > costs to add headroom for the WAL? Would using fewer jobs during a
> restore reduce the amount of WAL
> > created?
>
> If you are using minimal WAL logging and you restore the dump in a single
> transaction, you
> should see way less WAL generated, because data inserted into the table in
> the same transaction
> as the CREATE TABLE statement need not be WAL logged.
>
> But you might more easily solve the problem by speeding up or disabling
> the WAL archiver,
> so that PostgreSQL removes old WAL after the next checkpoint.
>
> Yours,
> Laurenz Albe
>
--
Wells Oliver
wells(dot)oliver(at)gmail(dot)com <wellsoliver(at)gmail(dot)com>
From | Date | Subject | |
---|---|---|---|
Next Message | Ron Johnson | 2024-11-17 17:21:05 | Re: RDS restore failed due to WAL log and disk space-- any tidy fixes? |
Previous Message | Laurenz Albe | 2024-11-17 15:41:38 | Re: RDS restore failed due to WAL log and disk space-- any tidy fixes? |