From: | Dipesh Pandit <dipesh(dot)pandit(at)gmail(dot)com> |
---|---|
To: | "Bossart, Nathan" <bossartn(at)amazon(dot)com> |
Cc: | Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Jeevan Ladhe <jeevan(dot)ladhe(at)enterprisedb(dot)com>, Stephen Frost <sfrost(at)snowman(dot)net>, Andres Freund <andres(at)anarazel(dot)de>, Hannu Krosing <hannuk(at)google(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: .ready and .done files considered harmful |
Date: | 2021-08-17 12:52:42 |
Message-ID: | CAN1g5_ECegnUtHM-fg66UN=f1qFXJ_rZyrC+QoGcURJMOB-+Rw@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Thanks for the feedback.
> + StatusFilePath(archiveStatusPath, xlog, ".ready");
> + if (stat(archiveStatusPath, &stat_buf) == 0)
> + PgArchEnableDirScan();
> We may want to call PgArchWakeup() after setting the flag.
Yes, added a call to wake up archiver.
> > + * - The next anticipated log segment is not available.
> >
> > I wonder if we really need to perform a directory scan in this case.
> > Unless there are other cases where the .ready files are created out of
> > order, I think this just causes an unnecessary directory scan every
> > time the archiver catches up.
> Thinking further, I suppose this is necessary for when lastSegNo gets
> reset after processing an out-of-order .ready file.
Also, this is necessary when lastTLI gets reset after switching to a new
timeline.
> + pg_atomic_flag dirScan;
> I personally don't think it's necessary to use an atomic here. A
> spinlock or LWLock would probably work just fine, as contention seems
> unlikely. If we use a lock, we also don't have to worry about memory
> barriers.
History file should be archived as soon as it gets created. The atomic flag
here will make sure that there is no reordering of read/write instructions
while
accessing the flag in shared memory. Archiver needs to read this flag at
the
beginning of each cycle. Write to atomic flag is synchronized and it
provides
a lockless read. I think an atomic flag here is an efficient choice unless
I am
missing something.
Please find the attached patch v7.
Thanks,
Dipesh
Attachment | Content-Type | Size |
---|---|---|
v7-0001-mitigate-directory-scan-for-WAL-archiver.patch | text/x-patch | 11.4 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Denis Hirn | 2021-08-17 12:58:05 | Re: [PATCH] Allow multiple recursive self-references |
Previous Message | Daniel Gustafsson | 2021-08-17 12:43:08 | Re: [PATCH] OpenSSL: Mark underlying BIO with the appropriate type flags |