From: | James Pang <jamespang886(at)gmail(dot)com> |
---|---|
To: | Laurenz Albe <laurenz(dot)albe(at)cybertec(dot)at> |
Cc: | Frits Hoogland <frits(dot)hoogland(at)gmail(dot)com>, pgsql-performance(at)lists(dot)postgresql(dot)org |
Subject: | Re: many sessions waiting DataFileRead and extend |
Date: | 2025-06-26 03:48:45 |
Message-ID: | CAHgTRfcZPWiw7zfBuFub59W5OURcZ2q67HsydTzLhU+54uowXQ@mail.gmail.com |
Views: | Whole Thread | Raw Message | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-performance |
Thanks, I make a summary of the issue, no connection storm(fork)
either, just suddenly many session waiting on "extend" and "DataFileRead",
it last 2 seconds, this server has 64 vcpu and running there long time
without issue, only last weekend, we patch from 14.8 to 14.14. We checked
with Infra and VMWARE, and OS monitoring, except very high %sys cpu, not
found other abnormal events. so, we are thinking if Postgresql buffer cache
or OS page cache level contention leading to that(if that can contribute to
IO wait, DataFileRead, extend).
attached please find details, just masking dbname, username,and ip
addresses.
Laurenz Albe <laurenz(dot)albe(at)cybertec(dot)at> 於 2025年6月26日週四 上午11:03寫道:
> On Thu, 2025-06-26 at 10:32 +0800, James Pang wrote:
> > thans for you suggestions, we have iowait from sar command too, copy
> here, checking with infra team not found abnormal IO activities either.
> > 02:00:01 PM CPU %usr %nice %sys %iowait %irq %soft %steal
> %guest %gnice %idle
> > 02:00:03 PM all 15.92 0.00 43.02 0.65 0.76 2.56 0.00
> 0.00 0.00 37.09
>
> Crazy high system time.
>
> If it is not transparent hugepages, the other suspect is fork().
> Turn on "log_connections" for a minute or two and see if there are lots
> of connections established. That can easily use your CPU.
>
> Other than that, I am out of guesses.
>
> > Frits Hoogland <frits(dot)hoogland(at)gmail(dot)com> 於 2025年6月25日週三 下午10:27寫道:
> > > > On 25 Jun 2025, at 07:59, Laurenz Albe <laurenz(dot)albe(at)cybertec(dot)at>
> wrote:
> > > >
> > > > Run "sar -P all 1" and see if "%iowait" is high.
> > >
> > > I would (strongly) advise against the use of iowait as an indicator.
>
> I have heard that before, and I am sure you are right.
>
> I grant that if it is low, it may just mean that the CPU is under load.
> But if %iowait is high, my experience is that that indicates an I/O
> problem.
>
> Yours,
> Laurenz Albe
>
Attachment | Content-Type | Size |
---|---|---|
many_sessions_waiting_extend_Datafileread_but_no_ioload_spike | application/octet-stream | 8.9 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Motog Plus | 2025-06-26 05:11:09 | Re: Guidance Needed: Scaling PostgreSQL for 12 TB Data Growth - New Feature Implementation |
Previous Message | Tatsuo Ishii | 2025-06-26 03:25:45 | Re: many sessions waiting DataFileRead and extend |