Skip site navigation (1) Skip section navigation (2)

Re: select on 22 GB table causes "An I/O error occured while sending to the backend." exception

From: Craig James <craig_james(at)emolecules(dot)com>
To: James Mansion <james(at)mansionfamily(dot)plus(dot)com>
Cc: david(at)lang(dot)hm, Florian Weimer <fweimer(at)bfk(dot)de>, henk de wit <henk53602(at)hotmail(dot)com>, pgsql-performance(at)postgresql(dot)org
Subject: Re: select on 22 GB table causes "An I/O error occured while sending to the backend." exception
Date: 2008-08-29 15:25:51
Message-ID: 48B814FF.3000600@emolecules.com (view raw or flat)
Thread:
Lists: pgsql-performance
James Mansion wrote:
> I can't see how an OS can lie to processes about memory being allocated 
> to them and not be ridiculed as a toy, but there you go.  I don't think 
> Linux is the only perpetrator - doesn't AIX do this too?

This is a leftover from the days of massive physical modeling (chemistry, physics, astronomy, ...) programs written in FORTRAN. Since FORTRAN didn't have pointers, scientists would allocate massive three-dimensional arrays, and their code might only access a tiny fraction of the memory.  The operating-system vendors, particularly SGI, added features to the various flavors of UNIX, including the ability to overcommit memory, to support these FORTRAN programs, which at the time were some of the most important applications driving computer science and computer architectures of workstation-class computers.

When these workstation-class computers evolved enough to rival mainframes, companies started shifting apps like Oracle onto the cheaper workstation-class computers.  Unfortunately, the legacy of the days of these FORTRAN programs is still with us, and every few years we have to go through this discussion again.

Disable overcommitted memory.  There is NO REASON to use it on any modern server-class computer, and MANY REASONS WHY IT IS A BAD IDEA.

Craig

In response to

Responses

pgsql-performance by date

Next:From: Matthew WakelingDate: 2008-08-29 15:56:30
Subject: Re: select on 22 GB table causes "An I/O error occured while sending to the backend." exception
Previous:From: Alvaro HerreraDate: 2008-08-29 13:48:06
Subject: Re: select on 22 GB table causes "An I/O error occuredwhile sending to the backend." exception

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group