Re: Information about WAL Configuration needs an update

On Mon, Jun 13, 2011 at 2:25 PM, Rafael Martinez
<r(dot)m(dot)guerrero(at)usit(dot)uio(dot)no> wrote:
> On Mon, 2011-06-13 at 13:24 -0400, Robert Haas wrote:
>> On Fri, Jun 3, 2011 at 3:42 AM, Rafael Martinez
>> >
>> > You can see the graph with the generation of WAL files + some extra
>> > information for this test here: http://folk.uio.no/rafael/total_wal/
>> >
>> > What do you think? Shouldn't we update the documentation with some
>> > information about this?
>>
>> Perhaps, but we'd have to think of something intelligent to say about
>> it first.  We can't remove the old WAL files until we successfully
>> checkpoint, and so I think if checkpoints are taking a very long to
>> complete or failing altogether, there's actually no upper bound.  I
>> don't think we have any kind of "hard stop" where, if no log space is
>> available, we just refuse to process write transactions - such a thing
>> would seem to be rather dangerous.
>>
>
> Well, a good start will be to try to identify or describe the situations
> where checkpoints can take very long to complete or fail altogether.
>
> I have the first one: Creating a large GIN index on a tsvector column. I
> don't know why, maybe somebody who knows postgres internals can explain
> why a creation of an index can create this situation.

I think we're discussing this on the wrong list. It sounds to me like
you have a performance or configuration problem (which likely has
nothing to do with GIN indexes specifically) that you haven't fully
diagnosed or understood (and I don't understand it either, at least
not based on the information so far provided) and because that problem
is manifesting itself as an excess of WAL files, you're homing in on
this part of the documentation. And it may very well be that we need
some better documentation here, because I too have seen a few systems
lately with quite a lot of WAL files floating around for no
immediately obvious reason, but we can't document what is going on
until we understand it. If you're interested in troubleshooting this
further, I think you should post to pgsql-performance and try to get
some help understanding what is happening. If we get to the point
where we have a clear explanation for what is occurring, then we can
work out where and how to document it.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company