Re: GIN data corruption bug(s) in 9.6devel

From: Teodor Sigaev <teodor(at)sigaev(dot)ru>
To: Jeff Janes <jeff(dot)janes(at)gmail(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, Noah Misch <noah(at)leadboat(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: GIN data corruption bug(s) in 9.6devel
Date: 2016-04-12 16:53:00
Message-ID: 570D27EC.9010203@sigaev.ru
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> There are only 3 fundamental options I see, the cleaner can wait,
> "help", or move on.
>
> "Helping" is what it does now and is dangerous.
>
> Moving on gives the above-discussed unthrottling problem.
>
> Waiting has two problems. The act of waiting will cause autovacuums
> to be canceled, unless ugly hacks are deployed to prevent that. If
> we deploy those ugly hacks, then we have the problem that a user
> backend will end up waiting on an autovacuum to finish the cleaning,
> and the autovacuum is taking its sweet time due to
> autovacuum_vacuum_cost_delay. (The "helping" model avoids this
> problem because the user backend can just catch up with and pass the
> io-throttled autovac process)
With pending cleanup patch backend will try to get lock on metapage with
ConditionalLockPage. Will it interrupt autovacum worker?

>
> For completeness sake, a fourth option would to move on, but only
> after inserting the tuple directly into the main index structure
> (rather then the pending list) like would be done with fastupdate off,
> once the pending list is already oversized. This is my favorite, but
> there is no chance of it going into 9.6, much less being backpatched.
Agree, will reimplement for 9.7
>
>
> Alvaro's recommendation, to let the cleaner off the hook once it
> passes the page which was the tail page at the time it started, would
> prevent any process from getting pinned down indefinitely, but would
> not prevent the size of the list from increasing without bound. I
> think that would probably be good enough, because the current
> throttling behavior is purely accidentally and doesn't *guarantee* a
> limit on the size of the pending list.
Added, see attached patch (based on v3.1)
--
Teodor Sigaev E-mail: teodor(at)sigaev(dot)ru
WWW: http://www.sigaev.ru/

Attachment Content-Type Size
gin_alone_cleanup-4.patch binary/octet-stream 7.2 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Stas Kelvich 2016-04-12 16:53:22 Re: Speedup twophase transactions
Previous Message Kevin Grittner 2016-04-12 16:49:25 pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold <