Re: We really ought to do something about O_DIRECT and data=journalled on ext4

From: Josh Berkus <josh(at)agliodbs(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: We really ought to do something about O_DIRECT and data=journalled on ext4
Date: 2010-12-01 17:58:24
Message-ID: 4CF68CC0.2080404@agliodbs.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Tom,

> Well, no, actually it's the same (only) argument. We'd never consider
> back-patching such a change if our hand weren't being forced by kernel
> changes :-(

I think we have to back-patch the change. The way it is now, a DBA who
thinks they are doing normal sensible configuration can cause PostgreSQL
to fail to restart. Imagine this scenario, for example:

1) DBA, using PostgreSQL 8.3, gets worried about possible disk issues
2) DBA changes their single Ext3/4 partition to "data=journal"
3) DBA restarts system
4) PostgreSQL won't start
5) DBA thrashes around for a few hours while the site is down
6) DBA gets fired and the new DBA migrates to some other DBMS.

I simply can't think of *anywhere* we could put the information about
opensync and Linux/Ext which would be prominent enough to avoid the
above scenario. And per replies, a lot of people have hit this issue
already.

It's a bug and it's our bug. Back when we added O_DIRECT, we assumed
that support for O_DIRECT/opensync could be determined on an OS/kernel
basis, because that was the information we had. Now it turns out that
support can vary *by filesystem* and *between remounts*. We didn't have
any way of knowing different back in 2004, but that doesn't mean we
don't need to fix our mistaken assumption now.

Ideally, we would change our code to test support for O_DIRECT on
startup, rather than at compile time, and backport *that*.

--
-- Josh Berkus
PostgreSQL Experts Inc.
http://www.pgexperts.com

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Josh Berkus 2010-12-01 18:01:20 Re: Hypothetical Indexes - PostgreSQL extension - PGCON 2010
Previous Message Ana Carolina Brito de Almeida 2010-12-01 17:48:40 Hypothetical Indexes - PostgreSQL extension - PGCON 2010