Re: New Linux Filesystem: NILFS

pgsql(at)j-davis(dot)com (Jeff Davis) writes:
> On Tue, 2006-09-05 at 23:28 -0400, mark(at)mark(dot)mielke(dot)cc wrote:
>> On Tue, Sep 05, 2006 at 05:54:50PM -0700, Jeff Davis wrote:
>> > On Tue, 2006-09-05 at 18:24 -0400, Chris Browne wrote:
>> > > Recently seen in ACM Operating Systems Review (this is the first time
>> > > I've found as many as 1 interesting article in it in a while, and
>> > > there were 3 things I found worthwhile...):
>> > > ...
>> > > NILFS is a log-structured file system developed for Linux.
>> > As I understand LFSs, they are not ideal for a database system. An LFS
>> > is optimized so that it writes sequentially. However, PostgreSQL already
>> > ...
>> > Do you see an advantage in using LFS for PostgreSQL?
>>
>> Hey guys - I think the original poster only meant to suggest that it
>> was *interesting*... :-)
>>
>
> I see, my mistake.

From a reliability perspective, I can see some value to it...

I have seen far too many databases corrupted by journalling gone bad
in the past year... :-(

>> Applying any database on top of another database seems inefficient
>> to me. That's one reason why I argue the opposite - PostgreSQL
>> *should* have its own on disk layout, and not being laid out on top
>> of another generic system designed for purposes other than database
>> storage. The reason it isn't pursued at present, and perhaps should
>> not be pursued at present, is that PostgreSQL has other more
>> important priorities in the short term.
>
> I think that it would be a higher priority if someone showed a
> substantial performance improvement. Some filesystems don't really
> cause much overhead that isn't needed by PostgreSQL.
>
> If someone did show a substantial improvement, I would be interested
> to see it.
>
> And if there is an improvement, shouldn't that be a project for
> something like Linux, where other databases could also benefit? It
> could just be implemented as a database-specific filesystem.

The classic problem with log structured filesystems is that sequential
reads tend to be less efficient than in overwriting systems; perhaps
if they can get "vacuuming" to be done frequently enough, that might
change the shape of things.

That would be a relevant lesson that _we_ have discovered that is
potentially applicable to filesystem implementors.

And I don't consider this purely of academic interest; the ability to:
a) Avoid the double writing of journalling, and
b) Avoid the risks of failures due to misordered writes
are both genuinely valuable.
--
output = reverse("ofni.sesabatadxunil" "@" "enworbbc")
http://cbbrowne.com/info/lisp.html
All ITS machines now have hardware for a new machine instruction --
PFLT Prove Fermat's Last Theorem.
Please update your programs.