Thursday, October 25, 2007

ZFS: disuptive technology?

Many years ago I thought that CORBA was a disruptive technology that would change the world for distributed systems. I sort-of guessed wrong - in retrospect I explain my poor prediction on the relatively high cost of early CORBA software. CORBA became 'free' too late.

I also think that ZFS (a new "file system" - yet so much more than a file system) also could be a disruptive technology that changes the world for storing and retrieving information, supporting: multiple levels of data protection, a transactional model for committing changes, on the fly expandability, etc. As open source software, wide and free availability is not a problem with ZFS, or at least should not be when ZFS is available for OpenBSD, FreeBSD, Linux, OS X, etc.

That said, I think that ZFS is more of a platform than a 'file system'. That is, I think that developers with specific data storage and retrieval problems (e.g., large scale graphs, databases, permanent record archival, etc.) will look to ZFS as a possible bottom data layer instead of file systems on the low end or relational databases on the high end.

As ZFS storage pools grow to span multiple (file) servers, we can start thinking in terms of using distributed data queries and accumulating results (a little like using map reduce operations on GFS). ZFS seems like such a wide open platform that I expect that we will all be surprised at future novel applications layered on top of ZFS.