Cheap secondary storage options

Looking at cheap storage for hosting backup/archive… driving force: cost.  NetApp is too expensive.

The following can be taken into consideration:

— Amazon S3 web services. Since it’s only used for backup/archive the cost should not be too high. Need to analyze the cost though. Another potential issue is legal/privacy…

     Technically the best way is probably to use a S3 file system driver so the backup/access is transparent to the apps and existing apps doesn’ t need to be modified. the following is a list of  S3 file system drivers, for example:

  •   Fuse over Amazon: http://

     Note that  the Hadoop project provides two file systems that uses S3: . However seems you have to use the hadoop to access the file systems and they are not accessible by normal apps.

— GFS-like distributed file systems, So that we can use cheap/commodity intel hardware to construct the storage cluster. Currently there are two open source GFS like DFS implementations.:

  • CloudStore ( formerly Kosmos File System / KFS):   quoted form the web site: “Web-scale applications require a scalable storage infrastructure to process vast amounts of data. CloudStore (formerly, Kosmos filesystem) is an open-source high performance distributed filesystem designed to meet such an infrastructure need” It’s written in C++ and can be mounted as a file system via FUSE on linux.
  • Hadoop HDFS File System:  part of the Hadoop Core project.   Hadoop is developed in Java.  There is also some effort to mount HDFS on linux/systems:

About: mmpower

Software Architect & Soccer Fun 黑超白袜 = IT 民工 + 摇滚大叔

Leave a Reply

Your email address will not be published. Required fields are marked *