Projects / Weed-FS

Weed-FS

Weed-FS is a simple and highly scalable distributed file system. There are two objectives: to store billions of files, and to serve the files fast! Instead of supporting full POSIX file system semantics, it implements only a key-file mapping. Instead of managing all file metadata in a central master, it manages file volumes in the central master and lets volume servers manage files and the metadata. This relieves concurrency pressure from the central master and spreads file metadata into volume servers' memories, allowing faster file access with just one disk read operation. It is modelled on Facebook's Haystack design paper. Only 40 bytes of disk storage are required for each file's metadata, and disk reads are O(1).

Tags
Licenses
Operating Systems

RSS Recent releases

  •  21 Oct 2013 02:40

    Release Notes: The weed upload tool now will keep the timestamp when saving files. The weed volume server correctly sets volume status after compacting.

    Screenshot

    Project Spotlight

    SlackPkg

    An automated package-management tool for Slackware Linux.

    Screenshot

    Project Spotlight

    Genson

    A Java library for doing conversion between JSON and Java.