Q. Huang, W. Zheng, G. Yang, Y. Wu, and M. Shen (PRC)
I/O load balancing, file allocation, file migration, cluster file system, file grain
The increasing load and diverse users' access patterns of today's cluster file servers prompt for a flexible file system design that can achieve dynamic load balancing across multiple nodes. This requirement becomes more necessary and complicated when the cluster is inevitably heterogeneous and non-dedicated. Realizing that current cluster file systems do not take those complexities into account, we present an adaptive, light-weight file-grain data distribution mechanism in TH-CluFS, a portable cluster file system developed in Tsinghua University. Profiting from the flexibility of the per file policy, this approach combines static file allocation with dynamic file migration effectively. It not only tailors file system behavior to environmental changes and individual file's characteristics, but also introduces little overhead with a decentralized mechanism and running expensive dynamic file migration component only when some nodes become overloaded. Preliminary analyses and experiments with our prototype have yielded good results, although mathematical validation and practical studies are still underway.
Important Links:
Go Back
