Published on Nov 15, 2015
Despite seemingly endless increases in the amount of storage and ever decreasing costs of hardware, managing storage is still expensive. Additionally, users continue to fill increasingly larger disks, worsened by the proliferation of large multimedia files and high-speed broadband networks. Storage requirements are continuing to grow at a rate of 50% a year.
Worse, existing hard disk technology is reaching physical limitations, making it harder and costlier to meet growing user demands . Storage management costs have remained a significant component of total storage costs. Even in the '70s, storage costs at IBM were several times more than hardware costs, and projected that they would reach ten times the cost of the hardware. Today, management costs are five to ten times the cost of underlying hardware and are actually increasing as a proportion of cost because administrators have a limited amount of storage each can manage. Up to 47% of storage costs are associated with administrators manually manipulating files .
Thankfully, significant savings are possible: studies show that over 20% of all files--representing over half of the storage-are regenerable . Other studies, indicate that 82%-85% of storage is allocated to files that have not been accessed in more than a month. The studies shows that storage management has been a problem in a past, continues to be a problem today, and is only getting worse--all despite growing disk sizes.
Recent trends have begun to address the management of storage through virtualization . Morris put forth the idea of Autonomic Computing, which includes "the system's ability to adjust to its configuration and resource allocation to achieve predetermined goals" . Elastic Quota system is designed to help the management problem via efficient allocation of storage while allowing users maxi-mal freedom, all with minimal administrator intervention.
Elastic quotas enter users into an agreement with the system: users can exceed their quota while space is available, under the condition that the system will be able to automatically re-claim the storage when the need arises. Users or applications may designate some files as elastic. When space runs short, the elastic quota system (Equota) may reclaim space from those files marked as elastic; nonelastic files maintain existing semantics and are accounted for in users' persistent quotas . This report focuses on policies for elastic space reclamation and is organized as follows.
Section 2 describes the overall architecture of the policy system. Section 3 discusses the various elastic quota policies. In Section 4 we discuss interesting implementation aspects of Elastic Quota. Section 5 presents measurements and performance results of various policies. Section 6 discusses work related to storage space management policies. Finally, Section 7 presents some concluding remarks and directions for future work.
The primary design goals were to allow for versatile and efficient elastic quota policy management. An additional goal was to avoid changes to the existing OS to support elastic quotas. To achieve versatility the Elastic Quota system is designed with a flexible policy management configuration language for use by administrators and users; a number of user-level and kernel features exist to support this flexibility.
To achieve efficiency the design allows the system to run as a kernel file system with DB3 databases accessible to the user-level tools. Finally, the design uses a stackable file system to ensure that there is no need to modify existing file systems such as Ext3