The research and design for a high availability object storage system

Ling Zhan; Zhihu Tan; Peng Gu; Jiguang Wan

doi:10.1117/12.822571

19 March 2009 The research and design for a high availability object storage system

Ling Zhan, Zhihu Tan, Peng Gu, Jiguang Wan

Proceedings Volume 7125, Eighth International Symposium on Optical Storage and 2008 International Workshop on Information Data Storage; 71251M (2009) https://doi.org/10.1117/12.822571
Event: Eighth International Symposium on Optical Storage and 2008 International Workshop on Information Data Storage, 2008, Wuhan, China

Abstract

With the growing scale of the computer storage systems, the likelihood of multi-disk failures happening in the storage systems has increased dramatically. Based on a thorough analysis on the fault-tolerance capability on various existing storage systems, we propose a new hierarchical, highly reliable, multi-disk fault-tolerant storage system architecture: High Availability Object Storage System (HAOSS). In the HAOSS, each object has an attribute field for reliability level, which can be set by the user according to the importance of data. Higher reliability level corresponds to better data survivability in case of multi-device failure. The HAOSS is composed of two layers: the upper-layer and the lower-layer. The upper-layer achieves the high availability by storing multiple replicas for each storage object in a set of storage devices. The individual replicas can service the I/O requests in parallel so as to obtain high performance. The lower-layer deploys RAID5, RAID6 or RAID_Blaum coding schemes to tolerate multi-disk failures. In addition, the disk utilization rate of RAID_Blaum is higher than that of multiple replicas, and it can be further improved by growing the RAID group size. These advantages come at the price of more complicated fault-tolerant coding schemes, which involve a large amount of calculation for encoding and cause an adverse impact on the I/O performance, especially on the write performance. Results from both our internal experiments and third-party independent tests have shown that HAOSS servers have better multi-disk- failure tolerance than existing similar products. In a 1000Mb Ethernet interconnection environment, with a request block size of 1024KB, the sequential read performance for a HAOSS server reaches 104MB/s, which is very close to the theoretical maximum effective bandwidth of Ethernet networks. The HAOSS offers a complete storage solution for high availability applications without the compromises that today's storage systems require in either performance or fault-tolerance.

Citation Download Citation

Ling Zhan, Zhihu Tan, Peng Gu, and Jiguang Wan "The research and design for a high availability object storage system", Proc. SPIE 7125, Eighth International Symposium on Optical Storage and 2008 International Workshop on Information Data Storage, 71251M (19 March 2009); https://doi.org/10.1117/12.822571

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available