3. Software-Defined Storage (SDS) for Storage Virtualization

The proliferation of Machine-to-Machine (M2M) communication is exponentially increasing the amount of unstructured data that is generated in the digital universe. It is estimated that face book generates roughly about 1 TB of data every day and most of it is unstructured data. The storage infrastructures like Storage Storage Area Networks (SAN) and Network-Attached Storage (NAS) are not designed to store and process unstructured data. Hence, it is the need of the day to design storage devices and networks which are robust enough to scale and accommodate huge amount of unstructured data without causing any performance impact. However, any present-day storage infrastructure like software-defined storage uses the traditional storage technologies like NAS and SAN as their underlying foundation. Hence, it is impossible to understand software-defined storage platforms without proper understanding of underlying technologies. In this chapter, we are explaining the foundations of storage technologies in the beginning and, later on, we are doing a deep dive to understand how their design has been transformed in such a way that they can store and process huge volumes of data in the present-day data centers using the concept of software-defined storage. In most of the scenarios, multiple storage platforms are combined and then some kind of enhancement is added to make them compatible to handle large volumes of data. In the first half of the chapter, we are examining the storage technologies like Direct-Attached Storage (DAS), NAS, and SAN. Software-defined storage refers to the use of storage virtualization and policy-based provisioning techniques to ensure that data storage is done in a cost-optimized and performance-optimized manner. One of the main reasons for the evolution of software-defined storage is the necessity to store and process big data. In the second half of the chapter, we discuss and focus on all techniques related to software-defined storages like Cloud storage and storage tiering and we also cover latest storage technologies which have been designed and optimized for big data processing like PANASAS, GFS, and HDFS