The Case for Disaggregated Storage

In this video we breakdown why we believe the future of storage architecture is disaggregated and how the data processing unit can realize storage disaggregation at all scales.
To learn more, visit: www.fungible.com
Follow us on Twitter: / fungible_inc
Follow us on LinkedIn: / fungible-inc.
#datacenter #technology #cloud #network #data #datacentric #datacenterarchitecture #datacentricsw #softwarearchitecture #fungible_inc
The case for disaggregated storage In today's hyper-converged infrastructure compute and storage are aggregated in a server the storage is attached to the compute in what is called a Direct Attached Storage or DAS model. The converged asked model worked well when storage protocols were slow and requirements for capacity were modest storage was placed near compute elements. Throughput and capacity were mainly constrained by the available slots in a server and storage media technology. Today modern workloads demand storage requirements beyond what can be supported in a DAS model and more importantly storage protocols have become so fast and efficient that storage no longer needs to be placed near compute elements. These trends are driving the disaggregation of storage. This aggregating storage involves taking away all the storage from the compute servers and grouping them together in a storage pool. Physically multiple compute and storage elements may be housed in separate compute and storage servers. So, what are the requirements to effectively disaggregate your storage. The commercialization of fast and modern SSDs opened up a slew of new use models and applications. However, the original protocol supporting SSDs such as SAS and SATA were archaic and inefficient. There is a need for an efficient protocol to communicate between the compute and storage nodes in a disaggregated model with the introduction of fast and efficient protocols such as NVMe and NVMe-oF over fabric to support cross fabric interconnectivity. High-performing low latency SSD storage can now be disaggregated outside of compute servers and access to latency that rival a direct-attached storage model.
Next is the need for a fast anywhere to anywhere network so that all the compute nodes can access storage very quickly this is called a network fabric. A storage node is commonly called a storage array and logically is made of a bunch of SSDs managed by a controller. It is crucial for the controller to be able to process data and access the SSDs at the maximum bandwidth and the lowest latencies for which the aggregated SSDs are capable of and that bandwidth needs to be pretty high to accommodate high demand content where a large number of compute nodes are attempting to access the same content simultaneously. Now what are the benefits of Disaggregated Storage compared to the traditional direct attached storage? Today's modern workloads demonstrate two attributes; first modern workload have a need for different hardware requirements. Some our compute heavy, some others are storage heavy. Second that can be really huge. Further at different times of the day the distribution of workload types may vary significantly. By Disaggregating Storage and then by pooling them virtually you can now create and manage larger shared data sets. These data sets enabled by ample bandwidth allow a huge number of compute nodes to tap into them to perform computations and gather insights. Otherwise limited by the traditional DAS model. To cater for the varied workloads, One possible solution is to over dimension your Hardware to fit the largest requirement. This of course is an expensive and wasteful solution as storage servers are underutilized. The second possible solution is to offer a menu of server variants. Deployment and management of servers become complex especially when the number of permutations within the
servers increase. In contrast with a disaggregated approach you can now design simpler compute and storage nodes. Server nodes can be homogeneous and pools of interconnected compute and storage resources can be flexibly apportioned to workloads. Further you now have the flexibility to upgrade replace or add specific resources. Instead of entire systems this pay as you grow planning allows for easier infrastructure management and better budget control. In the DAS model whenever you need to migrate workloads you move the data from one server to another. With a disaggregated approach because there is no state in a server there is no need to migrate workloads or move data. This stateless model is great for fault tolerance scalability and live deployment. Disaggregated storage also lets you take advantage of the inherent benefits of scale out architecture. If one of the storage knows failed data integrity can be protected by using techniques such as replication or erasure coding. These techniques are most effective when the storage nodes are geographically distributed.