Nutanix Complete Cluster: Historical Hyperconverged Architecture

This article describes the early Nutanix Complete Cluster architecture in the archived source. Names such as SOCS, HOTcache, FlashStore, DiskStore and the Flex-based Command Center belong to that design; this is a historical architecture reference, not a current release deployment or compatibility guide. Performance, availability and retention claims depend on the actual workload, configuration and failure conditions.

1. Introduction

Storage was presented as a major challenge for virtualized data centers. The source contrasts network-storage systems designed for comparatively static physical servers with dynamic virtual machines that multiply, move between hosts and demand more capacity and performance. Without an enterprise alternative, organizations had to combine traditional solutions that did not fit those needs well.

Nutanix Complete Cluster combined compute and storage in a scale-out infrastructure, removing the requirement for a separate external SAN or NAS. It used industry-standard server hardware, SSDs and established hypervisors to deliver compute, storage, availability, scalability and data-management capabilities in an integrated system.

2. Limitations of the traditional architecture

The source contrasts conventional data centers, illustrated by Yahoo/AltaVista and servers, switches, SAN and NAS, with cloud-scale centers such as Google, Facebook and Amazon using commodity hardware, MapReduce, distributed file systems and NoSQL.

Traditional data-center equipment in the source comparison.

Cloud-scale commodity-server infrastructure in the source comparison.

Connecting servers to storage across a network made sense for relatively stable physical workloads. Virtualization and cloud computing made workloads more dynamic: VMs could be created quickly, moved between servers and depended on shared resources. Managing each VM and its underlying infrastructure became more complicated.

Desktop virtualization and service-provider workloads increased VM counts and data volumes. More VMs placed pressure on the cost, performance and management of architectures that separated compute from storage over multiple network hops.

The source cites SSD access performance as 100–1000 times that of mechanical disks and argues that external-network bottlenecks could hide that advantage. This is the historical source’s broad comparison, not a measured ratio for every drive or workload. It also describes the cost of adding SSDs to SAN/NAS systems and increasing network bandwidth to reach them.

The Google approach

Google and other large cloud operators, including Amazon, Yahoo and Microsoft Azure, built distributed software to combine commodity servers and their local storage. The Google File System is the source’s example: applications across a cluster can access a unified storage pool while the distributed system handles disk and server failures.

The article presents Nutanix as applying a similar scale-out idea to a general-purpose enterprise virtualization platform. It does not claim that Nutanix is the Google File System or that all of those companies used an identical design.

3. The three architectural pillars

Distributed computing

Compute and storage scale together without requiring a separate SAN/NAS array. Unlike Google’s application-specific internal file system, the Nutanix design targets general virtualized environments and combines scale-out capacity with enterprise functions such as high availability, snapshots, backup and disaster recovery.

Virtualization

The design supports the hypervisor functions described by the source, including live VM migration and high availability. It is aware of individual VMs, rather than exposing storage management only through a LUN shared by many VMs. This makes per-VM backup, recovery and snapshots easier to reason about and helps associate storage activity with the VM that generates it.

Solid-state drives

Mechanical disks incur rotation and seek delays; SSDs have different access characteristics. Software designed only around rotating media cannot automatically make the best use of flash. The architecture therefore uses SSDs both for frequently accessed VM data and metadata, with distributed caching and persistent storage.

• Reserve SSD resources for I/O-intensive functions.
• Use capacity-saving techniques to hold more logical data in less physical space.
• Move cold or infrequently accessed data to HDDs automatically; allow lower-priority VM storage to bypass the SSD tier when configured.

4. Architecture overview

Each node is a server containing processors, memory, SSDs and HDDs, running a standard hypervisor and guest VMs. Nutanix Scale-Out Converged Storage (SOCS) virtualizes the nodes’ local drives into a unified pool. The source compares this pool to an advanced SAN built from the cluster’s local media.

VMs write to SOCS through virtual disks, or vDisks. SOCS places data close to the VM where possible while providing VM-level management. Capacity and resources grow as nodes are added. Figure 1 depicts the historical architecture, including the FusionIO label used in the original drawing:

The source describes these historical vDisks as standard iSCSI devices and says ordinary VM creation, live migration and high-availability workflows continue to work. This describes that architecture; it does not establish the storage protocol or hypervisor support of a different product release.

5. SOCS: main components

SOCS is the scale-out storage layer that coexists with compute. The source calls its techniques patent-pending, not already-granted patents. Its listed capabilities are per-VM provisioning and I/O awareness, scale-out growth, SSD integration, local-access optimizations, capacity efficiency and high availability against disk or node failures. High availability does not mean hardware cannot fail.

An n-way controller cluster

Where a traditional SAN might have 1, 2, 4 or 8 controllers, this design runs one Controller VM on each node: an n-node cluster has n controllers. The Controller VMs communicate as one distributed system, allowing controller resources to grow with the node count:

Heat-Optimized Tiering Cache (HOTcache)

HOTcache is backed by SSDs across the cluster. Guest writes enter the cache and are destaged in the background. A sequential data layout helps handle workloads that become random when many VMs are mixed together. The described path keeps one copy on a local SSD and another on a different node to protect against a node failure, subject to the actual replication and cluster state.

Distributed metadata service: Medusa

Keeping all metadata behind a small fixed set of controllers can limit storage scalability. Medusa distributes metadata across the cluster and replicates it to multiple nodes for fault tolerance. The source contrasts this with dependence on a few traditional controllers; tolerance of several failures depends on the actual placement, replication and health, rather than being unlimited.

Distributed data-maintenance service: Curator

Curator uses a MapReduce-based framework to carry out background work in parallel. The source lists these tasks:

• Move cold data to lower storage tiers for heat-based tiering.
• Reclaim deleted or unreferenced data through garbage collection.
• Scrub data to check consistency.
• Replicate data after disk or node failures.
• Rebalance data when nodes are added or removed.
• Move data after VM migration to maximize local access.

FlashStore

FlashStore is the persistent flash-backed tier formed from SSDs on the nodes. Data is initially placed there, moved to DiskStore as it becomes cold, and promoted back when it becomes hot again. The source allows administrators to bypass FlashStore for vDisks belonging to low-priority VMs.

DiskStore

DiskStore is the cluster-wide, high-capacity SATA tier for cold data. Heat-Optimized Tiering (HOT) combines FlashStore and DiskStore to balance performance, capacity and cost.

6. Capacity optimization

QuickClone

QuickClone creates writable snapshots that behave as ordinary vDisks. Administrators can attach them to VMs, write to them and take further snapshots. The source gives virtual-desktop deployment and test/development copies of production databases as examples that avoid copying an entire data set. Read-only snapshots are also available for backup purposes.

Thin provisioning

A vDisk has an assigned logical capacity, while physical space is allocated as needed. The described minimum-reservation setting can reserve a specified amount of capacity for a vDisk.

Integrated backup

The source describes immediate vDisk backup and recovery with on-appliance retention and restore to an earlier backup point. Its reference to keeping several months without external storage depends on capacity, data change and retention policy; it is not a fixed retention guarantee. Standard third-party tools can also provide off-site backup. A local snapshot is not by itself an independent off-site copy.

7. Management and interoperability

Deployment

Complete Cluster was presented as an integrated hardware/software system for virtual servers and desktops, intended to be set up quickly before administrators create VMs. The source’s claim that setup can take minutes is a product-era description, not a deployment-time commitment.

Command Center and CLI

The original Nutanix Command Center is described as a Flex-based interface for cluster compute and storage resources. Mapping VMs to their physical resources helps investigate VM-related problems. A command-line interface is also provided. SOCS aims to reduce the work of managing a separate network-storage infrastructure as the environment grows.

Existing IT tools

The source uses VMware ESXi as its standard-hypervisor example, explaining that existing compatible management tools can continue to operate. It describes vDisks connecting through the hypervisor’s standard iSCSI initiator. Confirm the actual supported hypervisor, software version and protocol when working with a real deployment.

8. Performance and scalability

SSD use for data and metadata

The design places SSDs in the servers to reduce dependence on an external SAN path. In addition to VM data, SSDs hold SOCS metadata for quick access and richer management functions. Aggregate SSD resources grow as nodes are added. Locality is an optimization; cluster communication and replication still use the network.

Heat-Optimized Tiering

A vDisk can use a mixture of SSD and HDD capacity. HOT keeps valuable or active data in flash and moves colder data to the large SATA tier.

Scale-out growth

There is no single centralized metadata controller in the described design: metadata and Controller VMs are distributed across the nodes. This avoids dependence on only a few storage controllers and allows an installation to start small and add resources. It does not mean that every workload or component scales without practical limits.

9. High availability and data protection

Nutanix Cluster RAID

The historical source uses the term Cluster RAID for placing data across disks for performance and replicating it across nodes for availability. The aim is to keep VM storage available through supported disk or node failures. This is the source’s distributed protection concept, not an instruction to create a conventional RAID group or a promise of surviving arbitrary concurrent failures.

Medusa redundancy

Distributed, replicated metadata supports continued service when components fail, within the cluster’s configured protection and health limits. The source repeats this as a contrast with a small fixed set of storage controllers.

Instant vDisk Motion

The design supports live migration even though storage is built from local node media. A vDisk is a movable logical entity; the described Instant vDisk Motion capability can move its data between nodes when needed, keeping storage aligned with VM placement. Live migration is a hypervisor capability integrated with that storage design.

Backup and recovery

Off-site protection is part of the source’s data-protection strategy. It describes compatibility with VMware vStorage APIs for Data Protection (VADP) and integration with compatible third-party backup/recovery products for VM protection.

Disaster recovery

The article describes failover and recovery through those compatible backup/recovery integrations to support business continuity. Recovery capability depends on the actual configured copies, products and tested procedures.

10. Conclusion

The source presents Complete Cluster as converging compute and local storage into a scale-out platform for VMs, with integrated performance, availability and data-management functions. Its central idea is to grow resources with the cluster while reducing reliance on a separate external storage array.