Since VMworld 2018 in Las Vegas, VMware has “sneak peeked” many details about the upcoming vSAN version features. I recall that it is going to be released in Q1/Q2 this year (it is in beta yet). And, considering that we approach closer and closer to the release date, I’d like to take a deeper look at some new technologies that we are so excited to see in the upcoming VMware vSAN version.

VMware vSAN Native Data Protection

VMware vSAN Native Data Protection is a snapshot-based SPBM-driven (Storage Policy Based Management) technology for virtual machine storage replication. It is also good for making VM backups.

Here are vSAN Native Data Protection benefits and overview:

vSAN Native Data Protection

Why one needs vSAN Native Data Protection? Here are three scenarios:

  • vSphere local virtual machine protection without using vSphere snapshots
  • Virtual machine data replication to an NFS share
  • Virtual machine data replication to another site (i.e., a vSAN cluster). Be it under orchestration of the same vCenter server or a different one, vSAN Native Data Protection works good.

vSAN Local Data Protection

For local storage, this tech is called vSAN Local Data Protection. It, in its turn, relies on native vSAN snapshots working on the storage level. Therefore, this feature does not alter VM performance. This tech also allows creating the application-consistent snapshots which can use Microsoft VSS / VMware Tools scripts for “freezing” applications.

Create VM

Linked Clone

For restoring VMs from the local snapshots, Linked Clone is used. With this technology in place, restoring a VM takes around a minute. Of course, for a standalone VM, this process takes more time and is determined by its storage capacity. While restoring a VM, you can select a destination cluster and VM Network.

Here are some things that are good to know about this technology:

  • It is not integrated with vSAN Native Data Protection and SRM
  • VMware is planning to make VM backups group-consistent. Linked Clone allows backing up groups of VMs even if they reside on different storage.
  • RPO can be shrunk to 5 minutes.
  • Use Microsoft VSS and own scripts to preserve application consistency of your backups.
  • The technology works with any VADP-compatible backup solution.
  • Snapshot-driven replication to the remote storage.
  • If storage has only crash-consistent snapshots, its snapshot is taken instantly.
  • You can replicate data between clusters even if they are orchestrated by different vCenter servers.
  • Only NFS share is yet available for archive storage. Amazon S3 support will be introduced soon.
  • Inline deduplication and compression of the native snapshots.

vSAN Native Data Protection is going to be introduced in Q1 2019.

First Class Disk (FCD) support

Working with VMDKs using just the traditional means is pretty inconvenient since you need VMs for managing such objects. Many just create a one-time virtual machine and delete it once they are done with the task. Here are some cases when you could need a temporary VM in vSphere environment:

  • App Volumes Backup Utility requires a temporary Backup VM:

App Volumes Backup Utility requires a temporary Backup VM

  • In vSphere Integrated OpenStack (VIO) infrastructure, you need a Shadow VM for each Cinder (OpenStack Block Storage) volume for agglomerating VMDK disks that are connected to Cinder volumes.

Obviously, creating a virtual machine every time you need to do something with a VMDK is a bit inconvenient. That’s actually why the First Class Disk (FCD) format emerged. With that service in place, you do not need a temporary VM to perform lifecycle tasks on VMDKs (i.e., creating, deleting, backing up, etc.).

VMware vSAN already supports FCDs. Sometimes, they are referred to as Improved Virtual Disks (IVDs) or Managed Virtual Disks (MVDs). In today’s article, I call them FCD.

VMware App Volumes, which are able to separate the OS from the application and user layers, are one of FCD examples. App Volumes are connected to a virtual machine only when a user works with applications on that volumes.

Another example of FCD is storage for cloud-native and container-based applications working on Docker Plugin and Kubernetes driver. Such applications allow creating Docker container persistent volumes. Speaking of containers, check out what people from VMware Project Hatchway do if you feel enthusiastic about that kind of tech.

All the information about FCD is kept in the vCenter database. In particular, they contain UUID and disk names allowing to ensure disk copying between environments without any conflicts.

APIs for working with FCD were introduced back in VMware vSphere 6.5. Here’s a good post on vSphere 6.5 APIs by William Lam. But, before, there were some limitations for backing up FCDs that are not connected to VMs. You could use Dummy VM, though.

In vSphere 6.7, this constraint is gone, but some requirements are still there. While restoring a FCD you had to keep UUID and restore the disk to the original datastore. And, last but not least, API could not do incremental backups, because it could not track changed blocks.

With vSphere 6.7 Update 1, vSAN is said to get limited FCD support. For now, health service and capacity monitoring are still poorly supported. But, Kubernetes can use FCD disks on vSAN storage for persistent storage of containers. At the same time, these vSAN volumes can be used for virtual machines.

vSAN Scalable File Services

vSAN not only provides underlying storage for VMDKs and FCDs but also allows presenting logical volumes over iSCSI.

vSAN File Services brings SMB and NFS support to vSAN. With these protocols, you can provide underlying storage for a Windows Server file share without creating a VM! NFS/SMB file shares are SPBM-based and they will support stretched clusters. SPBM will allow you to use various services like FTT (ESXi fault tolerance), encryption, and disk thin provisioning.

File Shares in vSAN

File sharing mechanisms work on vSAN Distributed File System (vDFS). This allows aggregating vSAN disk objects and presenting them over the network while management services are provided by Storage Services Platform. These services will be provided within vCenter and vSphere Client interfaces. There you can also set policies, quotas, and other options.

File Server Services is another cool thing about the new vSAN version. During VMworld 2018 these services were presented as Virtual Appliances. vSAN File Servers implement folder export and provide advanced mechanisms for detecting failures of these appliances. If something goes wrong with one of your VMs, it is restarted on another server. In this way, this feature allows upgrading ESXi hosts non-disruptively for file services. Furthermore, vSAN File Services can provide resources for Kubernetes containers on the file level.

Amazon EBS backed VMware vSAN

Amazon Elastic Block Store (EBS) support is last but not least vSAN feature sneak peeked during VMworld 2018. VMware called it EBS backed vSAN first, but later it was referred as VMware Elastic vSAN. This feature will be available very soon.

Elastic vSAN under the hood

Let’s start with the story behind this innovation. Initially, VMware vCloud on AWS, the service for building virtual infrastructure in AWS, was based on I3.Metal instances in the EC2 cloud storage. The problem with such infrastructure was its scalability.

For users with huge storage capacity, using EC2 was quite expensive. One I3.Metal instance came with 10TB of storage. The thing is that one could not just purchase more storage. Every time you ran out of storage, you had to pay for another instance, including both its storage and compute resource. Obviously, most of compute power remained unused in an infrastructure like that, so deploying virtualized infrastructure in EC2 was pretty like leaving money on the table.

The solution that VMware came up with was switching to R5.Metal instances connected to the scalable Elastic Block Storage. That’s VMware’s take on building scalable and simple all-cloud environments.

VMware Cloud on AWS

While creating an Elastic vSAN diskless host, users can fine-tune the storage capacity on it. The overall instance capacity ranges from 15 through 35 TB with a 5TB step. The infrastructure is built from EBS components.

Every host has 3 disk groups, each has 3-7 disks in the capacity tier. The image below represents how an Elastic vSAN cluster looks like:

Elastic vSAN cluster

Now, let’s talk about the requirements of this architecture. RAID-5 is highly recommended (note that you need at least 4 instances for that). You also need to set Compression only mode for smart disk utilization. With compression in place, deduplication is not needed (actually, you just won’t be able to enable it due to performance reasons).

R5.Metal: What’s the diff?

You also need remarkably fewer hosts than if you were running a virtualized environment with I3.Metal instances. 4-node cluster provides you up to 140TB of raw capacity, while 16-node one delivers around 560TB. This is very cool for services that need only little compute power (i.e., file shares).

I3.Metal instances are also notoriously hectic in management. For instance, before maintenance of such instance or a vSAN cluster, you had to migrate all VMs with their storage to another host. Obviously, it often takes like tons of time.

Things look not that tough with diskless R5.Metal instances! If you need to shut down the host, EBS allows you just to reconnect its storage quickly to another instance. And, that’s it! Obviously, many will enjoy the new maintenance workflow.

Apart from simplifying the maintenance, this architecture provides a room for flexibility. You can build Elastic vSAN environments that scale easier than any on-premises one. VMware says Elastic vSAN to be capable of delivering 10K IOPS and 160 Mb/s throughput regardless of the volume size.

VSAN from StarWind eliminates any need for physical shared storage just by mirroring internal flash and storage resources between hypervisor servers. Furthermore, the solution can be run on the off-the-shelf hardware. Such design allows VSAN from StarWind to not only achieve high performance and efficient hardware utilization but also reduce operational and capital expenses.

Learn more about ➡ VSAN from StarWind


New VMware vSAN version that appears very soon is full of innovative features. VMware did a really good job on vSAN, tailoring it for all-cloud environments. Even though this article addresses the most interesting vSAN features, you should stay tuned. What if VMware “sneak peeks” something interesting about the upcoming release?

Views All Time
Views Today
Appreciate how useful this article was to you?
1 vote, average: 5.00 out of 51 vote, average: 5.00 out of 51 vote, average: 5.00 out of 51 vote, average: 5.00 out of 51 vote, average: 5.00 out of 5
5 out of 5, based on 1 review
Back to blog
The following two tabs change content below.
Alex Samoylenko
Alex Samoylenko
Virtualization technology professional. 10 years ago he built #1 website on virtualization in Russia. Alex runs his own virtualization-focused company VMC. He is a CEO of a mobile game publisher Nova Games and a CEO of an international dating site