raw device mapping

What is raw device mapping (RDM)?

Raw device mapping (RDM) enables disk access in a virtual machine (VM) in the VMware server virtualization environment and allows a storage logical unit number (LUN) to be connected directly to a VM from the storage area network (SAN). Because the VM can directly access the LUN, it helps improve disk access performance, particularly in I/O-intensive operations.

Raw device mapping explored

Apart from RDM, VMware servers can also use a Virtual Machine File System (VMFS) to access storage. RDM is a mapping file that resides in a separate VMFS volume that maps a LUN directly to a VM. With RDM, any VM within the VM cluster can access the storage array, just like any other disk.

The RDM file is created when vSphere administrators configure the VM's virtual disk to point directly to a LUN on a storage array. It acts as a proxy and consists of metadata to manage and redirect the VM to raw physical storage devices. This metadata specifies:

• Compatibility modes. A physical mode for SAN-aware agents in the VM and a virtual mode for SAN snapshots and advanced file-locking operations.
• Dynamic name resolution. Identification data about each raw LUN, which is then resolved by the VMFS based on the current SCSI
• RDM with VM clusters. This is suitable for VM clusters that need to access the same SCSI device in failover
• SCSI device access modes. Three options are provided -- vDisk file on the VMFS data store, physical mode RDM and virtual mode RDM.

RDM contains the physical storage device's raw ID. The VM reads this ID to automatically send the data it reads and writes to the storage device. Thus, it no longer references the VMFS data store.

Raw device mapping use cases

RDM is suitable in the following use cases:

• in cluster configurations like VM-to-VM or physical-to-VM;
• in SAN snapshots;
• when multiple VMs are clustered; and
• in operations that depend on disk I/O -- e.g., high-performance computing (HPC).

The need for raw device mapping

Although VMFS data stores are commonly used for virtual disk storage, in some cases, raw LUNs or logical disks in SAN need to be accessed. In these situations, RDM is required.

Two such situations are as follows:

1. When the SAN snapshot or other layered applications run in the VM, RDM uses the features inherent to the SAN to enable backup offloading systems.
2. In a Microsoft Cluster Service (MSCS) scenario with physical hosts, the cluster data and quorum disks are configured as RDMs.

In these situations, RDM links a VMFS volume to a raw LUN so that the LUNs appear as files in a VMFS volume. The RDM contains a reference to the raw LUN and is referenced in the VM configuration.

Raw device mapping compatibility modes

RDM offers physical and virtual compatibility modes. The choice of mode depends on the features required within the specific environment.

Physical compatibility mode. The physical compatibility mode is also known as pass-through RDM, or pRDM. Here, the physical properties of the underlying storage unit or LUN is exposed to the guest OS within the VM. This mode is appropriate for SAN-aware agents in the VM and provides minimal SCSI virtualization of the raw LUN. It enables all SCSI commands to be passed directly to the guest OS, so the VM can effectively use several lower-level storage functions as required.

Physical RDM is preferred in environments where:

• SAN management agents and SCSI-based software run in the VM;
• cost-effective virtual-to-physical clustering is required; and
• physical-to-virtual clustering is required.

Virtual compatibility mode. In the virtual compatibility mode, RDM acts like a virtual disk file by virtualizing the physical properties of the underlying storage. Thus, it appears how a vDisk file would appear on a VMFS volume. Barring read and write commands, all commands are virtualized and sent directly to the raw LUN. In this mode, the RDM can use snapshots. Further, virtual RDMs are more flexible for moving throughout the environment.

Virtual RDMs are preferred for:

• cluster-in-a-box and cluster-across-boxes;
• VM snapshots and VM cloning; and
• file locking.

Advantages of raw device mapping

RDM offers numerous benefits in many use cases, including guest OS clustering, SAN snapshots, file system operations, distributed file locking and user-friendly persistent names.

These benefits are as follows:

• The VMFS distributed locking feature for SCSI devices enables the sharing of raw LUN without data loss.
• The guest OS can treat the mapping file like a vDisk file.
• RDM eliminates the need to use the device name to refer to SCSI devices with user-friendly persistent names.
• RDM supports three ways to implement clusters using VMs:
  • Cluster-in-a-box (CIB)
  • Cluster-across-boxes (CAB)
  • Physical-to-virtual cluster
• VMs can utilize file system utilities with the raw LUN.

Limitations of raw device mapping

RDMs are suitable when raw devices are required. However, in general, virtual disk files are preferable to RDMs, which have the following limitations:

• RDM cannot be used in direct-attached block devices and some redundant array of independent disks (RAID) devices as they don't support serial numbers to identify the SCSI device.
• The SAN snapshot cannot be used if the RDM is used in physical compatibility mode.
• Redirecting or mapping to a partition is not possible because RDMs need the whole LUN.
• Flash Read Cache, which enables a host to use a solid-state drive (SSD) as a caching layer for VMs, doesn't support RDMs in physical compatibility mode.

Additionally, migrating a live VM from one physical server to another must be done carefully. When migrating VMs with VMware's vMotion -- which is designed to simplify the VM migration process -- consistency must be maintained in LUN IDs for mapping across participating hosts.

See also: Getting VMware terminology straight, VMFS vs. RDM: Which cluster file system should you use? and how to keep virtual machine snapshots in check.