VMDK Partition Alignment when using NFS…is it necessary?

As the Magic 8 Ball sometimes says “Concentrate and ask again”.

I was re-reading the “Recommendations for Aligning VMFS Partitions“, published by VMware, and noticed this paragraph (on page one no less…how did I miss it before?!):

Note: These recommendations are for block-based storage solutions, not those that are IP-based. I/O characteristics on NFS are different from those of Fibre Channel and iSCSI storage systems. Though partition alignment eliminates track crossings and benefits performance on all storage platforms, the throughput improvements in specific types of I/O between SAN, NFS, and iSCSI are different.

(Emphasis added by me.)

This is fairly significant, as many administrators have devoted a large amount of time (especially during template creation) to ensure that their VMs are aligned in order to gain the benefits highlighted in the publication.

The somewhat old (July 2007) TR-3593 (it’s location leads me to believe it’s been retired…) states:

Since a .vmdk is a virtual block device, a .vmdk still needs to be correctly aligned to an NFS Datastore mounted from a Network Appliance NFS share.

That same document also states that when using a NetApp filer, so long as you select the “VMware” type for the file system/OS, then the FlexVol is correctly aligned, but that does nothing for the VMDKs.

The above document’s statement is reinforced by this NetApp knowledge base article (NOW login required), which is significantly newer (September 2008)…

Partial writes occur when using VMFS filesystem or NFS based VMDKs on ESX 3.x

So, it appears that alignment of the guest OS’ partitions is indeed required, however using NFS you do not have to worry about the VMFS file system alignment (which is really a moot argument since ESX 3.5 does this for you).

I would be curious to know if anyone has done any testing with this. The infamous TR-3697 clearly and plainly states that they used correctly aligned VMDKs during their tests, so it isn’t much value in comparing aligned vs non-aligned VMDKs on NFS datastores.

A side note: there are some perfstat counters that can be checked (the specific counters are in the above NOW kb article) to determine if you are experiencing IO that is not aligned, so it will be interesting to see if it’s possible to distill any performance impact from misaligned IO on the filer side.

2 thoughts on “VMDK Partition Alignment when using NFS…is it necessary?”

  1. I’ve been thinking about the same thing. I ran across an IBM TR for a solution that runs on rebranded NetApp filers:

    ftp://service.boulder.ibm.com/storage/isv/NS3593-0.pdf

    With the relevant quote being:

    Since a .vmdk is a virtual block device, a .vmdk still needs to be correctly aligned to a network file system (NFS) datastore mounted from an IBM N series NFS share. To correctly align a .vmdk file on an NFS datastore, use the steps listed in “Aligning .vmdks to an IBM N series VMFS datastore.”

    Which again points us to the necessity of aligning our guest OS installs on NFS.

    Reply
  2. I’ve been reviewing the alignment tools for modifying or creating new Linux systems. Unfortunately there are a stack of issues with the NetApp tool we’ve been using. It apparently sets aside the existing disk image, creates a new and slightly larger image, creates aligned primary partitions to match the existing partitions but with their starting sector aligned (number divisible by 8), leaves an empty gap to start the next partition aligned, and directly images the old partitions into the new locations.

    This requires spare space on your NetApp, which is awkward when your existing disk images are large. It also doesn’t help, at all, if you want to install your Linux operating sytems aligned to start with.

    One solution is to use a smaller, secondary disk image and a bootable rescue CD to mirror the contents of the old OS, even using a compressed copy of its contents, onto the second disk. Then rebuild the first disk image using parted, *NOT* fdisk, to set partition boundaries on cylinders, at sector counts of (255 * 63 * 8).

    To build such an image, label the disk image as “msdos”, start the first partition at sector 64, start all following paritions at sector numbers divisible by (255*63*8), and end the previous sector at one less than that that boundary. fdisk will stop whinging about misaligned partitions, which the NetApp tool causes, and you’ll have reasonable amounts of space to install and set up your old file systems.

    It takes more time, but it’s the only way to make it work with RHEL kickstart: the “anaconda” tools from RedHat are not sophisticated enough to handle pre-aligned partitions or to align them within the kickstart installer.

    Reply

Leave a Reply