PowerCLI: Balance LUN paths for a Cluster

As I’ve been managing more and more infrastructures using fibre channel storage, I’ve found that it’s been somewhat difficult to keep the LUN paths to each host balanced. By balanced, I mean that for each LUN to each host, there is a number of paths and I want to make sure that, for example, each LUN 10 to each of the hosts is using path A as the primary and path B as the stand by. LUN 11 uses path B as the primary, LUN 12 back to path A, and so on.

It so happens that I’m using a DMX-4 for storage, and the policy we have is to use a fixed path policy. I realize that Round Robin would make this entire script moot, well, except for making sure that the PSP is correct. I also realize that PowerPath would be the ideal solution for EMC storage, but we don’t use it…that’s a story for another day.

This script is, admittedly, long…longer than I expected it to be. The original inspiration for this script came from Justin Emerson’s very functional and succinct script, however I was not satisfied with the way LUNs were balanced. His script queries the host for LUNs then sorts them by canonical name and round robins the paths based on the number of paths present for the first LUN.

This works well, so long as all the LUNs are present on all the hosts and they all have the same number of paths. I can only presume that he assumes that those cases have already been checked for, and fixed, prior to execution. I wanted to do that all in one script.

Additionally, and it’s rather petty, I wanted the LUNs to be balanced based off their LUN identifier rather than the canonical name…they don’t always follow the same order, and in the case of my hosts with two HBAs (and consequentially, two paths per LUN), I wanted all odd LUNs to use one path for the primary and all even LUNs to use the other. Justin’s script does an excellent job of ensuring that the paths are evenly distributed, as you will end up with the same number on each, but not in the pretty fashion I desired.

Also, thank you to Glenn, who helped me “powershellize” this script…my PowerShell looks and reads like Perl, and therefore doesn’t use a lot of the optimizations that PoSH brings…such as automatic parameter handling and other niceties.

So, without further ado…

Read morePowerCLI: Balance LUN paths for a Cluster

PowerCLI: Show HBA Path Status

When you have a lot of hosts, with a lot of LUNs, it can be difficult to keep abreast of the status of the paths for them. I have encountered issues where a path was unknowingly marked as dead, plus it’s generally a good idea to ensure that your storage paths are actually available.

Consequentially, I searched for a PowerCLI script that would give me a simple report of the status of each of the LUN paths to each of the HBAs on my hosts. I found John Milner’s post to be very helpful, and it gave me exactly the results that I wanted. However, it took forever to execute…almost 30 minutes for just one of my clusters (to be fair, that cluster has 12 hosts with > 100 LUNs and two paths to each).

Using his script as an example, and keeping a good bit of the formatting code, I have modified his script to use views of the host objects and cull the information from there. This makes it significantly faster…what took 28 minutes before now takes about 30 seconds.

Read morePowerCLI: Show HBA Path Status

PowerCLI: Why you need this book!

PowerCLI Ref 1st Ed

The VMware vSphere PowerCLI Reference will officially go on sale April 12th, and if your a vSphere administrator this one is a must have. We gathered the collective automation experience of four vEXPERTS and a MVP, and then filtered it through a fifth vEXPERT. The end result was a collection of polished solutions that are not only technically proficient, but more importantly will work the first time every time! Having all built and maintained large infrastructures we combined our collective knowledge and wrote a complete reference, we cover the entire life cycle from creating your environment to monitoring it long term. I mean it, we left no stone unturned where VMware had no PowerCLI solution we wrote our own. This book includes a PowerShell Module with 79 custom advanced functions! We also considered how you would use our book, and chose to take a task driven approach, this enabled you to just flip to the answer you need. You don’t have to “read” all 780 pages… instead think of this book as a fire extinguisher for your virtual infrastructure.

So that’s great, but who needs a book? With blogs and the community forums you can just find the answer right? Yes, yes you can eventually, but if you do that your shopping at a fishery. In this book we WILL teach you to fish! As a commitment to that end we created a dedicated web site to support our work. If you find a bug, or if we missed something you can post a question in our forum!

Admittedly this is my first published work, but I couldn’t be prouder of it. In my opinion The ONYX, and vSphere SDK chapters alone would be worth the cover price. I look forward to the release, and to your honest opinions of our work!

~Glenn

PowerCLI: Force NetApp Virtual Storage Console (VSC) to use a FQDN

First let me say, I love VCS, it took all of the complexity out of using NetApp storage in a vSphere environment.  I have been tolerating one annoyance for quite some time now, and this morning said annoyance broke VCS at a customer site. What’s wrong with VCS? Well, for some reason it forces you to register the plugin with vCenter using an IP address.  Due to an over-restrictive proxy configuration, which caused only fully qualified domain names(FQDN) worked. Any IP address was redirected to an web page that explained said over-restricted policy, because VCS is mainly a web page the use of an IP address broke everything.  I searched around a little, and found Williams Lams post on removing plug-ins with the MOB. Once I found the pivot point for Plug-ins, I searched the API Reference, and found the ExtensionManager object.   Now that I had the Object in hand, I fired up PowerCLI and in less than 10 min figured out how to manually adjust the URL VSC used. It was so easy that I think I’m going to try and slap together a quick module to manage plug-ins via PowerCLI, but in the meantime if you, like me, have been frustrated by VSCs use of an IP address… try this.

PowerCLI: Reconnect VMhosts after changing vCenter certificates

If you have ever changed the vCenter server certificates, you’ve experienced having all your hosts disconnected from vCenter.  I couldn’t imagine reconnecting them one at a time… You could do this all natively in PowerCLI, but that would require you to fully remove and then add the hosts.  That is very inconvenient, and almost as much trouble as doing it by hand… In this case it is both faster and easier to just use the native vSphere API.

~Glenn

PowerCLI: Configure iSCSI one-liner

While migrating a small environment to vSphere today I ran into my nemesis Host Profiles again. When are they going to Fix these things? The fact that they are incapable of even rudimentary iSCSI configuration is embarrassing. I’m sure vmware will fix it, but until then I wrote a simple one-liner that will configure iSCSI on a new host.

~Glenn

PowerCLI: Apply-VMHostProfile passing parameters via $AdditionalConfiguration

I’ve ran across this particular issue myself, and submitted a bug to the PowerCLI team, but shortly after Andrew posted his ESXi 4.0 autoinstall Tim asked about this very issue.  There is a documentation error in Example #5 from the Apply-VMHostProfile cmdlet help.  Which contains the following code example.

Sadly if you tried to execute the above you would get the following error.

At first this may appear a little cryptic, but it get’s a lot clearer once we inspect the object types in use.

The example from the help docs was apparently expecting a Hashtable to be returned from apply-VMhostProfile.  Instead we found an array of DictionaryEntry objects… hence the error.

There are two possible work around’s we can employ until the PowerCLI team ships a fix.  The first one is complicated, but dynamic.

<pre class=’PowerShellColorizedScript’><span style=’color:#ff4500′>$profile</span> <span style=’color:#a9a9a9′>=</span> <span style=’color:#0000ff’>Get-VMHostProfile</span> <span style=’color:#000080′>-Name</span> <span style=’color:#8a2be2′>testProfile</span>

<span style=’color:#ff4500′>$additionalConfiguration</span> <span style=’color:#a9a9a9′>=</span> <span style=’color:#0000ff’>Apply-VMHostProfile</span> <span style=’color:#000080′>-ApplyOnly</span> <span style=’color:#000080′>-Profile</span> <span style=’color:#ff4500′>$profile</span> <span style=’color:#000080′>-Entity</span> <span style=’color:#8a2be2′>10.23.114.166</span>
<span style=’color:#000000′>(</span><span style=’color:#ff4500′>$additionalConfiguration</span> <span style=’color:#a9a9a9′>|</span> <span style=’color:#0000ff’>Where-Object</span> <span style=’color:#000000′>{</span><span style=’color:#ff4500′>$_</span><span style=’color:#a9a9a9′>.</span><span style=’color:#000000′>Name</span> <span style=’color:#a9a9a9′>-eq</span> <span style=’color:#8b0000′>’network.hostPortGroup[“key-vim-profile-host-HostPortgroupProfile-VMkernel”].ipConfig.IpAddressPolicy.address'</span><span style=’color:#000000′>}</span><span style=’color:#000000′>)</span><span style=’color:#a9a9a9′>.</span><span style=’color:#000000′>Value</span> <span style=’color:#a9a9a9′>=</span> <span style=’color:#8b0000′>’10.0.0.128′</span>
<span style=’color:#000000′>(</span><span style=’color:#ff4500′>$additionalConfiguration</span> <span style=’color:#a9a9a9′>|</span> <span style=’color:#0000ff’>Where-Object</span> <span style=’color:#000000′>{</span><span style=’color:#ff4500′>$_</span><span style=’color:#a9a9a9′>.</span><span style=’color:#000000′>Name</span> <span style=’color:#a9a9a9′>-eq</span> <span style=’color:#8b0000′>’network.hostPortGroup[“key-vim-profile-host-HostPortgroupProfile-VMkernel”].ipConfig.IpAddressPolicy.subnetmask'</span><span style=’color:#000000′>}</span><span style=’color:#000000′>)</span><span style=’color:#a9a9a9′>.</span><span style=’color:#000000′>Value</span> <span style=’color:#a9a9a9′>=</span> <span style=’color:#8b0000′>’255.255.255.0′</span>

<span style=’color:#0000ff’>Apply-VMHostProfile</span> <span style=’color:#000080′>-ApplyOnly</span> <span style=’color:#000080′>-Profile</span> <span style=’color:#ff4500′>$profile</span> <span style=’color:#000080′>-Entity</span> <span style=’color:#8a2be2′>10.23.114.166</span> <span style=’color:#000080′>-Variable</span> <span style=’color:#ff4500′>$additionalConfiguration</span></pre>

I actually don’t like this approach even though it’s a modified version of the included example.  I prefer just a simple static Hashtable.

All in all, the HostProfile cmdlets are surprisingly complete, and I think the majority of the “issues” I’ve ran across are a result of the SDK itself.  The Host Profiles sections of the API just don’t have the same fit and finish I’ve come to expect in a VMware API.

I’m sure carter and team will have this fixed in the next release, untill then…

~Glenn

PowerCLI: Remove SMVI snapshots

I wrote this script about a year ago to deal with errant SMVI snapshots, and was drafting this blog post when my rss feed caught me off guard. It appears Matt Robinson has beat me to the punch line.   He has produced a Perl script that cleans up any leftover snapshots, but if you favor a PowerShell approach… I give you Remove-SMVISnapshots.

Read morePowerCLI: Remove SMVI snapshots

PowerCLI: Find vCenter without vCenter

If you don’t know already PowerCLI now has two modes single and multiple.  It stands for exactly what you think it does.  In single mode when you execute a command PowerCLI runs that command against the server you’re connected too.  Multiple mode allows you to specify multiple vCenter/ESX/vSphere host, and when you execute a command it runs that command against every server you’ve specified! This had to be one painful feature to get right, but the PowerCLI team nailed it.

I’ll admit when I first played with it I thought I would never need/use multiple mode.  That is until our vCenter server was inadvertently shutdown instead of rebooted.  Normally this would lead to one of two out comes.
A.) forcefully register vCenter on the first host I hit and power it up.
B.) A twenty minute search for the host that has vCenter.

Well today I didn’t feel like doing either… On a whim I tried this new-fangled multiple connection thing… IT WORKED!

I’ve since wrapped all this up in a batch file and added it to our playbook for a lights out recovery of virtual center!

~Glenn

PowerCLI: Speed boost, Find VM Snapshots by Name.

I use NetApp SnapManager for Virtual Infrastructure(SMVI) to back up 95% of my environment. SMVI isn’t perfect, and it’s kinda a pain, but it gives me the ability to back up 100-250 vm’s in less than 10min! Well the actual NetApp snapshot takes seconds. The rest of the time is spent waiting on ESX snapshots. The only real downside here is from time to time SMVI will fail to delete the ESX snapshots. I have been using PowerCLI to find and delete these snapshot, but the cmdlets are just too slow, for what I’m trying to do. I’ll post my Finalized SMVI cleanup script later. Until then, I give you finding snapshots really fast!

Read morePowerCLI: Speed boost, Find VM Snapshots by Name.