PowerShell: DataOnTAP and SID Convertions

This morning while standing up a new vScan A/V server I wanted to look up our McAfee service account.  I knew the account would be a domain account, and I knew it would be a member of the backup operators group on the filer.  With that in mind I ran the following.

Well that’s rather useless… Unfortunately, the OnTAP API doesn’t provide a means to convert a SID to a NTAccount.  This is normally accomplished via the “cifs lookup” command on the Ontap CLI, but that doesn’t help us much from the toolkit.  Fortunately .Net provides a native means to perform this conversion.  This isn’t new to anyone who has been following PowerShell for a while (//o// first posted these function way back in the Monad days), but that doesn’t make them any less useful!

Now that’s more like it!  This is what I Love about powershell.  In the past I would have had to push back on my sales rep, who would have inturn pushed back on the development team.  fast forward a year, and maybe I would have a workaround.  Or I would have had to try and glue a couple third party exe together (yuck). With PowerShell if I don’t like something I simply extend it in script.  No development, nothing complicated, just a couple line of PowerShell.  Best of all I can then provide this to the vendor as a concreate example of what I want in the next release (hint hint NetApp cifs lookup needs to be in the SDK!)

It really is just great stuff.
~Glenn

vSphere: Console… we don’t need no stink’in console

I won’t attempt to provide a feature rundown or tell you why vSphere 4.1 is the greatest thing since sliced bread.  It appears to be a solid release, but  I’ll leave that analysis to the experts…Instead I want to talk about the vSphere hypervisor (previously ESXi).

Why the name change? Simple what was previously mis-branded as a separate technology is really the hypervisors core.  Previously in ESX3.5, ESXi was a separate technology, but as of vSphere 4 they have had a unified core.   In-fact the product we like to think about as vSphere 4.0/4.1 is really just a vSphere hypervisor with a special management VM!  This is important, the only difference is the console which is nothing more than a VM!

So why the distinction, Why now?  VMware is playing it’s hand this round because that special VM is going bye, bye.  The Next release of vSphere will not have a service console.. PAINIC…. RUN IN CIRCLES THE ZOMBIES ARE COMMING!!!

Don’t Panic, Personally I applaud the move.  Over the past year and a half I’ve heard every argument against the console less hypervisor, but honestly I chalk it all up to people fear change.  There are a couple thousand admins who have invested a lot of time mastering vSphere, and VMware is about to change the whole game on them.  These guys/gals bring up several arguments against the console less hypervisor, I’ll attempt to offer my counter argument to these points.

Q. No 3rd Party agents.

A. It has been public knowledge that the console was going away, and as of vSphere 4.0 VMware shipped a new management appliance vMA.  One of the intended uses of this appliance was to install 3rd party agents.  So you see we do still have 3rd Party agents they just need to be rewritten.  In most cases this will result in a better product. Unfortunately, the vast majority of 3rd party software, could better be described as a really complex perl script running over ssh!

Q. Hardware monitors/plug-in

A. Part of the original ESXi 3.5 release was the introduction of a rudimentary CIM provider.  This provider has been fully expanded , and made extensible.  While it is a change from the traditional agent based monitors CIM does fill in this gap.

Q. Automating common tasks.

A. As of vSphere 4.1 Tech Support Mode supports SSH, but you should really be using either PowerCLI or the vCLI!  While it is true that are still a couple of things that can only be done via the console.  I’m confident VMware will fix those gaps before putting the console out to pasture.

Q. Security

A. So this is the big one, and my personal pet peeve.  I’ve heard security experts bash the vSphere hypervisor claiming it was insecure.  I just don’t understand this stance, admittedly I’m no security expert.  I only work with the federal government in some of the most secure data centers in the world, but what do I know…

Let’s break this down shall we… The only difference is a VM.   Admittedly this VM has special connections into the vmkernal, but it’s still just a VM.  How exactly does the inclusion of a VM make the hypervisor more secure?  In my opinion the exclusion of this VM instantly increased the security posture of most organizations.  The reason for this simple, it was hard to properly harden the console.  Alternatively it was all too easy to open a critical security hole, and expose ones infrastructure with the console.

Yes you still have to do several things to really lock down the console less hypervisor, but it’s not nearly the feat the console once was.  In fact it’s simple;

1. Modify the Proxy.xml (turning off all unneeded web services, and make everything use https).
2. Enable Lockdown mode.
3. Physical security.

That’s it folks, that’s all it takes to secure the hypervisor.  There are a couple hundred other little things necessary to design a secure infrastructure, but as you can see the hypervisor is easy!  In fact I’m so confident in this I’m willing to hold a Bobby Flay style throw down.  If you have the means to provide a  pair of internet facing vSphere hosts. I’ll secure the console less hypervisor, we’ll get TexiWill to harden the legacy console based hypervisor, and then we’ll release the IP’s to the world.  Have at it, folks I bet the console less hypervisor holds up at least as long as the legacy hypervisor!

Why so brash? Well it will take an exploit to get in to the console less hypervisor, and any exploit will also be present in the legacy hypervisor.  The console less vSphere hypervisor without access to the physical host or vCenter there is simply no other way in.   Remember this isn’t Linux or BSD or UNIX… it’s vSphere it’s practicality firmware, and the whole point was to remove all that crap that weaken the security , and stability to begin with!

I really want to put this to bed!  Let’s develop the to do list for VMware.  The 10-20 things they need to fix before they can finally kill the console.  Then let’s collectively shut up about it.  It’s going to happen, and complaining with arbitrary little gripes… or demanding NDA meetings with engineers isn’t going to stop any of it.  The Task at hand is simple, weed out the crap, and focus on what needs to be fixed in vSphere v.Next.

If we missed something let us know in the comments.
~Glenn

PowerShell: DataOnTap Realtime Multiprotocol Volume Latency

I had some free time yesterday morning as I couldn’t sleep after the long weekend. I used the time to dig into into the DataOnTap PowerShell Toolkit.  I started with an easy port of one of Andrews performance monitoring scripts.  I won’t go into as it’s very straight forward, but I will say so far I am very pleased with the DataOnTAP toolkit. 

~Glenn

PowerCLI: Reconnect VMhosts after changing vCenter certificates

If you have ever changed the vCenter server certificates, you’ve experienced having all your hosts disconnected from vCenter.  I couldn’t imagine reconnecting them one at a time… You could do this all natively in PowerCLI, but that would require you to fully remove and then add the hosts.  That is very inconvenient, and almost as much trouble as doing it by hand… In this case it is both faster and easier to just use the native vSphere API.

~Glenn

PowerCLI: Configure iSCSI one-liner

While migrating a small environment to vSphere today I ran into my nemesis Host Profiles again. When are they going to Fix these things? The fact that they are incapable of even rudimentary iSCSI configuration is embarrassing. I’m sure vmware will fix it, but until then I wrote a simple one-liner that will configure iSCSI on a new host.

~Glenn

Color me astonished!!

I’ve been out of touch for most of this week, having only been able to be connected for an extended period of time today, and from somewhere out in left field I received an extremely surprising email from Mr. John Troyer…

I have been named a 2010 vExpert! Words can not describe how honored I am to receive this designation, I feel truly humbled by the others that have received the award and I can only hope that when I grow up I can be like them.

Thank you again to John Troyer and his team for this privilege!

Andrew

Winner, winner, chicken dinner

I WON!  If you haven’t heard, I won the big prize for the Scripting Games this year.  First place landed me a ticket to TechEd North America! I had a hell of a time getting the time off work, and getting my travel authorized, but it all worked out… Below is my agenda for the conference, feel free to contact me via twitter… with the short lead time I have Nothing scheduled (Feels kind of weird to just be going to a show…)

I should be near anything that says PowerShell in between sessions.

Monday, June 7

9:00 AM – 10:30 AM
KEY01 Tech·Ed North America Keynote Presentation

1:00 PM – 2:15 PM
WSV07-INT New Remote Management Technologies in Windows Server 2008 R2

2:45 PM – 4:00 PM
WSV334 Windows Server 2008 R2: Tips on Automating and Managing the Breadth of Your IT Environment

4:30 PM- 5:45 PM
MGT308 Microsoft System Center Configuration Manager v.Next: Overview

Tuesday, June 8

8:00 AM – 9:15 AM
WCL304 Best Practices Guide to Managing Applications

9:45 AM – 11:15 AM
KEY02 Business Intelligence Conference Keynote Presentation

1:30 PM – 2:45 PM
WCL313 Paradigm Shift: Microsoft Visual Basic Scripting Edition to Windows PowerShell

3:15 PM- 4:30 PM
MGT309 Microsoft System Center Configuration Manager v.Next: Software Distribution

5:00 PM – 6:15 PM
DAT203 Managing Microsoft SQL Server: For the "Reluctant" DBA

6:15 PM
WCL318 Using Windows Preinstallation Environment (PE) 3.0 to Troubleshoot and Fix Problems, and to Capture and Deploy WIM Images

Wednesday, June 9

8:00 AM – 9:15 AM
SIA333-R Useful Hacker Techniques: Which Part of Hackers’ Knowledge Will Help You in Efficient IT Administration? (repeated from 6/8 at 3:15pm)

9:45 AM – 11:00 AM
WSV319 Manage Your Enterprise from a Single Seat: Windows PowerShell Remoting

11:45 AM – 1:00 PM
WSV301 Administrators’ Idol: Windows and Active Directory Best Practices

1:30 PM – 2:45 PM
WSV313 Failover Clustering Deployment Success

3:15 PM – 4:30 PM
DAT407 Windows Server 2008 R2 and Microsoft SQL Server 2008: Failover Clustering Implementations

5:00 PM – 6:15 PM
MGT306 Microsoft System Center Configuration Manager v.Next: Hierarchy Design

Thursday, June 10

8:00 AM – 9:15 AM
SIA334 The Secrets of Effective Technical Talks: How to Explain Tech without Tucking Them In!

9:45 AM -11:00 AM
WCL06-INT Using Windows PowerShell for Enterprise Desktop Automation

1:30 PM – 2:45 PM
WCL314 Windows Sysinternals Primer: Process Explorer, Process Monitor, and More

3:15 PM – 4:30 PM
WSV401 Advanced Automation Using Windows PowerShell 2.0

5:00 PM – 6:15 PM
WCL315 The Case of the Unexplained, 2010: Troubleshooting with Mark Russinovich

6:30 PM – 11:00 PM
TechEd Party

See you all there! I’m the chubby white guy with a beard.

~Glenn

Scripting Games 2010: My Scripts


2010 Scripting Games--I was there!

This is more for my own records.  I know for sure I will be using these as a reference over the next few month.  All my 2010 Scripting Games solutions.

~Glenn

Scripting Games 2010: PostGame Roundup

2010 Scripting Games
I just submitted my last entry into this years games, and I wanted to capture a few things before I go and get busy. 

The Challenges

In years past the challenges where the equivalent of the scripting combine.  They would test ones ability to solve a logic problem in code. This created a developer friendly zone.  Whereby professional coders (aka developers) would inevitably produce a script that used some –xor feature or .net class I knew nothing about.  I would look back on my script feel incredibly inadequate and try harder.  This always lasted at most 5 days… until this year I never made it past 5 days.

This year the dread pirate Ed Wilson flipped the script. All of a sudden the event resembled a ticket at work.  As I read them I instantly knew how to do x or y, and would jump straight to the extra credit section.  This my friends is where it got interesting.  Every event this year could be solved by a PowerShell newbie, but to hit all the design criteria that would take some skill.

It was this combination that kept me in the games. Looking back it felt more like leveling in a MMO than work.  It wasn’t enough to solve the problem I wanted my stars.

PoshCode

There were a slew of issues with Poshcode V2 during the first couple days of the games.  They were fixed promptly, and the site has preformed admirably ever since.  This is the second year I’ve used PoshCode V2, and would just like to say…

JOEL… SHIP IT ALREADY…. ITS GOOD ENOUGH!!!!!1111

Seriously though, it is a massive upgrade, I’m excited to see what the poshcode.org upgrade may bring!

The Ratings

Judging criteria was put out here.  The short version is as follows.  If the script looks like it works you get two stars.  Every star beyond that is based on meeting the extra design criteria.  Now that is an incredibly open guideline…. and I am completely okay with it.

The same judges judge all the scripts so one liberal judge will give us all 5 stars.  The next judge might not like our approach and only give it three, but that stingy judge will be stingy to all the entries.  (I have some great scripts that got some whacked rankings.  Tell me how this only get’s three stars?)

That being said I would like to propose one change for next year.  We need feedback, why only four stars… did I miss something? Do you not like the formatting?  Did I go off the deep end for no reason and overcomplicated it?  All of these would be valid reasons to deduct points, but the deduction looses it’s real purpose if we can’t pass on the lesson it contains.

I’ll give you an example.  one of my close competitors this year kept creating the help by hand.  After day two or three I left a comment that said

hey great script, but you’re use of a here-string already locks you to V2.  Might as well go whole hog… check out help about_Comment_Based_Help

His very next script included comment based help.. heck he even gave me a shout out for the tip!  Awesome, this is what these games are about!

Except, that script didn’t rate too well… as of event 5 he was back to his old VBScript ways.  Somehow he associated that change with the bad rating… maybe it was part of it, he’s had all 5’s since.  Either way for on brief moment he was on track…

PowerShell Best practices

Which leads me to my final peeve about these games, and something we should fix for next year.  A script that is CLEARLY VBScript written in PowerShell should not get higher than a 3.  This isn’t about prizes or winning either.  These games are a learning event.  If we aren’t teaching best practices along the way what good are we doing anyone? You know simple stuff like don’t use the Scripting.FileSystemObject com object anymore…  I know this is a slippery slope, and I was truly blown away by the overall quality of the submissions. Nevertheless I think it’s a conversation worth having.

Motivation for the Games

As of this post I’m in first place… Looks like I might win this dang thing(what ever that means)!  Still way to early to call. Either way I didn’t participate this year for any monetary prize… I did it for the vanity of it all. I wanted to know where I stacked up.  I never thought I’d win anything… first place came out of left field.  All in all I got exactly what I wanted out of the games.  I do wish the judges had participated, they need only recuse themselves from the event they judge.  I would have liked to compete against a couple MVP’s.

New Tricks

These games forced me to step my game up.  I used EVERY trick I know, and I learned a few knew ones along the way.  Three specific tricks I will not soon forget.

Splatting:

I finally know what spatting is and why it’s a big deal.  Splatting helped me streamline half of my scripts removing 20-30 logic tree’s, by simply creating the parameter set’s upfront. I know we’ve all heard the definition before, but let me show you in code why it matters!

Before Spatting

After Splatting

As you can see splatting allows you to offload all the logic that isn’t related to the task at hand to the the script/function initialization.  This cleans up your working area, and results in cleaner code!

Windows Forms:

Last week I created my very first windows form GUI ever, and I did it by hand.  Not because I’m hardcore, but because once I got started it was REALLY easy.  Like .net itself windows forms have full reflection, meaning it’s all self descriptive from within PowerShell.

Obviously using a tool like PrimalForms will save a ton of time, but don’t feel like you need a third party tool.  This stuff is easier than you’d think!

WPF/WPK:

This was the gut shot that hurt my brain!  WPF was a lot to bit off in one chunk.  After two weeks with it I feel like I still barel y know what I’m doing, but once I do… WPK will make it easy.  WPK’s strength, and weakness come from it’s meta-programming roots.

If you don’t already know, James wrote a script that wrote WPK.  It’s all machine generated code.

The good part, it is 100% complete it can do anything possible in WPF.

The bad part, there is NO abstraction layer between you and WPF.

There are some really cool helper functions that make it easy to use PowerShell code directly.  But if you want to make a listbox you’re going have to look at a c# example. I originally planned on doing event 10 with boots, and I wish I had time… I want to compare the two head to head, but I ran short and wanted to get my final submission in.

Final Thoughts

100% worth the time, These few weeks have sharpened my skill significantly.  To the point where I’ve actually caught up at work.  At this pace I’ll be ahead of schedule next week.  I think it was just the hard break the games carried… forced me out of my rut.

Take a look at my work let me know what you think. I have very thick skin and prefer the truth over candy coated fairy tales.

If you spent this year on the side line… save the excuses.  These games, like learning PowerShell will advance you professionally; ergo they’re good for work; ergo quit you’re bitchin and get to it!  Unless you were at EMC world ;P

In closing I want to personally thank the sponsors and judges who made the games possible this year:

Thank you, Tech·Ed 2010 North America
Thank you, PoshCode.org
Thank you, Microsoft TechNet
Thank you, SAPIEN Technologies
Thank you, Quest Software
Thank you, /n software
Thank you, Software FX
Thank you, 101FreeTechBooks.com
Thank you, ShellTools
Thank you, Idera
Thank you Judges

I had a blast a learned a ton, Good job all, See ya next year!

~Glenn

@extraparams

PowerCLI: Apply-VMHostProfile passing parameters via $AdditionalConfiguration

I’ve ran across this particular issue myself, and submitted a bug to the PowerCLI team, but shortly after Andrew posted his ESXi 4.0 autoinstall Tim asked about this very issue.  There is a documentation error in Example #5 from the Apply-VMHostProfile cmdlet help.  Which contains the following code example.

Sadly if you tried to execute the above you would get the following error.

At first this may appear a little cryptic, but it get’s a lot clearer once we inspect the object types in use.

The example from the help docs was apparently expecting a Hashtable to be returned from apply-VMhostProfile.  Instead we found an array of DictionaryEntry objects… hence the error.

There are two possible work around’s we can employ until the PowerCLI team ships a fix.  The first one is complicated, but dynamic.

<pre class=’PowerShellColorizedScript’><span style=’color:#ff4500′>$profile</span> <span style=’color:#a9a9a9′>=</span> <span style=’color:#0000ff’>Get-VMHostProfile</span> <span style=’color:#000080′>-Name</span> <span style=’color:#8a2be2′>testProfile</span>

<span style=’color:#ff4500′>$additionalConfiguration</span> <span style=’color:#a9a9a9′>=</span> <span style=’color:#0000ff’>Apply-VMHostProfile</span> <span style=’color:#000080′>-ApplyOnly</span> <span style=’color:#000080′>-Profile</span> <span style=’color:#ff4500′>$profile</span> <span style=’color:#000080′>-Entity</span> <span style=’color:#8a2be2′>10.23.114.166</span>
<span style=’color:#000000′>(</span><span style=’color:#ff4500′>$additionalConfiguration</span> <span style=’color:#a9a9a9′>|</span> <span style=’color:#0000ff’>Where-Object</span> <span style=’color:#000000′>{</span><span style=’color:#ff4500′>$_</span><span style=’color:#a9a9a9′>.</span><span style=’color:#000000′>Name</span> <span style=’color:#a9a9a9′>-eq</span> <span style=’color:#8b0000′>’network.hostPortGroup[“key-vim-profile-host-HostPortgroupProfile-VMkernel”].ipConfig.IpAddressPolicy.address'</span><span style=’color:#000000′>}</span><span style=’color:#000000′>)</span><span style=’color:#a9a9a9′>.</span><span style=’color:#000000′>Value</span> <span style=’color:#a9a9a9′>=</span> <span style=’color:#8b0000′>’10.0.0.128′</span>
<span style=’color:#000000′>(</span><span style=’color:#ff4500′>$additionalConfiguration</span> <span style=’color:#a9a9a9′>|</span> <span style=’color:#0000ff’>Where-Object</span> <span style=’color:#000000′>{</span><span style=’color:#ff4500′>$_</span><span style=’color:#a9a9a9′>.</span><span style=’color:#000000′>Name</span> <span style=’color:#a9a9a9′>-eq</span> <span style=’color:#8b0000′>’network.hostPortGroup[“key-vim-profile-host-HostPortgroupProfile-VMkernel”].ipConfig.IpAddressPolicy.subnetmask'</span><span style=’color:#000000′>}</span><span style=’color:#000000′>)</span><span style=’color:#a9a9a9′>.</span><span style=’color:#000000′>Value</span> <span style=’color:#a9a9a9′>=</span> <span style=’color:#8b0000′>’255.255.255.0′</span>

<span style=’color:#0000ff’>Apply-VMHostProfile</span> <span style=’color:#000080′>-ApplyOnly</span> <span style=’color:#000080′>-Profile</span> <span style=’color:#ff4500′>$profile</span> <span style=’color:#000080′>-Entity</span> <span style=’color:#8a2be2′>10.23.114.166</span> <span style=’color:#000080′>-Variable</span> <span style=’color:#ff4500′>$additionalConfiguration</span></pre>

I actually don’t like this approach even though it’s a modified version of the included example.  I prefer just a simple static Hashtable.

All in all, the HostProfile cmdlets are surprisingly complete, and I think the majority of the “issues” I’ve ran across are a result of the SDK itself.  The Host Profiles sections of the API just don’t have the same fit and finish I’ve come to expect in a VMware API.

I’m sure carter and team will have this fixed in the next release, untill then…

~Glenn