Authentication weirdness…

For some reason, some of my servers have been having trouble with AD authentication. The symptoms were: when logging into the console using an AD enabled account (i.e. an account that should be authenticated by AD), despite providing the correct password the system will return password incorrect. Upon providing the username again, the user is immediately granted access without having to provide a password.

I’m not sure about the provide-once-denied-provide-name-authorized behavior, but after some testing I discovered that the reason that it’s not letting the user in the first time is because the lines for pam_unix and pam_kerberos (pam_krb5 to be exact) in /etc/pam.d/system-auth are reversed. It should have pam_unix before pam_kerberos. I have no clue what is causing these lines to be reversed in the configuration file. I’m configuring all of the authentication and security measures using the esxcfg-... commands so it seems weird that starting with the blade servers this behavior has been exhibited.

Anyway, the fix files: one bash, one awk…

Read more

Powershell: Fix Orphaned Standby hosts after vCenter reboot!

vCenter’s Distributed Resource Scheduler (DRS) is by far one of my favorite features of VI2.5.  Distributed Power Management (DPM) is an experimental extension of DRS that will power down unneeded hosts, and wake them when needed again.  How cool it that!  The good news is, it works great.  The bad news is, in the current release there is a bit of a bug in DPM.

When vCenter is restarted it looses track of any hosts that it (DPM), or you (the VI admin) have put in a suspended state.  The suspended hosts show up in VC as “NotResponding”, and VC is no longer capable of waking them up.  The fix is to manually power them on… how aggravating.  I finally had some free time this morning so I threw together a quick PowerShell script to bridge the gap until VMware releases a production ready version of DPM.

Read more

ESX User Authentication and Password Management

Even if you use AD to authenticate your users, ESX will still check it’s local authentication mechanism for the user and their password. You can see this activity when you look in /var/log/messages after a login. For example, on my servers, when I login there are two messages…one from pam_unix saying that authentication failed and another from pam_kerberos (Active Directory) saying that authentication succeeded.

What this behaviour means is that you can have users on your ESX system that are not authenticated via Active Directory, thereby bypassing any password requirements (complexity, reuse, expiration, etc) settings you have there. Unless you adjust the default password settings in ESX, these users can have pretty much any extremely unsecure password they want, which is bad. Additionally, depending on who you work for (US Gov’t, finance, health care, etc) there may be some security compliance issues you must deal with, which usually include password requirements.

Even with Active Directory authentication, there is a local user on the ESX host for authorization. If that local user also has a local password, then the user can use either credential to login to the ESX server. Your box is only as secure as the weakest password.

Read more

Unadvertized VI Perl Toolkit helper functions and options

As I’ve been getting more and more familiar with the VI Perl Toolkit over the last couple of weeks I’ve discovered that the perl modules created and provided by VMware have some options and routines that are not discussed in the programming guide. This is my attempt to document the ones that I have found useful and you may be interested in.

There are some additional helper routines that can be found in the perl modules (the files that end in .pm) provided by the Toolkit, and the VMware employee(s) who have written them have done a pretty good job of documenting them in the code. Of special note (I may add them to this post at a later date) are the HostUtil and VMUtil modules, which have routines that streamline some of the more common operations (getting host and VM views, migrating VMs, etc).

Without futher ado…

Read more