Andrew Sullivan

ESX User Authentication and Password Management

by

Even if you use AD to authenticate your users, ESX will still check it’s local authentication mechanism for the user and their password. You can see this activity when you look in /var/log/messages after a login. For example, on my servers, when I login there are two messages…one from pam_unix saying that authentication failed and another from pam_kerberos (Active Directory) saying that authentication succeeded.

What this behaviour means is that you can have users on your ESX system that are not authenticated via Active Directory, thereby bypassing any password requirements (complexity, reuse, expiration, etc) settings you have there. Unless you adjust the default password settings in ESX, these users can have pretty much any extremely unsecure password they want, which is bad. Additionally, depending on who you work for (US Gov’t, finance, health care, etc) there may be some security compliance issues you must deal with, which usually include password requirements.

Even with Active Directory authentication, there is a local user on the ESX host for authorization. If that local user also has a local password, then the user can use either credential to login to the ESX server. Your box is only as secure as the weakest password.

Read more

Unadvertized VI Perl Toolkit helper functions and options

by

As I’ve been getting more and more familiar with the VI Perl Toolkit over the last couple of weeks I’ve discovered that the perl modules created and provided by VMware have some options and routines that are not discussed in the programming guide. This is my attempt to document the ones that I have found useful and you may be interested in.

There are some additional helper routines that can be found in the perl modules (the files that end in .pm) provided by the Toolkit, and the VMware employee(s) who have written them have done a pretty good job of documenting them in the code. Of special note (I may add them to this post at a later date) are the HostUtil and VMUtil modules, which have routines that streamline some of the more common operations (getting host and VM views, migrating VMs, etc).

Without futher ado…

Read more

vMotion configuration from the ESX host command line and remotely using the Perl Toolkit

by

One of the things that I do when configuring my hosts after kickstart is setup a kernel interface and enable vMotion for that port group. This isn’t too difficult, but takes a little bit of futzing with some vmware-vim-cmd results to get the data we need.

Since I’m on the subject, I figured I may as well do the same thing using the SDK, which eliminates one more thing that the rCLI can’t do in order for me to configure a new ESX host completely.

Read more

VMware Tools Update Policy, errrrr, Updating

by

After hearing about the bug with ESX 3.5 Update 3 where a vMotion would cause a tools upgrade, and consequentially a VM reboot, I wanted to check and verify that my VMs would not exhibit this behavior. Apparently the bug is present when the tools update policy is set to “Update at Power On”. (Yes, I admit this bug has been known for a while, I just kept forgetting to post this script.)

There are a multitude of other scripts (including the powershell cmdlet “update-tools”) out there to kickoff a tools update task, so I wasn’t interested in that, all I wanted was to know the update policy and be able to change it.

This perl is the result of that…

Read more

Kickstart your host into configuration conformity

by

The last few posts I’ve been mentioning how much of the configuration for my ESX hosts is automated. This post I’m going to talk a little more about how that automation is done, and provide an example kickstart script. I have been holding off on this post for a while now, as I have been planning on rebuilding my PXE server, at which point I will document each step and be able to provide a much more detailed post. However, things keep getting in the way and I haven’t had time to rebuild the server yet, so this is a slightly less detailed post, but should still be enough to get you on your way 🙂 And I have no doubt that you, dear reader, are not afraid of asking questions in the comments…

This automation is currently handled (I say “currently” because I’m working to move the majority of it to a remote host and use the SDK) by kickstart when the system is loaded. Well, to be totally honest, kickstart only plays a partial role in the process…during the %post section of kickstart I copy a series of scripts from an NFS mount point into the startup process (/etc/init.d/rc3.d), which are executed at first boot and, like good one-time-only scripts, remove themselves.

This setup allows me to pxe boot a host, give it the boot command which has the host ID appended, and that’s it. I can then walk away and wait for the host to add itself to vCenter, indicating that it’s finished. Kickstart and the post install scripts then configure the hostname, ip, virtual network configuration, security policy, ntp, base user set, install any custom RPMs, etc. This makes it extremely easy for me to keep all of our hosts at the same configuration level.

In order to keep all the hosts the same I simply have to update the relevant post install script when we decide to make a global change and it will configure the host correctly the next time it is loaded/reloaded. For updating hosts that can’t be reloaded (I try to reload the hosts periodically with the newest binaries from VMware…every 4-6 months…so that the software is not a huge conglomeration of patches…I know, it’s unnecessary, but it gives me peace of mind) we use a combination of Glenn’s POSH prowess and the perl toolkit scripts I’ve created to remediate hosts en masse to our baseline configuration.

Read more

Adjusting Console OS RAM via rCLI

by

In order to facilitate my ability to configure all aspects of an ESX host automatically, I wanted to adjust the amount of memory that is assigned to the COS without having to use VI Client. The perl below is the result of that effort.

As always, I am not responsible for any damage caused to your infrastructure, I recommend you put your host in maintenance mode and move all VMs off of it before attempting any significant action upon it (there should be little risk involved with this script though…). The change will not take effect until you reboot the host (which can be accomplished with the hostops.pl sample script provided as a part of the Perl Toolkit).

Read more

vSwitch security policies

by

Update 2009-2-22: Yet again, I updated the script, this time just to simplify and shorten the code using the “normal” method of updating values in objects retrieved from the SDK (rather than creating a whole new object and copying values, the script now updates the object retrieved, then uses that to update).

Update 2009-01-02: I have updated the script again, this time using the standard “vihost” so that you can connect to vCenter and change a host’s switches, as opposed to just connecting to the ESX host directly. I have also started using the _default_ parameter, which means that it is no longer necessary to specify the “–vswitch” option, but rather it is the last option on the line (just like all the other VMware provided scripts).

Update 2008-12-30: I have updated the script so that it defaults to turning all options off without having to specify them. This makes it easier to use (thanks to Glenn for this idea…).

I haven’t posted in a while, but it’s not because I haven’t been busy. The bulk of my recent work has been in automating the droll configuration items for an ESX server. With the exception of hardening the COS, pretty much everything can be setup/configured remotely via the SDK or rCLI. If you can change or set something via the Virtual Infrastructure Client, then you can set it via the SDK.

I am no POSH coder (just ask Glenn…), but I do know some perl, so using the VI Perl Toolkit, I’ve been able to script most of the configuration items that I need to do for an ESX server. This post is the first in what I hope will be a line that will hopefully contain scripts on configuring most aspects of an ESX host remotely.

I set all of our vSwitches to have Promiscuous Mode, Forged Transmits and MAC changes disabled, and so far there are no port groups that override this setting, thus giving me at least a little sense of security for certain aspects of my virtual networking.

Read more

ITIL, U-TIL, we all scream for…Configuration Management?

by

Ok, so the title is a little misleading. Configuration Management is a part of ITIL, however I’m not going to talk about ITIL, at least not directly.

As an administrator I’m responsible for multiple systems. Some of these are identical, e.g. Apache servers, MySQL servers, some of them provide unique, stand alone, services. However, they all have some things in common…sshd configuration, log rotation schedules (logrotated), etc.

It’s a PITA to keep up with all of these servers individually. A global change can take quite a bit of time, especially with our ever increasing number of ESX hosts. So, how do I make my job easier, myself more productive, and next year’s raise larger? Automated configuration management.

Read more

Punctuality is Important

by

Time keeping is especially important for Active Directory and Kerberos. I encountered an error when I was attempting to ssh into one of my AD enabled ESX hosts. The SSH error was “Permission Denied”, however after inspecting the logs (/var/log/messages) I discovered that pam_krb5 was throwing “Clock skew too great” errors.

This was odd to me, as I know every one of the ESX servers has NTP configured. Apparently ntpd died at some point, which caused the clock to begin losing time. Once the time difference between the domain controller and the ESX host exceeded 300 seconds (5 minutes), ESX no longer allowed me to login using AD credentials.

The fix was somewhat easy…reset the clock. Since I was able to login to the console, I did so as root, and executed ntpdate name.of.domain.controller, which forced it to sync the clock with the DC. After that was taken care of (which confirmed that it was ntp that broke), I went back to Virtual Infrastructure Client and reset the NTP settings for the host (it’s on the Configuration tab).