xVM, LDOMs, Zones: Sun’s slightly confusing SPARC virtualization offerings

One of my long term tasks has been to figure out how to effectively virtualize our SPARC infrastructure. Turns out it isn’t as easy as I originally thought it would be, mostly because of Solaris 8 and the fact that I can’t get rid of it :). Don’t ask me why (cause it irritates me to no end…) but I can not convince the stodgy Solaris 8 admins that their binaries will run in Solaris 10 without modification.

Anyway, my initial thoughts were to use Solaris 10 zones (commonly called containers as well) to hold the miscellaneous Solaris 8 and 10 servers, however this is really a consolidation solution. It is possible to move a zone from server to server, however this requires down time for the zone. Solaris 8 branded zones are interesting and look like they may help with the inability to update, however since I haven’t been able to actively test with them, I’m not sure whether Solaris 8 will run in a zone on a box that uses the T1, T2 or T2+ processors.

The T (or “Niagara”) series of processors uses the sun4v (or is it sun4u?) architecture, which only Solaris 10 is capable of using, it’s my understanding that Sun has no plans to write libraries for Sol 8 and 9 to utilize this technology. Consequentially, Solaris 8 and 9 can not run directly on the T series of processors.

The second option that entered my mind was to use Logical DOMains (LDOMs), which is a rather interesting virtualization technology from Sun that works only on their CoolThreads technology (a core part of the T series of processors). LDOMs have a hypervisor similar to to VMware’s vKernel, however theirs is a firmware that is installed via the package manager. The hypervisor is managed by the primary domain, kind of like the COS from VMware. LDOMs are the only virtualization technology aside from VMware (that I’m aware of, I could be wrong) that allows live migration of a guest, which is very interesting. LDOMs also allow the dynamic provisioning of hardware to a guest, so the amount of RAM and other properties can be changed on the fly.

LDOMs don’t use the same method of sharing resources that ESX and VMware do…ESX uses all cores and spreads the processing across them. As a VM needs CPU resources, so long as it doesn’t exceed any constraints placed upon it, it allocates them on the first available core. LDOMs, in contrast, require the administrator to assign “threads” in the cores to each domain. A T1 processor, for example, has eight cores, each capable of four threads of processing. This means that there are 32 threads available at any point in time, so the maximum number of domains capable of being on the server at any one time is 31…the hypervisor manager (a Solaris 10 “primary” domain) requires at least one, and preferably four or more, threads for effective management of resources.

LDOMs are interesting, and promising, but because they require the CoolThreads technology, which is exclusive to the T series of processors, Solaris 8 can not run directly in a domain. This means that Solaris 10 would have to be installed first, and the Solaris 8 services would have to be hosted in a branded zone. This adds an additional OS to manage, another layer of complexity, and still doesn’t avoid the problem that even a Solaris 8 branded zone might not work correctly on the T series of processors.

The third possibility isn’t even a possibility at this point. LDOMs, currently, are fairly user unfriendly. If you’re a *nix admin, they seem quite normal, and even more so if you’re a Solaris admin…everything is done from the command line. This is somewhat familiar for us Solaris admins, as I do about 99% of my work from the command line (zpool/zfs commnds anyone?). However, most people like GUIs. The solution to this is Sun’s xVM. xVM, when used on the x86 architecture, utilizes the Xen code base for it’s virtualization moxie. When you move it over to SPARC it uses LDOMs to achieve the virtualization mojo. The non-possibility part of xVM for SPAR: it isn’t available yet. They do have an early access program for xVM, but as far as I know only the x86 portion is available through it.

All of this leads to some confusion about Sun’s virtualization offerings. Most people are unaware of LDOMs, and those that are aware of xVM don’t know of it’s planned support for SPARC. xVM appears to be rather expensive as well.

Sun adds to the general confusion about their virtualization support/offerings because, seemingly, the only virtualization they talk about is for x86. They are extremely proud of their Sun Fire X4450 and it’s vMark score just shy of 20, which implies strong support for VMware. As stated by Alessandro, Sun seems to have a bit of an identity crisis when it comes to virtualization. xVM (Xen and LDOM based sides), Hyper-V support, VMware support, Zones, LDOMs (as a standalone technology), and so on. Will one or more of these technologies fall out of Sun’s favor? Especially if/when xVM is a released technology. Assuming it’s released sometime this year, xVM has approximately the same OS support as Hyper-V, but supports live migration, which puts it a step ahead of Microsoft, but a step below VMware (who has significantly better OS support).

All this despite some people saying that SPARC should be abandoned by Sun.

1 thought on “xVM, LDOMs, Zones: Sun’s slightly confusing SPARC virtualization offerings”

Leave a Reply