LINUX Virtualization on Virtual Iron VFe

After years of research, the goal of seamlessly migrating applications from shared memory multi-processors to a cluster-based computing environment continues to be a challenge. The main barrier to adoption of cluster-based computing has been the need to make applications cluster-aware. In trying to solve this problem two approaches have emerged. One consists of the use of middleware tools such as MPI, Globus and others. These are used to rework applications to run on a cluster. Another approach is to form a pseudo single system image environment by clustering multiple operating system kernels[Pfister-98]. Examples of these are Locus, Tandem NonStop Kernel, OpenSSI, and openMosix.

However, both approaches fall far short of their mark. Middleware level clustering tools require applications to be reworked to run a cluster. Due to this, only a handful of highly specialized applications sometimes referred to as embarrassingly parallel - have been made cluster-aware. Of the very few commercial cluster-aware applications, the best known is Oracle Database Real Application Clustering. OS kernel clustering approaches present other difficulties. These arise from the sheer complexity of supporting a consistent, single system image to be seen on every system call made by every program running on the system: applications, tools, etc; to making existing applications that use SystemV shared-memory constructs to run transparently, without any modifications, on this pseudo single system.

In 2003, Virtual Iron Software began to investigate the potential of applying virtual machine monitors (VMM) to overcome difficulties in programming and using tightly-coupled clusters of servers. The VMM, pioneered by IBM in the 1960s, is a software-abstraction layer that partitions hardware into one or more virtual machines[Goldberg-74], and shares the underlying physical resource among multiple applications and operating systems.

The result of our efforts is Virtual Iron VFe, a purpose-built clustered virtual machine monitor technology, which makes it possible to transparently run any application, without modification, on a tightly-coupled cluster of computers. The Virtual Iron VFe software elegantly abstracts the underlying cluster of computers with a set of Clustered Virtual Machine Monitors (CVMM). Like other virtual machine monitors, the CVMM layer takes complete control of the underlying hardware and creates virtual machines, which behave like independent physical machines running their own operating systems in isolation. In contrast to other virtual machine monitors, the VFe software transparently creates a shared memory multi-processor out of a collection of tightly-coupled servers.

Within this system, each operating system has the illusion of running on a single multi-processor machine with N CPUs on top of M physical servers interconnected by high throughput, low latency networks.

Using a cluster of VMMs as the abstraction layer greatly simplifies the utilization and programmability of distributed resources. We found that the VFe software can run any application without modification. Moreover, the software supports demanding workloads that require dynamic scaling, accomplishing this in a manner that is completely transparent to OSs and their applications.

In this paper we'll describe Linux virtualization on Virtual Iron VFe, the virtualization capabilities of the Virtual Iron Clustered VMM technology, as well as the changes made to the LINUX kernel to take advantage of this new virtualization technology.


Download PDF.