Transcription
Virtualization for Embedded SystemsIs an open source solution right for you?6/26/2013Julia KefferPage i
Virtualization for Embedded SystemsTable of ContentsTable of ContentsIntroduction . 1What is Virtualization? . 1Virtualization Applications . 2Operating Systems with Different Run-time Requirements . 2Isolate Security Conscious Applications . 2Open Source Compliance . 2Virtualization Architectures . 2Key Criteria for a Virtualization Solution . 4Hardware Support. 4Operating System Support . 4Resource Allocation and Sharing . 4Memory Isolation . 5Processor Scheduling . 5Guest Communication . 5Size of the Trusted Computing Base . 5Open Source Solutions . 6Xen . 7Hardware Support. 8Operating System Support . 8Resource Allocation and Sharing . 8Memory Isolation . 9Processor Scheduling . 9Page ii
Virtualization for Embedded SystemsTable of ContentsGuest Communication . 9Size of the Trusted Computing Base . 9Xtratum . 10Hardware Support. 10Operating System Support . 11Resource Allocation and Sharing . 11Memory Isolation . 11Processor Scheduling . 11Guest Communication . 11Size of the Trusted Computing Base . 11OKL4 . 12Hardware Support. 12Operating System Support . 12Resource Allocation and Sharing . 13Processor Scheduling . 13Guest Communication . 13Size of the Trusted Computing Base . 14Conclusions .15Works Cited .17Glossary .18Page iii
Virtualization for Embedded SystemsTable of FiguresTable of FiguresFigure 1: Non-Virtualized and Virtualized Computer . 1Figure 2: Paravirtualized System . 3Figure 3: Xen Hypervisor . 7Figure 4: PCI Pass-Through . 8Figure 5: Xtratum Architecture. 10Figure 6: OKL4 Architecture . 12Figure 7: OKL4 IPC Model . 13Page iv
Virtualization for Embedded SystemsIntroductionIntroductionEmbedded computers are part of our everyday lives, from smart phones, to cars, to gaming consoles.Virtualization was predominantly used first in the server market, but today it has come to the embeddedcomputer. Undoubtedly, you have used an embedded computer that employs virtualization technology.This paper explains what virtualization is, how different virtualization technologies work, and how virtualizationis applied in embedded applications. It examines a set of criteria for choosing a virtualization solution andevaluates three open source implementations against each of the criteria.What is Virtualization?The Computer Desktop Encyclopedia defines a virtual machine (VM) as “An operating system that runs like a"machine within a machine", and functions as if it controls the entire computer. The operating systems in eachVM partition are referred to as guest operating systems or partitions, and they communicate with the hardwarevia a program called a virtual machine monitor (VMM), which is also referred to as a hypervisor. The hypervisors"virtualizes" the hardware for each guest operating system (1).” Figure 1 shows the difference between nonvirtualized and virtualized systems.This paper uses the terms hypervisor and guest to refer to the VMM and the guest operating system,respectively.Figure 1: Non-Virtualized and Virtualized ComputerThe hypervisor operates in a privileged environment, also referred to as kernel mode or supervisor mode, whereit has access to low level system calls to mediate resource access.The entire guest operates in a non-privileged environment, also referred to as user mode. Guests communicatewith the hardware only through the hypervisor and cannot use low level system calls directly.The number of guests that can run on a single hardware platform is constrained by the available hardwareresources, typically the amount of memory. Each guest can run a different operating system.Page 1
Virtualization for Embedded SystemsIntroductionVirtualization ApplicationsVirtualization is useful either to consolidate multiple computer systems on the same hardware to reduce costsor to isolate programs running on the same hardware. This section describes three situations where it is usefulto run multiple isolated operating systems on the same hardware in embedded systems.Operating Systems with Different Run-time RequirementsVirtualization provides the ability to run different types of operating systems on the same hardware, such as afull featured OS for user interface functions, and a real-time OS for time critical applications. For example, in anautomobile, the computer that controls the anti-lock brake system has real-time requirements, while theinfotainment system does not. Without virtualization an automobile required two different computers; usingvirtualization both systems can run on the same hardware, reducing costs.Isolate Security Conscious ApplicationsVirtualization can isolate security-conscious applications from insecure applications. For example, if anapplication running on a smart phone introduces a computer virus, it is necessary to protect the wirelessprotocol stack to ensure that the system can still make phone calls. One way to do this is to run each of thesecomponents inside separate operating systems in a virtual environment.Open Source ComplianceOpen source licenses typically allow proprietary code to interact with open source code if the two communicateonly via a messaging interface. If the open source and proprietary code run in separate operating systems, thehypervisor fulfills this requirement.Virtualization ArchitecturesThere are different virtualization technologies; this paper focuses on type 1, or bare metal hypervisors, wherethe hypervisor software runs between the hardware and the guest. There are three categories of type 1hypervisors: full virtualization, paravirtualization, and a microkernel.With full virtualization, a whole system is emulated (basic input/output system (BIOS), disk, processor, networkinterface) and a guest runs unmodified on a hypervisor that provides the abstraction of the underlying computersystem. The guest is not aware of the hypervisor. The hypervisor intercepts hardware access instructions fromthe guests and invokes the instructions on behalf of the guest. Full virtualization requires hardware extensionsin the computer processor, such as Intel’s VT-x technology.Page 2
Virtualization for Embedded SystemsIntroductionFigure 2 shows a paravirtualized system in which the guest requires modifications to work in a virtual machine(2) and communicate with the hypervisor. Specifically, some or all of the device drivers in the guest are modifiedto replace the privileged instructions with direct requests to the hypervisor, which are referred to as hypercalls.Figure 2: Paravirtualized SystemIn theory, any operating system which provides access to the source code, such as Linux, can be paravirtualized,unlike proprietary operating systems, such as Windows. Paravirtualization is typically used in systems withhardware that does not support virtualization, although it may still be advantageous to use it for performancereasons. Full virtualization is often not as efficient as paravirtualization, because of the extra step required tointercept the privileged instructions.Although not originally designed for virtualization, it is possible to implement virtualization on top of amicrokernel. A microkernel is a reduced version of a regular operating system kernel that provides a set ofpolicies and mechanisms to access hardware resources.Any component can run on top of the microkernel. An operating system is a type of component, but it can runalongside a standalone application, such as a special device driver that runs directly on top of the microkernel.As with a hypervisor, the microkernel software runs in kernel mode between the hardware and the guest. Unlikea hypervisor, a microkernel does not perform the instructions on behalf of the guest. Instead of calling theprivileged instructions directly, the microkernel forwards the request to a user mode virtualization component,which interprets the request. The component may reside inside a guest or it may be a standalone component.The mechanism used to forward the request is called inter-process communication (IPC). As withparavirtualization, the device drivers in the guest must replace hardware access instructions with IPC messages.Page 3
Virtualization for Embedded SystemsKey Criteria for a Virtualization SolutionKey Criteria for a Virtualization SolutionBefore considering virtualization in an embedded system, some of the factors you should consider are: hardware support operating system support resource allocation and sharing memory isolation processor scheduling guest communication size of the trusted computing baseHardware SupportThe hypervisor or microkernel software runs directly on top of the hardware, and therefore must support theinstructions required by the hardware architecture. Common architectures in embedded systems are x86, ARM,PowerPC, and Sparc. The x86 processor is typically used in industrial and medical applications. Smart phones andtablets almost exclusively use ARM processors. The Sparc architecture is common in military and avionicssystems. Gaming consoles use PowerPC architectures.To support full virtualization, the hardware must include virtualization extensions. Intel and AMD both includevirtualization support in their x86 processors. The ARM Cortex A15 and A7 processors also support virtualization.Full memory isolation requires a memory management unit (MMU).Operating System SupportHardware and operating system support are closely related. As mentioned previously, proprietary operatingsystems (such as Microsoft Windows) need hardware support because they cannot be paravirtualized.If paravirtualization is necessary or desirable, the operating system source code must be available. FreeBSD,NetBSD, and all variants of Linux freely distribute their source code. Some Linux operating systems alreadyinclude the paravirtualized drivers (3).Resource Allocation and SharingGuests must share some hardware resources, such as disks and network interfaces. An application in the systemmay require dedicated access to a particular device, such as a USB port, which means that the hypervisor mustprovide a mechanism to assign exclusive access to the device to a specific guest.Page 4
Virtualization for Embedded SystemsKey Criteria for a Virtualization SolutionMemory IsolationThe memory allocation scheme must ensure that guests cannot access memory outside their own addressrange. It is important to note that any truly secure implementation requires hardware support by an MMU toguard against a malicious device driver that uses direct memory access (DMA). Both Intel and AMD processorshave MMU support, as do some ARM processors.Processor SchedulingExecution isolation is important to prevent a rogue application on a guest from monopolizing the CPU, whichessentially functions as a denial-of-service attack on the rest of the guests. If one of the guests requires a realtime response, the hypervisor must use a scheduling algorithm that can assign it a higher priority. It is alsodesirable to have a way of ensuring that lower priority tasks in one guest do not preempt higher priority tasks inanother guest.Guest CommunicationGuests may want to exchange information. For example, a component may need to provide status informationfor a user interface to display. If there is a mechanism to enable guests to communicate, the solution mustprevent a security breach using this mechanism. Any risk typically results from the mechanism the guests use tostore the data to exchange.Size of the Trusted Computing BaseThe size of the trusted computing base that implements virtualization affects system robustness. All code has acertain number of defects and the smaller the trusted computing base, the fewer defects there are likely to be.Because the code runs in privileged mode, it must be possible to contain the faults to the virtualization codewithout affecting the guests. The code with access to the privileged instructions is referred to as the trustedcomputing base (TCB).Page 5
Virtualization for Embedded SystemsOpen Source SolutionsOpen Source SolutionsOpen source solutions are often appealing because the code is free and can be modified according to the needsof the system. An ideal open source solution has active development and community members willing toinformally support users.One potential drawback of open source is the requirements under GNU GPL 1 to release the source code for anyderived work. If is necessary to make proprietary changes to the virtualization code, open source code may notbe an appropriate solution.The open source solutions described in Table 1 are licensed under either GPL or a proprietary license with thesame conditions as GPL.Table 1: Summary of Virtualization Solutions1CriteriaXenXtratumOKL4Hardware supportIntel x68, AMD, ARM v5v7, ARM CortexA15(experimental)LEON3 (Sparc V8) andIntel Itanium-64processorARM v5/v6 and Inteli386 processor; requiresan MMUOperating systemsupportAny paravirtualized guest,proprietary OS on Intelx86 and AMDAny paravirtualized guestAny paravirtualizedguestResource allocationand sharingDomain 0 mediatesshared access; exclusiveaccess using PCI passthrough interface on IntelVT-x or AMD-V hardwareExclusive access usingconfiguration settings;shared access requiresuser-implementedcommunication protocolMediated using IPCmessages; exclusiveaccess via policy moduleconfigurationMemory isolationHypervisor memorytracking, Domain 0 granttables for each guestConfigurable memoryarea statically assigned toeach guest, no sharedmemoryStatic and shared accessconfigured in theresource and policymoduleProcessor schedulingConfigurable weight andCPU cap, no exclusiveguest CPU accessConfigurable timeslot andduration with a fixed,cyclical schedulingalgorithmGlobal policy whichassigns process prioritiesacross the guestsGuest communicationVirtual network interfacein domain 0 usingstandard communicationprotocolsPort-basedcommunication using apredefined protocolThe basis of theimplementationSize of the trustedcomputing baseLarge – Domain 0 is anentire operating systemSmall hypervisor codebaseSmall microkernel codebaseRefer to (17) for more information.Page 6
Virtualization for Embedded SystemsOpen Source SolutionsXenXen is a bare metal hypervisor. There are two parts to the hypervisor implementation: a hypervisor and a specialparavirtualized guest (see Figure 3). The hypervisor code that runs directly on top of the hardware is responsiblefor virtualizing the CPU, memory, and input/output (I/O) control, including interrupt handling.Figure 3: Xen HypervisorThe special paravirtualized guest (referred to as Domain 0) has privileged access to the hardware. It managesprocessor and memory sharing, network and disk access, and communication between guests. Domain 0 can runany paravirtualized operating system, but it is typically a variant of Linux, as many Linux distributions includenative Xen support.Xen supports both full virtualization and paravirtualization. Xen has an active development community. Refer to(4) for more information.Page 7
Virtualization for Embedded SystemsOpen Source SolutionsHardware Support - Xen can run on x86 processors from Intel and AMD. Xen support for ARM processors is aproject led by Samsung which delivers and maintains Xen support for a range of ARM processors (ARM v5 - v7)for mobile devices. The project is also working on providing real-time guarantees in a virtualized environmentand multi-processor support. Refer to (5) for information. An experimental version of Xen which uses thevirtualization support introduced for the ARM Cortex A15 is underway. An experimental project to port Xen toPowerPC was abandoned.Operating System Support – Xen supports any guest that can be paravirtualized. It supports full virtualizationfor any guest that runs on Intel or AMD x86 hardware with virtualization extensions. Many Linux distributionsinclude the virtual device drivers to support paravirtualization on Xen.Resource Allocation and Sharing – Domain 0 mediates access to I/O devices by receiving requests from theguests on a virtual channel. Xen also supports a PCI pass-through interface (see Figure 4) to allow guests directand exclusive hardware access to PCI devices, such as the network interface. This hardware access methodrequires Intel VT-x or AMD-V hardware support, and can be used with paravirtualized and fully virtualizedguests. The guest must have a native device driver for the device and the Domain 0 guest must have a “pciback”version of the driver.Figure 4: PCI Pass-ThroughPage 8
Virtualization for Embedded SystemsOpen Source SolutionsThe PCI pass-through feature must be enabled in both the BIOS and in Xen, and is a potential security risk if aguest runs a malicious application or has defects in its device driver.Additional pass-through support is available for USB devices and selected graphics devices. A mechanism calledSR-IOV allows devices to be assigned to guests but shared among them. Refer to (6) and (7) for informationabout pass-through functionality.Memory Isolation – Xen assigns a static area of memory to each guest. The hypervisor uses shadow pages totranslate virtual memory access requests from the guests into the physical address. It tracks which guest ownsthe memory to enforce isolation (8). Newer releases of Xen implement a feature to allow identical guests toshare physical memory for common binaries and libraries. This feature is still in the beta stage and does nothave security support. Xen uses shared memory to implement guest communication. Domain 0 manages granttables which grant access to guests on a per page basis to ensure safe memory sharing (9).Processor Scheduling – Xen has settings to configure the CPU usage across guests. It load balances acrossCPUs using a weight and cap (limit) for each guest. It is possible to configure a guest to use only to a specified setof CPUs; however, a guest cannot be assigned exclusively to a single CPU. CPU configuration takes into accountphysical devices and hyperthreading.Guest Communication – Guests communicate through a virtual network interface in domain 0, whichimplements routing and bridging functionality. Communication is based on standard networking protocols. Thedefault implementation of guest communication requires significant overhead. The XenLoop and XenSocketsprojects attempt to address this issue (10).Size of the Trusted Computing Base – Although the portion of Xen that resides on top of the hardware issmall, domain 0 is a complete operating system. The TCB for Xen is quite large and susceptible to defects inmany areas. A newer version of Xen on ARM is experimenting with moving device drivers outside of domain 0 toisolate them from the rest of the computing base.Page 9
Virtualization for Embedded SystemsOpen Source SolutionsXtratumXtratum is a bare metal hypervisor that runs in privileged mode and virtualizes the CPU, memory, interrupts, andany devices that endanger isolation. A guest, which Xtratum documents refer to as a partition, must beparavirtualized to run on Xtratum and replace calls to privileged instructions with hypercalls. See Figure 5.Xtratum supports a special system guest with extra privileges. It can use a special set of hypercalls to manageand monitor system resources, and stop, start, or reset partitions. The access rights of a system guest are set inthe Xtratum configuration file. System guests do not have direct hardware access.The scheduling and IPC mechanisms are modeled on the Avionics Application Standard Software Interface(ARINC) 653 standards, although its goal does not include compliance to the specification. The UniversidadPolitecnica de Valencia in Spain developed Xtratum. Refer to (11) for more information.Figure 5: Xtratum ArchitectureHardware Support – Xtratum supports the LEON3 (Sparc V8) and Intel Itanium-64 processor, neither of whichhave virtualization extensions. A theoretical paper was published about porting Xtratum to PowerPC but wasnever implemented (12).Page 10
Virtualization for Embedded SystemsOpen Source SolutionsOperating System Support – Any paravirtualized guest can run on Xtratum.Resource Allocation and Sharing – I/O ports and interrupts that the hypervisor does not manage areassigned exclusively to guests in the configuration file. The device driver resides in the guest. To share devices,the system designer must implement an I/O server partition which receives requests from other guests via IPCand processes them according to its policy configuration.Memory Isolation – The configuration file defines the memory area statically assigned to the guest. There areno shared memory regions. Xtratum will run without an MMU, in which case, there is a risk of unauthorizedmemory access.Processor Scheduling –Xtratum uses a fixed, cyclical scheduling algorithm based on the timeslot and durationsettings for each guest in the configuration file. Each guest may define multiple scheduling plans and can notifythe hypervisor to switch plans using a hypercall. For example, a guest may want to switch into a maintenancemode when it has only low priority tasks to do, freeing the processor for use by other guests with higher prioritytasks. A system guest can also change the scheduling plan of a normal guest.Guest Communication – The hypervisor implements a port-based communication mechanism. Guests sendand receive messages from each other or the hypervisor on a channel that links two ports. The protocol isspecific to the sending and receiving parties. Both broadcast and direct messaging modes are available.Channels, ports, maximum message sizes, and maximum number of messages (queuing ports) are defined in theconfiguration file. Data exchange relies on buffer copying mechanisms, as there are no shared memory regions.Size of the Trusted Computing Base – The critical code for Xtratum is limited to the small hypervisor codebase. It uses a health monitor feature to detect and react to errors to contain them within the proper scope:process, guest, hypervisor, or firmware.Page 11
Virtualization for Embedded SystemsOpen Source SolutionsOKL4OKL4 3.0 2 is a microkernel implementation of virtualization. The microkernel runs on top of the hardware inkernel mode and uses IPC to mediate requests for interrupts and device drivers between guests. See Figure 6.Separate components running in user mode provide system services; the microkernel does not provide them.A separate resource and policy model, which runs outside the microkernel in user space, holds the configurationof the CPU scheduling policy and the memory allocation. The microkernel runs in privileged mode and the guestsrun in user mode. Guests must replace hardware access instructions in their device drivers with IPC messages.Open Kernel Labs, which is owned by General Dynamics, sponsors the OKL4 project. Refer to (13) for moreinformation.Figure 6: OKL4 ArchitectureHardware Support - OKL4 supports ARM v5/v6 and Intel i386 processors, none of which have virtualizationextensions. OKL4 must run on a processor with an MMU.Operating System Support – Any paravirtualized guest can run on OKL4. Open Kernel Labs provides aparavirtualized version of Linux to use as a guest.2Not to be confused with the OKL4 4.0 microvisor, which requires a commercial licensePage 12
Virtualization for Embedded SystemsOpen Source SolutionsResource Allocation and Sharing – OKL4 mediates device access using IPC messages (Figure 7). It relaysrequests for device access from a guest’s virtual driver to the physical driver, which may be either a standalonedriver, or reside in another guest. The policy module controls which guest can drive a particular device bymapping device registers.Figure 7: OKL4 IPC ModelMemory Isolation – The memory allocated for each guest is statically configured in the resource and policymodule. There are also configuration settings for shared memory regions and policy settings to determine whichguests can access the shared regions. The policy module has a monopoly over operations that consume kernelmemory; it can control which guest is allowed to consume such kernel resources to guard against denial-ofservice attacks on the system, for example, by a rogue guest kernel.Processor Scheduling – The s
Open Source Compliance Open source licenses typically allow proprietary code to interact with open source code if the two communicate only via a messaging interface. If the open source and proprietary code run in separate operating systems, the hyperviso
