EXtensible Versatile HypervISOR

1y ago
535.67 KB
37 Pages
Last View : 18d ago
Last Download : 6m ago
Upload by : Konnor Frawley


Little About Me Hypervisor and Linux kernel developer with 12 years of industry experience Post-graduated (Masters) in 2009 from IIT Bombay, India Work full-time for Qualcomm as Server virtualization expert Maintain Xvisor as hobby project in personal time (since 2010) Open source contributions: 3300 patches in Xvisor (http://xhypervisor.org/)100 patches in Linux ARM/ARM64/RISC-V (https://www.kernel.org/)24 patches in Linux KVM ARM64 (https://www.kernel.org/)16 patches in Atomthreads RTOS (https://atomthreads.com/)Few patches in Xen ARM, QEMU, KVMTOOL, etcXVISOR: EXTENSIBLE VERSATILE HYPERVISOR2

Agenda Overview Virtualization Infrastructure Domain Isolation Device Virtualization Domain Messaging Footprint Xvisor for Automotive ReferencesNOTE: Domains in Automotive world are referred to as Guests in XvisorXVISOR: EXTENSIBLE VERSATILE HYPERVISOR3


What is Xvisor? XVISOR eXtensible Versatile hypervISOR Xvisor is an open-source GPLv2 Type-1 monolithic (i.e. Pure Type-1) hypervisor Community driven open source project(http://xhypervisor.org, xvisor-devel@googlegroups.com) 8 years of development and hardening (since 2010) Supports variety of architectures: ARMv5, ARMv6, ARMv7, ARMv7ve, ARMv8,x86 64, and RISC-V (work-in-progress) First paper in IEEE PDP 2015 titled “Embedded Hypervisor Xvisor: Acomparative analysis”XVISOR: EXTENSIBLE VERSATILE HYPERVISOR5

Hypervisor Classification - TraditionalType1 Examples: Xvisor, Xen, VMWareESX Server, Microsoft HyperV, OKL4Microvisor, etcType2 Examples: Linux KVM, FreeBSDBhyve, VMWare Workstation, OracleVirtualBox, etcXVISOR: EXTENSIBLE VERSATILE HYPERVISOR6

Xvisor - Complete Monolithic - Type1GuestUser SpaceGuestUser SpaceGuestUser SpaceGuest KernelGuest KernelGuest sGuest0Guest1GuestNGuest IOEmulationDevice Drivers(Host HWAccess)CPUVirtualizationManagementTerminalXvisor HypervisorIOMMUNetwork, Block, Input, VirtualizationHost HardwareXVISOR: EXTENSIBLE VERSATILE HYPERVISOREL0-NS (ARM64)EL1-NS VCPUsVCPUsVCPUsEL2-NS (ARM64)Hypervisor Component7

Lots of features Virtualization Infrastructure: Device tree based configurationSoft real-time pluggable schedulerHugepages for Guest and HostTickless and high-resolution timekeepingHost device driver frameworkThreading frameworkRuntime loadable modulesManagement terminalLight-weight filesystemWhite-box testing Many More XVISOR: EXTENSIBLE VERSATILE HYPERVISOR8

Lots of features (Contd.) Domain Isolation: VCPU and Host Interrupt Affinity Spatial and Temporal Memory Isolation Device Virtualization: Pass-through device supportBlock device virtualizationNetwork device virtualizationInput device virtualizationDisplay device virtualizationVirtIO v0.9.5 for Para-virtualization Domain Messaging: Sharing On-chip Coprocessor Zero-copy Inter-Guest CommunicationXVISOR: EXTENSIBLE VERSATILE HYPERVISOR9


Device Tree Based ConfigurationThree types of device tree (DT):1. Host DT: Device tree which describes underlyinghost HW to Xvisor Used by Xvisor at boot-time2. Guest Xvisor DT: Device tree which describes Guestvirtual HW to Xvisor Used by Xvisor to create Guest3. Guest OS DT: Device tree which describes Guestvirtual HW to Guest OS Used by Guest OS at boot-timeGuest OSGuest OSGuest st OS DTGuest OS DTGuest OS DTGuest0Guest1GuestNGuest Xvisor DTGuest Xvisor DTGuest Xvisor DTXvisorHypervisorGuest IO EmulationDevice Drivers(Host HW Access)CPUVirtualizationVirtIO CPUsvCPUsHost DTHost HardwareXVISOR: EXTENSIBLE VERSATILE HYPERVISOR11

Soft Real-time Pluggable Scheduler Scheduling entity is a VCPU Two types of VCPUs:1. Normal VCPU: A VCPU belonging to Guest/VM2. Orphan VCPU: A VCPU belonging to Hypervisor for background processing Orphan VCPUs are very light-weight compared to Normal VCPUs Scheduler supports pluggable scheduling policy, available policies: Fixed priority round-robin Fixed priority rate monotonic Scheduling policies are soft real-time Scheduler supports multi-processors (or SMP Host)XVISOR: EXTENSIBLE VERSATILE HYPERVISOR12

Hugepages for Guest and Host Xvisor uses Stage1 (regular) page table for “Hypervisor virtual address” to “Hostphysical address” mappings Host hugepages are bigger mappings in Stage1 (regular) page table Host hugepages make Xvisor memory accesses faster Xvisor uses Stage2 (nested) page table for “Guest physical address” to “Hostphysical address” mappings Guest hugepages are bigger mappings in Stage2 (nested) page table Guest hugepages make Guest OS memory accesses faster For ARM64 and x86 64, hugepage sizes are 2M and 1GXVISOR: EXTENSIBLE VERSATILE HYPERVISOR13


VCPU and Host Interrupt Affinity VCPU affinity is an attribute of VCPU specifying Host CPUs on which it isallowed to run Using VCPU affinity, we can assign particular Host CPUs for: Guest VCPUs (Normal VCPUs) Xvisor background threads (Orphan VCPUs) Host interrupt affinity is an attribute of Host interrupt specifying Host CPUs onwhich it can be processed Using Host interrupt affinity, we can assign particular Host CPUs for Hostinterrupts of a Guest pass-through device Host interrupt affinity of per-CPU Host interrupts (such as IPIs) cannot bechangedXVISOR: EXTENSIBLE VERSATILE HYPERVISOR15

Spatial and Temporal Memory Isolation Spatial memory isolationachieved using cache-coloring onlast level cache for Guest RAM Temporal memory isolationachieved using CPU performancemonitoring unit (PMU) to controlmemory access rate by GuestCPU3CPU2CPU1CPU0PMULast Level CacheSet0Set1Set2Set3Set4Set5Set6Guest0 - INTEGRITY/QNXGuest1 - Android/AGLGuestN - RTOSSetNIEEE ICIT 2018 paper: “SupportingTemporal and Spatial Isolation in aHypervisor for ARM MulticorePlatforms”InterconnectXVISOR: EXTENSIBLE VERSATILE HYPERVISORMEMC0MEMC0DRAMDRAM016


Device Virtualization Types Types of Virtual Devices:1.Software Emulated Device: Real-world device emulated by hypervisor. Examples,Emulated UART 8250, etc.2.Paravirtual Device: Pseudo-devices emulated by hypervisor which are designed tominimize register programming. Examples, VirtIO Net, VirtIO Block, VirtIO Consoleetc.3.Pass-through Device: Direct access of host device from Guest/VM. This requiresIOMMU support in Host. Examples, PCI e1000 network adapter accessed byGuest/VM, SATA AHCI controller accessed by Guest/VM, etc.4.Partial Pass-through Devices: Access part of a host device from Guest/VM. Thisrequires IOMMU support in Host and Host device should have special support.Examples, SRIOV based PCI Network Adapter, GPU with multiple channels, etc. All types of virtual devices supported by XvisorXVISOR: EXTENSIBLE VERSATILE HYPERVISOR18

Pass-through Device Support Linux compatibilityheaders for driversrunning in Xvisor IOMMU and Interruptcontroller virtualizationfor drivers running inGuest OS Access part of devicefrom Guest OS usingpartial pass-through: Custom driver Custom FrontendsGuest1Device Drivers(Host HW Access)Partial PassThroughGPUDisplay0XVISOR: EXTENSIBLE VERSATILE estNXvisorHypervisorGuest IOEmulationOrphanOrphanVCPUsDevicesVCPUsRTOSVirtIO anVCPUsVCPUsvCPUsSensorsHost Hardware19

Block Device Virtualization Consist of:1. vdisk: Logical entity whichgets block read/writerequests from Guests.Examples, Storage deviceemulators, and VirtIO Blockbackends.2. blockdev: Logical entitywhich represents hoststorage device or a partitionon host storage device.Examples, MMC, NAND,SATA, etc.INTEGRITY/QNXAndroid/AGLRTOSVirtIO BlockFrontendVirtIO BlockFrontendVirtIO BlockFrontendGuest0Guest1GuestNGuest IOEmulationDevice Drivers(Host HW Access)VirtIO BlockBackends(vdisks)XvisorHypervisorDisk VirtualizationFramework We can attach ablockdev to a vdiskHost HardwareXVISOR: EXTENSIBLE VERSATILE HYPERVISORCPU VirtualizationHost CPUsStorageDevice20

Network Device Virtualization Consist of:1. netport: Logical entitycapable of consuming andgenerating packets.Examples host networkdrivers, NIC emulators, andVirtIO Net backends.2. netswitch: Logical entitywhich does packet routingbetween netports. Variousrouting policy available:hub, bridge, vlan, etc.INTEGRITY/QNXAndroid/AGLRTOSVirtIO NetFrontendVirtIO NetFrontendVirtIO NetFrontendGuest0Guest1GuestNGuest IOEmulationDevice Drivers(Host HW Access)VirtIO NetBackends(netports)XvisorHypervisorNetwork VirtualizationFramework(netswitch)Host HardwareXVISOR: EXTENSIBLE VERSATILE HYPERVISORCPU VirtualizationHost vCPUsNetworkAdapter21


Sharing On-chip Coprocessor SOCs can have on-chipcoprocessors for securedprocessing Linux applications cancommunicate with on-chipcoprocessor using RPMSGcharacter device Virtual messaging domainto define a set Guestsallowed to communicatewith on-chip ApplicationApplicationVirtIO RPMSGFrontendVirtIO RPMSGFrontendVirtIO RPMSGFrontendGuest0Guest1GuestNGuest IOEmulationDevice Drivers(Host HW Access)VirtIO RPMSGBackendsXvisorHypervisorVirtual MessagingFrameworkHost HardwareXVISOR: EXTENSIBLE VERSATILE HYPERVISORCPU VirtualizationOn-chipCoprocessorRPMSG hipCoprocessorCoprocessor23

Zero-copy Inter-Guest Communication Achieved using:1. VirtIO RPMSP: Used for control messages Name-service notifications2. Shared Memory: Used for actual data transfers Very fast zero-copy Linux applications cancommunicate acrossGuests using RPMSGcharacter device Virtual messaging domainto define a set of Guestsallowed to ApplicationApplicationVirtIO RPMSGFrontendVirtIO RPMSGFrontendVirtIO RPMSGFrontendGuest0Guest1GuestNShared MemoryData (Shared Memory)Control (VirtIO RPMSG)Guest IOEmulationDevice Drivers(Host HW Access)VirtIO RPMSGBackendsXvisorHypervisorVirtual MessagingFrameworkCPU VCPUsVCPUsvCPUsHost HardwareXVISOR: EXTENSIBLE VERSATILE HYPERVISOR24


Code Size and Memory FootprintLines of Code CommentsCodeBLOBSizearch/arm/714320614.text969 KB*core/897435419.data129 KBcommands/102510145.rodata329 KB*daemons/147526.bss202 KBdrivers/9427*43922*vmm.bin1445 034* Can be further decreased or increased basedon compile-time configurationRuntime MemorySizeText memory freed at boot-time112 KBTypical memory usage21 MB*Max VAPOOL limit32 MB*NOTE: Stats gathered from Xvisor-next on 22nd September 2018 for ARM64XVISOR: EXTENSIBLE VERSATILE HYPERVISOR26


Why Xvisor is ideal for Automotive? No dependency on any Guest OS for running management tools Single software providing complete virtualization solution Guest types described using device tree instead of fixed Guest types Para-virtualization complying open-standards (such as VirtIO) Pass-through (or direct access) device support Zero-copy inter-guest communication Spatial and temporal memory isolation between Guests Low memory footprint with reasonable code size Playground for academic researchXVISOR: EXTENSIBLE VERSATILE HYPERVISOR28

On-going Work in Xvisor Guest image authentication VirtIO input VirtIO GPU Netport Rx/Tx throttling Vdisk IO request rate-limiting Fixed priority deadline scheduler RISC-V support Upgrade to VirtIO 1.0 support And other stuff XVISOR: EXTENSIBLE VERSATILE HYPERVISOR29


References Embedded Hypervisor Xvisor: A comparative analysis (IEEE PDP 2015)(http://ieeexplore.ieee.org/xpl/login.jsp?tp &arnumber 7092793 ) Xvisor: An open-source, lightweight, embedded hypervisor for your car(FOSDEM nt/car hypervisor/ ) Xvisor VirtIO CAN: Fast virtualized CAN (ERTS 2016)(http://xhypervisor.org/pdf/Xvisor VirtIO CAN Fast virtualized CAN.pdf ) Supporting Temporal and Spatial Isolation in a Hypervisor for ARM Multicore Platforms(IEEE ICIT sue.jsp?punumber 8342303 ) Reconciling Security with Virtualization: A Dual-Hypervisor Design for ARM TrustZone(IEEE ICIT sue.jsp?punumber 8342303 )XVISOR: EXTENSIBLE VERSATILE HYPERVISOR31



Hypervisor Classification - New Design of a hypervisor can be further classified based on three aspects:1. CPU virtualization What part of hypervisor virtualize CPU registers and MMU ?2. Host hardware access What part of hypervisor access host devices (i.e. Host device drivers) ?3. Guest IO emulation What part of hypervisor virtualize peripherals (i.e. Guest I/O devices) ?Hypervisor DesignComplete MonolithicPartially MonolithicMicro-KernelizedIEEE PDP 2015 paper: “Embedded Hypervisor Xvisor: A comparative analysis”XVISOR: EXTENSIBLE VERSATILE HYPERVISOR34

Hypervisor Classification - New (Contd.)Complete MonolithicPartially MonolithicMicro-KernelizedSingle software layerExtends an existing OS kernelMicro-kernel providing virtualizationMain Hypervisor: Host hardware access CPU virtualization Guest IO emulationRemaining Stuff: Optional host hardware accessfrom virtual machine(s)Main Hypervisor: Host hardware access CPU virtualization in host OSRemaining Stuff: Optional host hardware access fromvirtual machine(s) Guest IO emulation from user-spacesoftwareMain Hypervisor: Basic host hardware access CPU virtualization in hypervisormicro-kernelRemaining Stuff: Complete host hardware access inmanagement virtual machine(s) Guest IO emulation in managementvirtual machine(s)Type-1Type-2Type-1Examples: Xvisor, VMware ESXserverExamples: Linux KVM, FreeBSD Bhyve,VMware Workstation, OracleVirtualBoxExamples: Xen, Microsoft HyperV, OKL4MicrovisorXVISOR: EXTENSIBLE VERSATILE HYPERVISOR35

Xen - Micro-kernelized - Type1Dom0 User SpaceXen Toolstack (Management) QEMU (Guest IO Emulation)Dom0 KernelDevice Drivers(Host HWAccess)Xen PVBackendsDom0 (Guest0 - Control)CPUVirtualizationDomUUser SpaceDomUUser SpaceDomU KernelDomU KernelXen PVFrontendsXen PVFrontendsDom1 (Guest1)DomN (GuestN)XenHypervisorHost HardwareXVISOR: EXTENSIBLE VERSATILE HYPERVISORBasic Host HWAccessEL0-NS (ARM64)EL1-NS (ARM64)EL2-NS (ARM64)Hypervisor Component36

KVM - Partially Monolithic - Type2Host User SpaceGuest User SpaceEL0-NS (ARM64)Process1ProcessNQEMU/KVMTOOL(Guest IOEmulation VirtIOBackends)EL0-NS (ARM64)Guest KernelVirtIOFrontendsEL1-NS (ARM64)Guest0Device Drivers(Host HW Access)Linux Kernel(Host Kernel)KVM Module(CPU Virtualization)KVM Module Low-Visor (Only for ARM/ARM64)Host HardwareXVISOR: EXTENSIBLE VERSATILE HYPERVISOREL1-NS (ARM64 non-VHE)EL2-NS (ARM64 VHE)EL2-NS (ARM64)Hypervisor Component37

Lots of features (Contd.) Domain Isolation: VCPU and Host Interrupt Affinity Spatial and Temporal Memory Isolation Device Virtualization: Pass-through device support Block device virtualization Network device virtualization Input device virtualization Display device virtualization VirtIO v0.9.5 for Para-virtualization

Related Documents:

number of open-source hypervisors available such as Xen, Linux KVM and OKL4 Microvisor, this is the first paper to present the open-source embedded hypervisor eXtensible Versatile hypervISOR (Xvisor) and compare it against two of the commonly used hypervisors KVM and Xen in-terms of comparison factors that affect the whole system performance.

2.1 XML (Extensible Markup Language) 13 2.2 XSD (XML Schema Definition) 18 2.3 MathML (Mathematical Markup Language) 23 2.4 SPS (StyleVision Power Stylesheet) 25 2.5 XSL (Extensible Style Language) 27 2.6 XSLT (Extensible Style Language Transformations) 31 2.7 XSL:FO (Extensible Style Language: Formatting Objects) 32 2.8 XPath (XML Path Language) 33 3 Estudi de l'estàndard XML DocBook 37 3.1 .

Chaos monkey Ephemeral resources Strong consistency Eventual consistency. Designing a zone for a traditional workload vCenter/XenCenter Hypervisor Cluster Hypervisor Cluster Hypervisor Cluster Enter

Integrated gateway for VLAN, VxLAN, and NVGRE networks from virtual to physical Normalisation for NVGRE, VXLAN, and VLAN networks Customer not restricted by a choice of hypervisor Fabric is ready for multi-hypervisor Virtual Integration Network Admin Application Admin PHYSICAL SERVER VLAN VXLAN VLAN NVGRE VLAN VXLAN VLAN ESX Hyper-V KVM Hypervisor

system to enterprise network y establish service has is cloud hypervisor system connect hypervisot API emulators to service bus T 4S. connect additional front-erid 420 plug-ins to service bus 2. connect back-end pitig phag-ins to 3.425 Service bus establish hypervisor instatices expose hypervisor instances to enterprise network ----- 35

Virtualization: Jailhouse Hypervisor on AM572x Reference Design External Memory Faces: Memory management Level 3 (L3) and level 4 (L4) interconnects System and serial peripherals 2.2 Design Considerations Jailhouse is a static partitioning hypervisor based on Linux. Jailhouse can run bare-metal applications or

Extensible Markup Language (XML), Extensible 3D (X3D), Extensible Stylesheet Language (XSL), US Message Text Format (USMTF), XML-MTF, Land Command and Contol Information Exchange Model (LC2IEDM), Generic Hub (GH), Amphibious Raid 16. PRICE CODE 17. SECURITY Unclassified 18. SECURITY Unclassified 19. SECURITY Unclassified 20. LIMITATION OF ABSTRACT UL NSN 7540-01-280-5500 Standard Form 298 (Rev .

Data Modeling @Uber September 12, 2019 Anand Mundada, Gaurav Tungatkar. 01 Transportation Business 02 Extensible Entity Data Model Agenda. Transportation Business. Rider Vehicle Driver . Fleet Driver Car Driver Courier A Restaurant. Extensible Entity Data Model. Data Model for Entity Unique identifier Common schema shared by all customers .