A Smart Way To Manage Packages In Yocto Project - ELinux

A Smart Way to Manage Packagesin Yocto ProjectMar 11th, 2016Fan Xin, Fujitsu Computer Technologies Limited1378ka02Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

whoami Fan Xin, Fujitsu Computer Technologies Ltd. Embedded Linux Developer In-House Embedded Linux Distributorof Fujitsu Our Distribution includes LTSI Kerneland is built with Yocto Project Our Distribution is used for IVI, Server System Controller, Storage System, Network Equipment,Printer, etc.IVI：In-Vehicle Infotainment1Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Fujitsu’s contribution for Yocto Projectmeta-openembedded.gittop contributors by employer (2015-01-01 to 2015-12-31)None625 (46.1%)Wind River309 (22.8%)Fujitsu160 (11.8%)Intel82 (6.0%)Mentor Graphics34 (2.5%)O.S. Systems24 (1.8%)National Instruments19 (1.4%)MontaVista16 (1.2%)OpenDreambox12 (0.9%)Freescale9 (0.7%)Linaro8 (0.6%)ENEA AB5 (0.4%)Leica5 (0.4%)Aker Security Solutions4 (0.3%)BMW3 (0.2%)2Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Agenda Package Manager Comparison Package Management Problem Analysis in Yocto Project Introduction of Smart Package Manager Fujitsu’s Contribution and Next Step3Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Package Manager Comparison4Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Package Manager Package Manager A collection of software tools that automates the process of installing,upgrading, configuring, and removing computer programs Deals with packages, distributions of software and data in archive files Maintain a database of software dependencies and package information Common Package Management System rpm, deb, ipkg, opkg base of package manager Common Package Managers Advanced Packaging Tool (APT） Yellowdog Updater Modified （YUM） Dandified Yum (DNF)https://en.wikipedia.org/wiki/Package managerhttps://en.wikipedia.org/wiki/List of software package management systems5Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Advanced Packaging Tool (APT） APT ( Advanced Packaging Tool ) The Advanced Package Tool, or APT, is a free software userinterface that works with core libraries to handle the installationand removal of software Used on Debian Linux distribution and its variants. Depends on deb package format Works with repositories, which are collections of packages,https://en.wikipedia.org/wiki/Advanced Packaging Tool6Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Yellowdog Updater Modified （YUM） A package manager for automatic install, updates, uninstallpackage and dependency management. Originally developed to manage Red Hat Linux systems atDuke University's Physics department Used on RPM-based distributions, such as RedHat Yum depends on RPM Works with repositories, which are collections of packages,https://en.wikipedia.org/wiki/Yellowdog Updater, Modified7Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Dandified Yum (DNF) Next generation version of the Yellowdog Updater Modified(yum) Using RPM, libsolv and hawkey libraries Introduced in Fedora 18 and become the default packagemanager for Fedora since version 22https://en.wikipedia.org/wiki/DNF (software)8Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Comparison of Package ManagersItemsAptYumDNFLatest Version1.1.103.4.31.1.4.deb .rpm (apt-rpm) dora(Since version 22)LicenseGNU GPLGPL v2GPL iki/Advanced Packaging Toolhttps://en.wikipedia.org/wiki/Yellowdog Updater, Modifiedhttps://ja.wikipedia.org/wiki/DNF 82%A2)9Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Comparison of Package ManagementSystemsItemsrpmCheck PackageLicenseInformation File format.rpmUsed byOpenEmbedded Develop g.deb.udeb.ipk.ipk ActiveActive ( forkfrom ipkg )DiscontinuedGPLGPL v2GPL en.wikipedia.org/wiki/RPM Package Managerhttps://en.wikipedia.org/wiki/Deb (file a.wikipedia.org/wiki/Ipkg10Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Present Situation in Yocto ProjectRecipe yDeployEmbedded Devices Flexibility Flex to select package before building Efficiency Create root file system, binary packages (rpm, ipkg, opkg) Optimization Easy to optimize for each distribution11Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Package Management ProblemAnalysis in Yocto Project12Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Problems“ Yocto creates a custom one for you ” The Advantage of custom embedded Linux The value of custom The balance between custom and minimize ght 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Problems Can not know what package is built due to the packagesdependence. Have to rebuild the root file system every time when havesome updates or modifications for some packages. Produce fragmentationFlexibility: Prefer graphical interface and detailed infoEfficiency: Inefficiency, in some degreeOptimization : Produce fragmentation14Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

ImageBuildDeployBuild DeployBuild DeployBuild Deploy Build and Deploy for Each TargetsThese cars use same architecture and own individual build directory.15Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

ImageBuildDeployBuild DeployBuild DeployBuild Deploy Build and Deploy for Each Targets16Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

ImageBuildDeployBuild DeployBuild DeployBuild Deploy Build and Deploy for Each TargetsSimulationBuildUsing Package ManagerDeployFull build time: 5 6 hoursDeploy time: 1 hourTarget Type: 50Before:Cost (5 1)*50 300 hoursAfter:Cost 5 1*50 55 hours Build Once, Deploy Anywhere17Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Cost ComparisonBuild glibc package and deploy for car ABuild glibc package and deploy for car BBuild glibc package and deploy for car Chttps://access.redhat.com/articles/2161461Build glibc package and deploy for car A, B, C Feb 17th, glibc publics critical security flaw. This vulnerabilityshould be fixed in all targets immediately. For each kind of target, the glibc package has to been buildedand deployed.SimulationCost ( Build Time Deploy Time ) * Device Number ( 1 0.2 ) * 50 60 hoursCost Build Time Deploy Time * Device Number 1 0.2 * 50 11 hours18Glibc Build Time: 1 hoursDeploy Time: 0.2 hoursDevice Number: 50 kindsCopyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

What we should doRebuild the root file systemand redeploy them into productionevery time ?Use the package manager to manageindividual package efficiently?19Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Introduction of Smart PackageManager20Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Smart Package Manager Overview Aim to create smart and portable algorithms for solvingadequately the problem of managing software upgrading andinstallation. Homepage: smartpm.github.io/smart License: GPL v2 Works with APT, APT-RPM, YUM, etc. Started on May 4th, 2004, and version 1.0 was released onAug 14th, 2008 The latest version is 1.5 Merged into Yocto Project 1.4 (dylan), but unavailable in YoctoProject so /2013/getsmart-smart-package-manager21Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Features Modular Support PRM, DPKG, and Slackware package management systems Smart Transactions Thousands of packages and relations are being considered Not only find a solution, but find the best solution (install, remove,upgrade, etc). Channels Many different channel types are supported Priority Handing Hand integration of multiple channels and setup preferred packageversions Downloading Mechanism Fast parallel downloading mechanism, multiple connections http://smartpm.github.io/smart/22Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Smart UsageSmart package manager uses the following command tomanage packages.Usage: smart command Action commands update, install, reinstall, upgrade, remove, check, fix, download, clean Query commands search, query, newer, info, stats Setup commands config, channel, priority, mirror flagRun “smart command –help ” for more information23Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Project StatisticsStatistics aboutCommit change by year monthCommit change by year, authorCommit change by authorFile change by yearTag -08-1583624Copyright 20152016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Problems Smart still has some problems needed to solve before using it. Problem #1 There is bug in the rpm and smart command in default toolchain. Send the patch to community to fix it http://patchwork.openembedded.org/patch/106097/ Problem #2 Need to configure the environment before using smart, or it can not usenormally Prepare the script to configure environment automatically25Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Problems Problem #3 Smart Package Manager uses GTK/Qt4 as the graphical framework, butGTK/Qt4 is disabled in toolchain Develop Text GUI with python to make smart user-friendly and easy touse26Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Smart Package Manager Usage Install sh Initial environment. /opt/environment-setup-core2-64-linuxWe use this script to finish the environment initialization,such as environment variable setting Install package into root file system Command line interfacesmart --data-dir {rootfs-dir}/var/lib/smart install Graphical interfacesmart --data-dir {rootfs-dir}/var/lib/smart --interface tgui27Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Select install type28Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Select packages29Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Search package30Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Select dbg packages31Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Installing32Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Created Root Filesystem33Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

DemoSmartPackageManager34Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Conclusion and Next Step35Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

License ManagementPackagesPackage ManagerEmbedded DevicesSRPM packagesRecipe filesBuildSPDX files Accompanied with the package files, SRPM packages andSPDX files are created to manage license information SRPM Packages are classified by license type36Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Community Activity Send our patches to community, but not accepted In order to solve the problem, need to active the tch/106097/37Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Conclusion Summary the current package managers and problems Compare the package managers Introduce package management problem in Yocto Project The package manager is necessary Contribution to Ycoto Project Introduce the Smart Package Manager Fix the bugs and make Smart Package Manager easy to use License management Not stick to Smart Package Manager, but want to solvepackage management problem38Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Next Step Continue to make contribution for Yocto Project The interface of Smart is not so smart currently. Fujitsu will improve the interface to make it easier to use. Remote access over a network connection like APT, YUM andDNF Add the option to create image automatically39Copyright 2016 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Copyright 2015 FUJITSU COMPUTER TECHNOLOGIES LIMITED

Build glibc package and deploy for car B Build glibc package and deploy for car C Build glibc package and deploy for car A, B, C Cost ( Build Time Deploy Time ) * Device Number ( 1 0.2 ) * 50 60 hours Cost Build Time Deploy Time * Device Number 1 0.2 * 50 11 hours Glibc Bu