AI And Autonomous Driving: Key Ethical - Institute For Ethics In .
Technical University of MunichMunich Center for Technology in SocietyInstitute for Ethics in Artificial IntelligenceResearch Brief – February 2021AI and Autonomous Driving: Key ethicalconsiderationsby Franziska Poszler and Maximilian GeißlingerResearchers and companies around the world are working onadvancing autonomous driving technology with a major goal ofincreasing road safety and comfort of motorized transport.Autonomous vehicles (AVs) are expected to have a global impact thatwill change society, the safety of roadways and transportationsystems in the future. The field is moving quickly, leaving governanceand ethical consideration to catch up. However, it is important toconsider policy and ethical challenges and trade-offs, as well aspotential solutions, already while AVs are being developed.
Institute for Ethics in Artificial IntelligenceTechnical University of MunichMore than 25.000 people lost their lives on the roads of the European Union in 2018(ERSO, 2018). Several studies claim that more than 90 % of these fatalities werecaused by human error (Smith, 2013). In light of these facts, researchers andcompanies around the world are working on advancing autonomous drivingtechnology with a major goal of increasing road safety and comfort of motorizedtransport. Autonomous vehicles (AVs) are expected to have a global impact that willchange society, the safety of roadways and transportation systems in the future.What are autonomous vehicles and how dothey use AI?The objective of AVs is to move in a goal-orientedway without the intervention of a human driver. Inthis case, sensors and actuators controlled byintelligent software perform the driving task.According to SAE (SAE International, 2018), fivelevels of autonomy in AVs are distinguished. Whileup to level 2 the human driver remains in charge,from level 3 the complete driving task is handedover to the software. At level 3, however, a humandriver must be ready to take over the driving taskfrom the system within a predefined time horizon(e.g., of 10 seconds). At level 4, this is no longernecessary under a set of conditions (such as goodweather). In contrast, in level 5 the software candrive under all conditions. While level 2 ofautonomy is already available in productionvehicles today, developers worldwide are workingtowards level 3 and level 4. All major carmanufacturers (such as Volkswagen or Toyota),but also software companies (such as Google'ssubsidiary Waymo or Apple) and start-ups (e.g.,Zoox) currently engage in a technologycompetition for level 3 and level 4 systems. Mostrecently, Waymo drew attention with itsdevelopments on level 4 by foregoing the presenceof a safety driver in their robotaxis in Phoenix(Waymo, 2020). Their vehicles are now onlymonitored remotely.One day AVs will have to make decisionswhich would be morally difficult forhumanshttps://ieai.mcts.tum.de/In order to further advance the development ofautonomous vehicles from level 3, methods fromthe field of artificial intelligence (AI) are now beingused more and more frequently. Particularly in theareas of detection and behavior prediction of otherroad users and the subsequent decision making ofthe AV, various AI methods already constitute thestate of the art. Thus, the field is moving quicklyand, as often is the case with rapid advancing intechnology, governance and ethical considerationare left to catch up.Potential impacts of AV on efficiency,accessibility and safetyIn addition to positive impacts on traffic safety,significant efficiency gains are expected due to theelimination and coordination of driving tasks.Researchers and companies already demonstratealternatives for spending time in the vehicle(Wadud & Huda, 2021). By automating transportroutes and thus possibly shifting traffic away fromrush hours, traffic could be relieved in the long termand time spent in traffic jams could be reduced.The use of so-called robotaxis or autonomousIEAI Research Brief2
Institute for Ethics in Artificial Intelligencebuses in urban traffic could also potentially reducethe costs and increase accessibility of mobility inthe long term. AVs enable people with limitedmobility (e.g., due to age or disability) to increasetheir participation in traffic. By giving these groupsof people the chance of increased mobility, theywill gain greater social and societal participation.This effect is particularly important in times ofdemographic change, where there are increasingnumbers of people with limited mobility.AVs should conduct a responsibleassessment and balancing of risksTechnical University of Munich(1)Technical safety:Vision Zero states that eventually no one will andshould be killed or seriously injured in road traffic(Ministry of Transport and Communications, 1997).In line with this goal, a prime rationale forintroducing AVs onto streets is the expectedpotential of decreasing fatalities that usually wouldarise from human error (Bartneck et al., 2019).However, to do so, AVs’ technical robustness andsafety needs to be ensured. In this regard, relevantquestions to be addressed are: What are ‘safe’ fallback plans for AVs? How can potential threats to AVs (e.g.cybersecurity threats) be prevented? How can we experiment with AVs and test AVson the road without harming humans?The potential positive effects that autonomousvehicles are not without controversy, particularly interms of safety. For example, road safety dependsheavily on where and how autonomous vehiclesare introduced on public roads. In addition, newsources of danger open up, for example throughhacker attacks on the AV. The fears of potentialusers also include the issue of data privacy andsurveillance. Thus, for the final introduction of AVson public roads, the technological perspective isonly one aspect.It is assumed that one day AVs will have to makedecisions which would be morally difficult forhumans, and to which industry and research havenot yet provided solutions. This is whypolicymakers as well as car manufacturers have tofocus on the inclusion of ethical considerations intothe software of AVs.What are important ethical considerations?The following represent some of the identified keyissues that need to be addressed in AVdevelopmentandcorrespondingrecommendations to advance the developmentand implementation of AVs in a responsiblemanner, as well as some potential solutions tothese problems. These insights are based on thefindings of the AI4People-Automotive Committee(Lütge et al., 2021) as well as on the work of theANDRE-project.1Potential solutions:To bypass technical failures and outages, AVsneed to pass an official test that assures thesystem’s accuracy, reliability and fallback options.Standards such as the IEEE P7009 (Standard forFail-Safe Design of Autonomous and SemiAutonomous Systems) (IEEE, 2019) or the SAEDriving Safety Performance Assessment Metrics(SAE International, 2018) could serve as abaselinetodevelopappropriatetests.Furthermore, cybersecurity threats are particularly“new” and important to AVs compared to regularvehicles. Therefore, in addition to conventionalsafety tests, cybersecurity management systemsshould be developed relying on existing guidelinessuch as SAE J3061 (SAE International, 2016).Concerning the rollout of AVs, a stepwiseapproach is recommended meaning that1This is a project of the IEAI, for more information mous-drivingethics/https://ieai.mcts.tum.de/IEAI Research Brief3
Institute for Ethics in Artificial Intelligencesimulations and hardware-in-the-loop testingshould be conducted before experimenting onopen roads (European Commission, 2020a).(2)Responsible balancing of risks:Realistically, AVs do not need to make decisionsbetween the outright sacrificing of some individualsto protect others. Instead, they need to implicitlydecide about who is exposed to greater risks(Bonnefon et al., 2019). For example, adjusting thelateral position of AVs on a lane can influence therisk posed to other traffic participants (e.g., granteddistance to cyclists). Therefore, at every time inmundane traffic scenarios, AVs should conduct aresponsible assessment and balancing of risks.This balancing should never be based on personalcharacteristics of individuals such as gender orage (Lütge, 2017), but rather should take intoconsideration more objective features. In thisregard, the relevant questions to be addressedare: What are the objective factors that AVs can relyon in their decision-making and risk allocationprocess? How can this be technically implemented inAVs?By rapidly processing huge amounts ofdata, AI can replace complex transportsystems problemsPotential solutions:More objective factors are, for example, factorsthat influence the collision probability and/or theestimated harm, such as the speed of the trafficparticipants or the impact angle under which thecollision would occur (Geißlinger et al., 2021).These risk assessments can then be integratedinto the trajectory planning of AVs in the form of anoptimization problem and validity checks (e.g., formaximum acceptable risk). A correspondingmathematical formulation of risk in the context ofAVs is developed within the ANDRE-project.(3)Human agency:AVs have enormous potential to influence humanagency, either in a positive manner by, forexample, offering solutions to mobility-impairedindividuals, or in a negative manner by, forhttps://ieai.mcts.tum.de/Technical University of Munichinstance, restricting self-determined, independentdecisions and interventions by drivers. To ensureeffective human agency and clarity over personalresponsibility during the operation of AVs, relevantquestions to be addressed are: To what extent and in which situations shouldhumans be able to override an AV? Through what exact processes can we enhancehuman agency in AVs?Potential solutions:The admissibility of human override should beconditioned on the level of automation (up to level3: at any time; level 4: corresponding to safetymechanisms of an AV, perhaps using a time lag;level 5: not required), as well as on the state andbehavior of the driver (e.g., impaired ability).Furthermore, the exact processes that are neededto enhance human agency are threefold andinclude monitoring drivers (e.g., help driversremain awake through driver availabilityrecognitions systems), training drivers (e.g., on thelimitations and capabilities of AVs) and providingexternal human-machine interfaces (e.g., LEDstrips to convey perception information) (Lütge etal., 2021).(4)Privacy & data governance:AVs will need to collect and process a vast amountof data to ensure proper and safe functioning(Future of Privacy Forum, 2017). Despite the AVsdependence on such data, personal privacy stillshould be respected by, for example, transparentlycommunicating how and what kind of data iscollected and governed or by explicitly requestingaffirmative consent from the driver. In this regard,relevant questions to be addressed are: What types of data inside and outside the AVneed to and should be collected? Under which circumstances and in which formatcan valuable data be shared with third parties?IEAI Research Brief4
Institute for Ethics in Artificial IntelligencePotential solutions:First, products or services that collect and sharedata such as AVs should comply with pertinentdata protection standards and regulationsincluding the GDPR, the ePrivacy directive forinformation access on the terminal equipment of auser (EDPB, 2020; European Commission,2020b). In addition, manufactures of AVs shouldfollow a strict privacy and data governance policies(Future of Privacy Forum, 2017) that prescribetransparent communication to drivers about datacollection and usage, demand affirmative andexplicit consent before sensitive data is collected,and allow only limited and anonymous sharing ofvehicle data with third parties (includinggovernments) (Lütge et al., 2021).(5)Responsibility, liability &accountability:In case of an accident where an AV is involved, thevehicle itself cannot be held morally accountablefor the outcomes (Gogoll & Müller, 2017).Responsibility will rather be distributed between avarious amount of involved parties such asmanufacturers, component suppliers, technologycompanies, infrastructure providers or car holdersand drivers. To identify the true cause of anaccident and subsequently the responsible partyduring an investigation, explicit measures oftransparency need to be implemented beforehand.In this regard, relevant questions to be addressedare: In what way do we need to change regulationson product liability for AVs? To what extent should AVs comply with trafficlaws? What are explicit measures of transparency thatallow retrospective investigation of the truecause of an accident where an AV wasinvolved?Potential solutions:As mentioned earlier, due to the increasinginvolvement of various parties during thedevelopment and operation of AVs, regulations on(product) liability need to be reviewed and adapted(European Commission, 2018). For example, onecould argue that liability should be determined bythe driver’s level of autonomy and solo action(Lütge et al., 2021). To test such differentregulatory approaches in a controlled manner,regulators could introduce Law Labs (Joaquinhttps://ieai.mcts.tum.de/Technical University of MunichAcosta, 2018), similar to regulatory sandboxes.Lastly, applicable measures of transparency couldbe to prescribe storing records and data of theunderlying system logic (e.g., used training datasets) (European Commission, 2020b) andimplementing logging mechanisms and blackboxes into AVs (e.g., event data recorder) (Lütge,2017).To ensure that AVs are programmed andfunction in a non-discriminatory manner,the systems need to be trained and testedfor unfair bias(6)Non-discrimination & inclusiveness:Past studies have shown that implicit biases anddiscrimination may unintentionally be incorporatedinto algorithms (e.g., Goddard et al., 2015). Forexample, some AI object detection systems areless likely to detect pedestrians with darker skincolor compared to those with lighter skin (Wilson,Hoffman & Morgenstern, 2019), which mayinfluence the occurrence and distribution offatalities between individuals of different ethnicity.To ensure that AVs are programmed and functionin a non-discriminatory manner, the systems needto be trained and tested for unfair bias. In addition,AVs should exhibit a non-discriminatory design,meaning that they are equally usable for andaccessible to all individuals (Lütge et al., 2021). Inthis regard, relevant questions to be answered are: How can companies ensure and test that biasesare not incorporated into the systems of theirAVs and that certain fairness standards aremet? What exact features need to be included in thedesign of AVs to allow accessibility andinclusiveness to all individuals?Potential solutions:To eliminate biases during the creation ofalgorithms, companies should test their vehicle’sAI system for unfair performance differencesacross personal characteristics such as skin tone,gender and age (Lütge et al., 2021). In doing so,companies can rely on existing standards such asIEEE P7003 that provides protocols to developersand highlights key criteria for selecting validationdata sets (IEEE, 2019). To ensure the possibility ofwide-scale adoption and inclusiveness, companiesIEAI Research Brief5
Institute for Ethics in Artificial Intelligenceneed to demonstrate plans and actions that showhow their AVs can be customized to differingabilities and needs (e.g., possibility to include rampfor entering via a wheelchair) (Lütge et al., 2021).Technical University of MunichFurthermore, since AVs will be gradually rolled outonto streets, the co-existence of conventionalvehicles and AVs will be inevitable. Therefore, it isnecessary to adapt the physical and digitalinfrastructure simultaneously to allow mixedvehicle traffic flows (Lütge et al., 2021).Several programs, such as the Inframix project,work on designing and testing physical and digitalelements (e.g., novel visuals signs or electronicsignals) that may be relevant for the roadinfrastructure of mixed vehicle flows (Inframix,2020). Such research efforts will be key to preparefor the introduction of AVs without jeopardizingsafety and efficiency of the road network. The useof AVs as shared mobility and in connection withelectromobility also has great potential to positivelyinfluence environment in the long term.(7)Societal & environmental wellbeing:In line with the United Nation’s SustainableDevelopment Goals (United Nations, 2015), AVshave great potential to bring forward societal andenvironmental benefits such as increased mobility,better traffic flow, less congestion and decreasedcarbon emission. On the other hand, as AVs willmake driving more convenient and easy forindividuals, it is also likely that per vehicle-miletraveled will increase, potentially leading to greatertotal pollution and congestion (Geary & Danks,2019). Therefore, if not managed properly orwithout according policies in place, inefficienciesand counterproductive effects may arise. In thisregard, relevant questions to be answered are: How can AVs be deployed to increase societaland environmental benefits? How can autonomous vehicles be safelyintegrated into mixed traffic with human drivers? How should the appropriate infrastructure bedeveloped accordingly?Potential solutions:To achieve net benefits, the rationale behindintroducing AVs should be to enhance mobility(e.g., though increased options offered in publictransport) without promoting an increase in overallroad traffic that could arise, for example, fromprivate drivers. To moderate demand andincentivize more socially and environmentallyoptimal travel choices, for instance, theimplementation of congestion pricing schemes orroad tolls has been proposed (Simoni et al., 2019).https://ieai.mcts.tum.de/Final ThoughtsIn this research brief, we highlighted somepressing questions that relate to important ethicalconsiderations in the field of autonomous driving.Certainly, incompatibilities and tradeoffs betweenthese ethical considerations can emerge. Forexample, AVs may meet the principle ofinclusiveness by offering greater mobility for allindividuals but, at the same time, AVs maydecrease environmental wellbeing if the amount ofoverall travel and congestion rises as a result ofbetter accessibility and convenience. AI can play amajor role in mitigating some of these tradeoffs.For example, by rapidly processing huge amountsof data, AI can replace complex transport systemsproblems (e.g., traffic congestion or overcrowding)with smart traffic (Voda & Radu, 2018).However, what becomes evident from thisargument is that vast amounts of data will benecessary for bypassing inefficiencies. This drawsattention to another important tradeoff, namely thatAVs may meet the principles of technical safety,responsible balancing of risks and accountability,but this may come at a cost of needing increasedaccess to and disclosure of personal data (such asthe vehicle’s position). In the future, industry,policymakers, researchers in the automotivesector will need to focus on the above-identifiedissues, develop an agreement on derations, as well as advance relevantsolutions.IEAI Research Brief6
Institute for Ethics in Artificial IntelligenceReferencesBartneck, C., Lütge, C., Wagner, A., & Welsh, S. (2019). Ethik inKI und Robotik. Carl Hanser Verlag GmbH Co KG.Bonnefon, J. F., Shariff, A., & Rahwan, I. (2019). The trolley, thebull bar, and why engineers should care about the ethicsof autonomous cars. Proceedings of the IEEE, 107(3),502-504.ERSO, “Annual Accident Report 2018, European Road SafetyObservatory“, pp. 1–86, 2018.Retrieved fromhttps://ec.europa.eu/transport/road a/asr2018.pdfEuropean Commission (2018). Liability for emerging ropa.eu/legalcontent/EN/TXT/PDF/?uri CELEX:52018SC0137&from enEuropean Commission (2020a). Ethics of connected a.eu/info/sites/info/files/research and innovation/ethics of connected and automated vehicles report.pdfEuropean Data Protection Board (EDPB) (2020). Guidelines1/2020 on processing personal data in the context ofconnected vehicles and mobility related ites/edpb/files/consultation/edpbguidelines 202001 connectedvehicles.pdfFuture of Privacy Forum (2017). Data and the connected /2017/06/2017 eary, T., & Danks, D. (2019). Balancing the benefits ofautonomous vehicles. In Proceedings of the 2019AAAI/ACM Conference on AI, Ethics, and Society (pp. 181186). New York, NY: Association for ComputingMachinery.Geißlinger, M., Poszler, F., Betz, J., Lütge, C., & Lienkamp, M.(2021). Autonomous driving ethics: From Trolley problemto ethics of risk. Working paper.Goddard, T., Kahn, K. B., & Adkins, A. (2015). Racial bias in driveryielding behavior at crosswalks. Transportation researchpart F: traffic psychology and behaviour, 33, 1-6.Gogoll, J., & Müller, J. F. (2017). Autonomous cars: In favor of amandatory ethics setting. Science and engineering ethics,23(3), 681-700.IEEE (2019). Ethically aligned design – A vision for prioritizinghuman well-being with autonomous and ocuments/other/ead1e.pdfInframix (2020). Expected impact – a step by step introduction expected-impact/https://ieai.mcts.tum.de/Technical University of MunichJoaquin Acosta, A. (2018). Autonomous vehicles: 3 practicaltools to help regulators develop better laws andpolicies. Berkman Klein Center for Internet andSociety at Harvard University. Retrieved 2018-07/201807 AVs04 1.pdfLütge, C. (2017). The German ethics code for automated andconnected driving. Philosophy & Technology, 30(4),547-558.Lütge, C., Poszler, F., Acosta, A. J., Danks, D., Gottehrer, G.,Mihet-Popa, L., & Naseer, A. (2021). AI4People: EthicalGuidelines for the Automotive endations. InternationalJournalofTechnoethics, 12(1), 101-125.Robinson, J. (2014). Would You Kill the Fat Man? The TrolleyProblem and What Your Answer Tells Us about Rightand Wrong.SAE International (2018). J3016 – Taxonomy and definitionsfor terms related to driving automation systems for s.sae.org/content/J3016 201806SAE International – Vehicle Cybersecurity SystemsEngineering Committee (2016). SAE J3061:Cybersecurity guidebook for cyber-physical andards/content/j3061 201601/Simoni, M. D., Kockelman, K. M., Gurumurthy, K. M., &Bischoff, J. (2019). Congestion pricing in a world of selfdriving vehicles: An analysis of different strategies inalternative future scenarios. Transportation ResearchPart C: Emerging Technologies, 98, 167-185.Smith, B. 2013. “Human Error as a Cause of Vehicle mson, J. J. (1984). The trolley problem. Yale LJ, 94, 1395.United Nations (2015). Transforming our world: The 2030agenda for sustainable development. Retrieved fromhttps://www.un.org/ga/search/view doc.asp?symbol A/RES/70/1&Lang EVoda, A. I., & Radu, L. D. (2018). Artificial intelligence and thefuture of smart cities. BRAIN. Broad Research inArtificial Intelligence and Neuroscience, 9(2), 110-127.Waymo (2020). Waymo is opening its fully driverless serviceto the general public in Phoenix Retrieved gits-fully-driverless.htmlWadud, Z. & Huda, F. (2021) Fully automated vehicles: the useof travel time and its association with intention to use.Proceedings of the Institution of Civil Engineers Transport 0 0:0, 1-15Wilson, B., Hoffman, J., & Morgenstern, J. (2019). Predictiveinequity in object detection. arXiv preprintarXiv:1902.11097.IEAI Research Brief7
advancing autonomous driving technology with a major goal of increasing road safety and comfort of motorized transport. Autonomous vehicles (AVs) are expected to have a global impact that will change society, the safety of roadways and transportation systems in the future. The field is moving quickly, leaving governance
Page 2 Autonomous Systems Working Group Charter n Autonomous systems are here today.How do we envision autonomous systems of the future? n Our purpose is to explore the 'what ifs' of future autonomous systems'. - 10 years from now what are the emerging applications / autonomous platform of interest? - What are common needs/requirements across different autonomous
autonomous driving solutions is a highly valuable skill in today's software engineering field. Robot Operating System (ROS) is a meta-operating system that simplifies the process of robotics programming. This master's thesis aims to demonstrate how ROS could be used to develop autonomous driving software by analysing autonomous driving
Audi’s initiative &Audi aims to contribute to the introduction of autonomous driving in the inter-ests of individuals and of society. The initiative’s approach is to tackle the challenges along the way to autonomous driving together with various social stakeholders. Since 2015, Audi ha
improvement opportunities for autonomous driving technologies. In particular, more than half road fatalities occur among pedestrians, cyclists and motorcyclists. Controlling for variables such as fatalities caused by car accidents could further show the potential of autonomous driving features.
9th Vector Congress Simon Fürst Scalable Architecture for Autonomous Driving 21 -Nov-2018. PAST FUTURE. Within just a few years only two operating systems for smartphones got . established in the market: Android & iOS. All others died out! Just as in the smartphone operating system market , only a few Autonomous Driving Platforms will .
PWC Driving Licence In NSW it is compulsory for every person driving a PWC to hold a current PWC driving licence. There are two types of PWC driving licence: 1. PWC driving licence for those aged 16 years and over. 2. Young Adult PWC driving licence for people aged from 12 to less than 16 years. A Young Adult PWC driving licence
Eating while driving and talking on a hand-held cell phone while driving are the two most common distracted driving behaviors. 15 %Yes-No Total Eaten food while driving 46%-53% Talked on a hand-held phone while driving 32%-67% Dealt with your car's navigation system while driving 23%-75% Texted or answered a text on your hand-held cell phone .
Autonomous Differential Equations 1. A differential equation of the form y0 F(y) is autonomous. 2. That is, if the right side does not depend on x, the equation is autonomous. 3. Autonomous equations are separable, but ugly integrals and expressions that cannot be solved for y make qualitative analysis sensible. 4.
