Autonomous Mobile Robots - Columbia University
ASLAutonomous Systems LabLocalization Introduction to Map-Based LocalizationAutonomous Mobile RobotsRoland SiegwartMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide ScaramuzzaAutonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland SiegwartLocalization Introduction to Map-Based Localization 1
ASLAutonomous Systems LabIntroduction probabilistic map-based localizationknowledge,data basemissioncommands“position“global mapCognitionPath Planningenvironment modellocal mappathInformationExtractionPathExecutionraw on ControlPerceptionLocalizationMap BuildingReal WorldEnvironmentAutonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland SiegwartLocalization Introduction to Map-Based Localization 2
ASLAutonomous Systems LabLocalization definition, challenges and approach Map-based localization The robot estimates its position using perceived information and a map The map might be known (localization) Might be built in parallel (simultaneous localization and mapping – SLAM)Where am I? Challenges Measurements and the map are inherently error prone Thus the robot has to deal with uncertain information Probabilistic map-base localization Approach The robot estimates the belief state about its positionthrough an ACT and SEE cycleAutonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland SiegwartLocalization Introduction to Map-Based Localization 3
Robot Localization: Historical Context Initially, roboticists thought the world could be modeled exactly Path planning and control assumed perfect, exact, deterministic world Reactive robotics (behavior based, ala bug algorithms) were developeddue to imperfect world models But Reactive robotics assumes accurate control and sensing to react –also not realistic Reality: imperfect world models, imperfect control, imperfect sensing Solution: Probabilistic approach, incorporating model, sensor andcontrol uncertainties into localization and planning Reality: these methods work empirically!
Requirements of a Map Representation for a Mobile Robot The precision of the map needs to match the precision with which therobot needs to achieve its goals The precision and type of features mapped must matcht he precisionof the robot’s sensors The complexity of the map has direct impact on computationalcomplexity for localization, navigation and map updating
3Map RepresentationContinuous Line-Baseda) Architecture mapb) Representation with set of finite or infinite linesLocalization IIZürich
4Map RepresentationExact cell decomposition Exact cell decomposition - PolygonsLocalization IIZürich
5Map RepresentationApproximate cell decomposition Fixed cell decomposition Narrow passages disappearLocalization IIZürich
8Map RepresentationTopological map A topological map represents the environment as agraph with nodes and edges. Nodes correspond to spaces Edge correspond to physical connections between nodes Topological maps lack scale anddistances, but topologicalrelationships (e.g., left, right, etc.)are ion IIZürich
9Map RepresentationTopological map London underground mapLocalization IIZürich
ASLAutonomous Systems LabConcept SEE and ACT to improve belief state Robot is placed somewhere in theenvironment location unknown SEE: The robot queries its sensors finds itself next to a pillarAutonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland Siegwart ACT: Robot moves one meter forward motion estimated by wheel encoders accumulation of uncertainty SEE: The robot queries its sensors again finds itself next to a pillar Belief updates (information fusion)Localization Introduction to Map-Based Localization 4
ASLAutonomous Systems LabConcept SEE and ACT to improve belief state Robot is placed somewhere in theenvironment location unknown SEE: The robot queries its sensors finds itself next to a pillar ACT: Robot moves one meter forward motion estimated by wheel encoders accumulation of uncertainty SEE: The robot queries its sensors again finds itself next to a pillar Belief updates (information fusion)SEEAutonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland SiegwartLocalization Introduction to Map-Based Localization 5
ASLAutonomous Systems LabConcept SEE and ACT to improve belief state Robot is placed somewhere in theenvironment location unknown SEE: The robot queries its sensors finds itself next to a pillar ACT: Robot moves one meter forward motion estimated by wheel encoders accumulation of uncertainty SEE: The robot queries its sensors again finds itself next to a pillar Belief updates (information fusion)SEEAutonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland SiegwartLocalization Introduction to Map-Based Localization 6
ASLAutonomous Systems LabConcept SEE and ACT to improve belief state Robot is placed somewhere in theenvironment location unknown SEE: The robot queries its sensors finds itself next to a pillar ACT: Robot moves one meter forward motion estimated by wheel encoders accumulation of uncertainty SEE: The robot queries its sensors again finds itself next to a pillar Belief updates (information fusion)ACTAutonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland SiegwartLocalization Introduction to Map-Based Localization 7
ASLAutonomous Systems LabConcept SEE and ACT to improve belief state Robot is placed somewhere in theenvironment location unknown SEE: The robot queries its sensors finds itself next to a pillar ACT: Robot moves one meter forward motion estimated by wheel encoders accumulation of uncertainty SEE: The robot queries its sensors again finds itself next to a pillar Belief updates (information fusion)ACTAutonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland SiegwartLocalization Introduction to Map-Based Localization 8
ASLAutonomous Systems LabConcept SEE and ACT to improve belief state Robot is placed somewhere in theenvironment location unknown SEE: The robot queries its sensors finds itself next to a pillar ACT: Robot moves one meter forward motion estimated by wheel encoders accumulation of uncertainty SEE: The robot queries its sensors again finds itself next to a pillar Belief updates (information fusion)SEEAutonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland SiegwartLocalization Introduction to Map-Based Localization 9
ASLAutonomous Systems LabConcept SEE and ACT to improve belief state Robot is placed somewhere in theenvironment location unknown SEE: The robot queries its sensors finds itself next to a pillarAutonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland Siegwart ACT: Robot moves one meter forward motion estimated by wheel encoders accumulation of uncertainty SEE: The robot queries its sensors again finds itself next to a pillar Belief update (information fusion)Localization Introduction to Map-Based Localization 10
ASLAutonomous Systems LabACT using motion model and its uncertainties The robot moves and estimates its position through its proprioceptive sensors Wheel Encoder (Odometry) During this step, the robot’s state uncertainty growsAutonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland SiegwartLocalization Introduction to Map-Based Localization 11
ASLAutonomous Systems LabSEE estimation of position based on perception and map The robot makes an observation using its exteroceptive sensors This results in a second estimation of the current positionProbability ofmaking thisobservationRobot’s belief beforethe observationSEESEE′Autonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland SiegwartLocalization Introduction to Map-Based Localization 12
ASLAutonomous Systems LabBelief update fusion of prior belief with observation The robot corrects its position by combining its belief before the observationwith the probability of making exactly that observation During this step, the robot’s state uncertainty shrinksRobot’s beliefupdateProbability ofmaking thisobservationRobot’s belief beforethe observation′′Autonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland Siegwart′Localization Introduction to Map-Based Localization 13
ASLAutonomous Systems LabProbabilistic localization belief representationKalman FilterLocalizationa) Continuous map withsingle hypothesis probability distributionb) Continuous map withmultiple hypotheses probability distributionMarkov Localizationc) Discretized metric map (grid ) withprobability distributiond) Discretized topological map (nodes ) withprobability distributionAAutonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland SiegwartBCDEFGLocalization Introduction to Map-Based Localization 15
ASLAutonomous Systems LabTake home message ACT - SEE Cycle for Localization SEE: The robot queries its sensors finds itself next to a pillar ACT: Robot moves one meter forward motion estimated by wheel encoders accumulation of uncertainty SEE: The robot queries its sensorsagain finds itself next to a pillar Belief update (information fusion)Autonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland SiegwartLocalization Introduction to Map-Based Localization 16
ASLAutonomous Systems LabProbability theory how to deal with uncertainty Mobile robot localization has to deal with error prone information Mathematically, error prone information (uncertainties) is best represented byrandom variables and probability theory : probability that the random variablehas value( is true). : random variable : a specific value that might assume. The Probability Density Functions (PDF) describesthe relative likelihood for a random variable to take ona given value PDF example: The Gaussian distribution:12Autonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland SiegwartLocalization Refresher on Probability Theory 2
ASLAutonomous Systems LabBasic concepts of probability theory joint distribution , : joint distribution representing the probability that the random variabletakes on the value and that takes on the value and is true. Ifandare independent we can write:,Autonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland SiegwartLocalization Refresher on Probability Theory 3
ASLAutonomous Systems LabBasic concepts of probability theory conditional probability : conditional probability that describes the probability that the randomvariable takes on the value conditioned on the knowledge that for suretakes .,and ifandare independent (uncorrelated) we can write:Autonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland SiegwartLocalization Refresher on Probability Theory 4
ASLAutonomous Systems LabBasic concepts of probability theory theorem of total probability The theorem of total probability (convolution) originates from the axioms ofprobability theory and is written as:for discrete probabilitiesfor continuous probabilities This theorem is used by both Markov and Kalman-filter localization algorithmsduring the prediction update.Autonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland SiegwartLocalization Refresher on Probability Theory 5
ASLAutonomous Systems LabBasic concepts of probability theory the Bayes rule The Bayes rule relates the conditional probabilityto its inverse Under the condition that0, the Bayes rule is written as: normalization factor (.1 This theorem is used by both Markov and Kalman-filter localization algorithmsduring the measurement update.Autonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland SiegwartLocalization Refresher on Probability Theory 6
ASLAutonomous Systems LabUsage application of probability theory to robot localization Probability theory is widely and very successfully used for mobile robotlocalization In the following lecture segments, its application to localization will beillustration Markov localization Discretized pose representation Kalman filter Continuous pose representation and Gaussian error model Further reading: “Probabilistic Robotics,” Thrun, Fox, Burgard, MIT Press, 2005.“Introduction to Autonomous Mobile Robots”, Siegwart, Nourbakhsh, Scaramuzza, MIT Press 2011Autonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland SiegwartLocalization Refresher on Probability Theory 7
ASLAutonomous Systems LabMarkov localization applying probability theory to localizationpositionPosition Update(estimation/fusion)Encoder(e.g. odometry)ACT: Motionpredictedposition(motors)predictedposition Information (measurements)is error prone (uncertain)Map(data base) Odometry Exteroceptive sensors (camera, laser, ) Map Probabilistic map-based localizationAutonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland tchingmeasuredobservations(sensor data / features)SEE: Perception(Camera, Laser, )Localization the Markov Approach 2
ASLAutonomous Systems LabMarkov localization basics and assumption Discretized pose representation grid mapusing an arbitrary Markov localization tracks the robot’s belief stateprobability density function to represent the robot’s position Markov assumption: Formally, this means that the output of the estimationprocess is a function only of the robot’s previous stateand its mostrecent actions (odometry)and perception ., , ,, Markov localization addresses the global localization problem, the positiontracking problem, and the kidnapped robot problem.Autonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland SiegwartLocalization the Markov Approach 3
ASLAutonomous Systems LabMarkov localization applying probability theory to localization ACT probabilistic estimation of the robot’s new belief statethe previous locationand the probabilistic motion model,with action(control input).based on application of theorem of total probability / convolutionfor continuous probabilities,,Autonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland Siegwartfor discrete probabilitiesLocalization the Markov Approach 54
ASLAutonomous Systems LabMarkov localization applying probability theory to localization SEE probabilistic estimation of the robot’s new belief statefunction of its measurement data and its former belief stateas a: application of Bayes rule ,where,is the probabilistic measurement model (SEE), that is, theprobability of observing the measurement data given the knowledge of the mapand the robot’s position . Thereby is the normalization factor sothat 1.Autonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland SiegwartLocalization the Markov Approach 55
ASLAutonomous Systems LabMarkov localization the basic algorithms for Markov localizationFor alldo ,,(prediction update)(measurement update)endforReturn Markov assumption: Formally, this means that the output is a function onlyof the robot’s previous state and its most recent actions (odometry)andperception .Autonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland SiegwartLocalization the Markov Approach 56
ASLAutonomous Systems LabACT using motion model and its uncertaintiesprior belief0.750.50.2512345678910 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32uncertain ion update0.510 11 12 13 14 15 16convolution,0.2512345678910 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32Autonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland SiegwartLocalization the Markov Approach 7
ASLAutonomous Systems LabACT using motion model and its uncertaintiesprior belief0.750.50.2512345678910 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32uncertain motion(odometry)ACT0.750.50.25120.75345678910 11 12 13 14 15 16prediction update0.5,0.2512345678910 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32Autonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland SiegwartLocalization the Markov Approach 8
ASLAutonomous Systems LabSEE estimation of position based on perception and map0.75pillaris atunit 35prediction update0.50.2512345678910 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32,0.75Mapperception0.5SEESEE0.251measurement update0.752345678910 11 12 13 14 15 16 17 18Multiplication and normalization 0.5 0.251234567,8910 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32Autonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland SiegwartLocalization the Markov Approach 9
Mobile Robot Localization31bel x 0 p x 1 u 1 x 0 (a)(b)ACT: motion probability.0.5 probability the robot moves 2 units,0.5 probability the robot moves 3 unitsbel x 1 (c)p z 1 x 1 M (d)bel x 1 initial belief of robot's position(e)After applying the ACT probability of motionto the current belief state in (a) we computeupdated belief in (c). Note uncertainty increasesSEE: a range sensor on the robot measuresthe robot's distance from the origin. Thesensor has equal probability of measuring the robotas 5 or 6 units from the origin (0.5 probability each.This is the sensor error model.The robot corrects its position by combining itsbelief before the observation with the probabilityof that observation using Bayes rule. This reducesthe uncertainty.Note we need to use a scaling factor to make sureall probabilities add up to 1Figure 5.23 Markov localization using a grid-map.Calculation of the robot's position after the ACT move in (a),(b) above:p x1 2 p x 0 0 p u 1 2 0.125 (5.44p x1 3 p x 0 0 p u 1 3 p x0 1 p u 1 2 0.25(5.45)p x1 4 p x 0 1 p u 1 3 p x0 2 p u 1 2 0.25(5.46)p x1 5 p x 0 2 p u 1 3 p x0 3 p u 1 2 0.25(5.47)p x1 6 p x 0 3 p u 1 3 0.125(5.48)
24Markov localization Let us discretize the configuration space into 10 cells Suppose that the robot’s initial belief is a uniform distribution from 0 to 3. Observe that all theelements were normalized so that their sum is 1.Localization IIZürich
25Markov localization Initial belief distribution Action phase:Let us assume that the robot moves forward with the following statistical model This means that we have 50% probability that the robot moved 2 or 3 cells forward. Considering what the probability was before moving, what will the probability be after the motion?Localization IIZürich
26Markov localizationAction update The solution is given by the convolution (cross correlation) of the two distributions,*Localization II , Zürich
28Markov localizationPerception update Let us now assume that the robot uses its onboard range finder and measures the distancefrom the origin. Assume that the statistical error model of the sensors is:This plot tells us that the distance of the robot from the origin can be equally 5 or 6 units. What will the final robot belief be after this measurement?The answer is again given by the Bayes rule: , Localization II Zürich
Markov Localization Example, p. 313 Siegwart1INITIAL BELIEF: Bel(X) at time tGRID CELL2Now move the robot with probabilities below:3MOTION PROBABILITY: U(t) -robot moves 2 or 3 unitsGRID CELL4Now CONVOLVE Bel(X) with U(t)5UPDATED BELIEF: Bel(X)GRID CELL6Now use sensor to update your Bel(X)7SENSOR Probabilities: Z(t) - origin is 5 or 6 units awayGRID CELL8Apply sensor measurement to current Bel(X)9UNNORMALIZED SENSOR UPDATEGRID CELL10NORMALIZATION .0625 0.125 0.187511NORMALIZED SENSOR UPDATE: Bel(X) at t 1GRID 10203040.550.5607080900010203040708090.125 0.0625560.125 / 0.1875 .667 , 0.0625/ 0.1875 .3300010203040.6667 0.333356070809
ASLAutonomous Systems LabMarkov localization extension to 2D The real world for mobile robot is at least 2D (moving in the plane) discretized pose state space (grid) consists of , , Markov Localization scales badly with the size of the environment Space: 10 m x 10 m with a grid size of 0.1 mand an angular resolution of 1 100 100 360 3.6 10 grid points (states) prediction step requires in worst case3.6 10multiplications and summations Fine fixed decomposition grids result in a huge state space Very important processing power needed Large memory requirementAutonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland SiegwartLocalization the Markov Approach 10
ASLAutonomous Systems LabMarkov localization reducing computational complexity Adaptive cell decomposition Motion model (Odomety) limited to a smallnumber of grid points Randomized sampling Approximation of belief state by a representative subsetof possible locations weighting the sampling process with the probabilityvalues Injection of some randomized (not weighted) samples randomized sampling methods are also known asparticle filter algorithms, condensation algorithms, andMonte Carlo algorithms.Autonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland SiegwartLocalization the Markov Approach 11
ASLAutonomous Systems LabKalman Filter Localization Basics and assumption Continuous pose representation Kalman Filter Assumptions: Error approximation with normal distribution:,(Gaussian model) Output distribution is a linear (or linearized)function of the input distribution: Kalman filter localization tracks the robot’sbelief statetypically as a singlehypothesis with normal distribution. Kalman localization thus addresses theposition tracking problem, but not theglobal localization or the kidnapped robotproblem.Autonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland SiegwartLocalization the Kalman Filter Approach 3
ASLAutonomous Systems LabKalman Filter Localization in summery1.2.3.4.5.Prediction (ACT) based on previous estimate and odometryObservation (SEE) with on-board sensorsMeasurement prediction based on prediction and mapMatching of observation and mapEstimation position update (posteriori position)Estimation:Robot’s beliefupdatePrediction:Robot’s beliefbefore theobservationAutonomous Mobile RobotsMargarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland SiegwartObservation:Probability ofmaking thisobservationLocalization the Kalman Filter Approach 11
Autonomous Mobile Robots Margarita Chli, Paul Furgale, Marco Hutter, Martin Rufli, Davide Scaramuzza, Roland Siegwart ASL Autonomous Systems Lab localization “Introduction to Autonomous Mobile Robots
50 robots 100 robots 200 robots 1 robot Foraging Performance Over Time - Random Movement with Clustered Forage Items 0 10 20 30 40 50 60 70 80 90 100 Increasing Time % Completion 5 robots 10 robots 25 robots 50 robots 100 robots 200 robots 1 robot. COMP 4900A - Fall 2006 Chapter 11 - Multi-Robot Coordination 11-18
Introduction to Introduction to Autonomous Mobile Robots Roland Siegwart and Illah R. Nourbakhsh Mobile robots range from the teleoperated Sojourner on the Mars Pathfinder mission to cleaning robots in the Paris Metro. Introduction to Autonomous Mobile Robots offers
11) MEDICAL AND REHAB Medical and health-care robots include systems such as the da Vinci surgical robot and bionic prostheses TYPES OF MEDICAL AND HEALTHCARE ROBOTS Surgical robots Rehabilitation robots Bio-robots Telepresence robots Pharmacy automation Disinfection robots OPERATION ALERT !! The next slide shows
ment of various robots to replace human tasks [2]. There have also been many studies related to robots in the medical field. These in - clude surgical robots, rehabilitation robots, nursing assistant ro - bots, and hospital logistics robots [3]. Among these robots, surgi - cal robots have been actively used [4]. However, with the excep-
Introduction Model Learning on a Sony Aibo Model Learning on an Autonomous Car Conclusions Model Learning for Autonomous Robots Goal: To increase the effectiveness of autonomous mobile robots Plan: Enable mobile robots t
Autonomous Mobile Robots: Simultaneous Localization and Mapping Dr. Hamid D. Taghirad Advanced Robotics and Automated Systems (ARAS) K.N. Toosi University of Technology . Introduction SLAM Problem Autonomous Mobile Robots The ultimate goal of
Introduction to autonomous mobile robots. - 2nd ed. / Roland Siegwart, Illah R. Nourbakhsh, and Da-vide Scaramuzza. p. cm. - (Intelligent robotics and autonomous agents series) Includes bibliographical references and index. ISBN 978-0-262-01535-6 (hardcover : alk. paper) 1. Mobile robots. 2. Autonomo
ASTM C 76 specification for reinforced sewer and storm drain pipes. (Only on special requests) inTernaTional & local approVals ISO 9001 registered firm. M.P.W Approval QualiTy assurance Concrete pipe factory has an independent quality control department with a well equipped laboratory, aided by experienced staff to give the necessary backup for the quality assurance program .
