Defect Prevention And Detection In Software For Automated .
Defect Prevention and Detection in Software forAutomated Test EquipmentMaster's Project ReportEric Beanebean@byu.netRev: 11/28/2006This project report is submitted to the Department of ElectricalEngineering and Computer Science and the Faculty of the GraduateSchool of the University of Kansas in partial fulfillment of therequirements for the degree of Master of ScienceAbstractSoftware for automated test equipment can be tedious and monotonous making it justas error-prone as other software. Active defect prevention and detection are alsoimportant for test applications. Incomplete or unclear requirements, a cryptic syntaxused for some test applications—especially script-based test sets, variability in syntaxor structure, and changing requirements are among the problems encountered in onetester. Such problems are common to all software but can be particularly problematicin test equipment software intended to test another product. Each of these issuesincreases the probability of error injection during test application development. Thisreport describes a test application development tool designed to address these issuesand others for a particular piece of test equipment. By addressing these problems inthe development environment, the tool has powerful built-in defect prevention anddetection capabilities. Regular expressions are widely used in the development tool asa means of formally defining test equipment requirements for the test application andverifying conformance to those requirements. A novel means of using regularexpressions to perform range checking was developed. A reduction in rework andincreased productivity are the results. These capabilities are described along withlessons learned and their applicability to other test equipment software. The testapplication development tool, or “application builder”, is known as the PT3800 AMCreation, Revision and Archiving Tool (PACRAT).Keywords: Defect Prevention, Defect Detection, Mistake Proofing, Regular Expressions,Automated Test Equipment, Test Application Builder
Table of Contents1INTRODUCTION.51.11.22APPLICATION BUILDER OBJECTIVES .72.12.22.33DEFECT PREVENTION .7DEFECT DETECTION .8OTHER OBJECTIVES .8ACHIEVEMENT OF OBJECTIVES .93.13.23.34BACKGROUND .6MOTIVATION FOR PACRAT .6DEFECT PREVENTION OBJECTIVES .93.1.1 Build tester requirements into tool through the use of an applicationdatabase.93.1.2 Simplify and automate complex tasks .113.1.3 Reduce or eliminate syntax variability .143.1.4 Eliminate file structure variability.143.1.5 Encourage the use of the development process .14DEFECT DETECTION OBJECTIVES .153.2.1 Test Application Data Validation.153.2.2 Test Application Source Code Analysis.17OTHER OBJECTIVES .183.3.1 Maintainability .183.3.2 Platform for Future Expansion .183.3.3 Increased Productivity.18LESSONS LEARNED . 194.14.24.34.44.54.6WELL-DEFINED REQUIREMENTS .20KEEP IT SIMPLE .21REDUCE VARIABILITY .21ENCOURAGE USE OF DEVELOPMENT PROCESS .21VERIFICATION OF CONFORMANCE TO REQUIREMENTS .21MAINTAINABILITY AND FLEXIBILITY .225CONCLUSION . 226REFERENCES . 22APPENDIX AA.1A.2A.3ALGORITHM .25EXAMPLES.26LIMITATIONS AND POSSIBILITIES .28APPENDIX BB.1B.2B.3B.4Eric BeanRANGE CHECKING REGULAR EXPRESSIONS. 25PACRAT IMPLEMENTATION DETAILS. 29OBJECT ORIENTED DESIGN AND GRAPHICAL USER-INTERFACE .29DATABASE OF APPLICATION DATA .33TEST APPLICATION FILE STRUCTURE .36COMPREHENSIVE TEST APPLICATION VALIDATION .37Defect Prevention and Detection in ATE Software2/39
Table of FiguresFIGURE 1FIGURE 2FIGURE 3FIGURE 4SOFTWARE COMPLEXITY VS. DEFECT DENSITY .12HIGH LEVEL SOFTWARE DIAGRAM .29MISCELLANEOUS DIALOGS AND FORMS .31GLOBAL OBJECTS AND UTILITIES .32Table of TablesTABLE 1 EXAMPLES OF PARAMETER VALIDATION RULES USING REGULAR EXPRESSIONS .16TABLE 2 TEST APPLICATION COMPLEXITY .19TABLE 3 TEST APPLICATION DEVELOPMENT TIME (IN HOURS) .19Eric BeanDefect Prevention and Detection in ATE Software3/39
AcknowledgementsI would like to acknowledge the guidance of my fellow team members on the PT3800project, Dave Irwin and Doug Gutekunst, for their mentoring and guidance.I also acknowledge the guidance of Professor Hossein Saiedian in completing thisproject and throughout my graduate school experience.Finally, and most importantly, I want express my sincere thanks to my wife, Kammi,and our two children, Kyle (3) and Bryce (1), for their patience and support.Eric BeanDefect Prevention and Detection in ATE Software4/39
1IntroductionAutomated test equipment plays an increasingly important role in the manufacturingworld. This is especially true with the emphasis on lean manufacturing processes.Automated test equipment can be a cost effective means of testing products in thefactory. Most automated test equipment is controlled by software. This software maybe embedded, PC based, or even script-based.Software development is a highly people-oriented activity and therefore, a highlyerror-prone activity. Defects may be injected during requirements analysis orspecification of the test application, test application design, and test applicationdevelopment or coding. Defect removal would typically be done at review points alongthat process and during testing of the test application itself. Software for automatedtest equipment is subject to same defect injection issues as other software.In particular, though, test equipment application development can be tedious andunvaried. Depending on the equipment and the product under test, it may also becomplex. Often, it is domain specific. Typically, though, test applications are made upof similar types of functions involving some stimulus and a measurement. Testapplications must meet both the requirements of the product to be tested and theautomated test equipment. All of this is true whether it is embedded software or ascript. These things make automated test equipment software as error prone as othersoftware both in initial development and in maintenance.As a result, active defect removal and prevention is required to ensure quality. Thequality of the test equipment application may have a significant impact on the testequipment’s success at detecting defects in the product under test.A good software engineering process is designed to build quality into a softwareproduct during development and to remove defects as early as possible. Much ofsoftware engineering research is focused on defect prevention and removal. In mostcases, the focus is on the software development process. Significant results can bemade from following a well-designed, mature software engineering process. The key isfollowing the process and continually improving the process.Some work has been done in the area of automated software error detection tools suchas Lint, though, such tools typically have limited error-checking ability or report falseerrors (Hallem, 2003). Integrated development environments have improved thesoftware development experience and provide lots of tools to make softwaredevelopment easier. But for test equipment applications that are often scripts andsometimes written in proprietary languages with their own domain specific syntax,tools such as these are not as widely available.This project report describes a test application development tool designed with a highdegree of defect prevention and detection built-in. While this tool is specific to aparticular tester, the PT3800, the approach that it uses may be employed for otherautomated test equipment. The PT3800 tester is a refurbishment of an older tester, thePT3300. This refurbishment provided an opportunity to improve not only the testerEric BeanDefect Prevention and Detection in ATE Software5/39
but to improve the test application development experience. It was an opportunity tocreate a test application development tool known as the PT3800 AM Creation,Revision and Archiving Tool (PACRAT). This paper details the built-in defectprevention and detection techniques employed by PACRAT.1.1BackgroundThe PT3800 is a universal tester of basic electrical capabilities. Over 1000 productsmanufactured under a particular government contract have used its predecessor, thePT3300. About 1600 test applications were used with that tester. The PT3800 testerperforms the following basic types of tests: Continuity – measures the voltage drop across two pins when a specifiedcurrent is appliedOhmmeter – measures resistance between positive and negative leadsVoltage – measures the voltage output pins when a voltage is applied betweeninput pins; also measures the current through the negative input leadInsulation Resistance – measures the leakage current through a set of negativepins when a voltage is applied between a set of positive pins and a set ofnegative pinsInductance and Capacitance Tests – measures inductance and capacitance.The test applications used by the PT3300 tester are text files defining the product tobe tested, connections and adapters to the product under test, the tests to run withstimuli and limits to be applied, and the test points. Instructions to the tester operatorregarding how to connect cable adapters to the product under test or to other adaptersare also included in the test application file.The new tester, the PT3800, has been designed to replace the old tester with the samefunctionality. Some enhancements have also been included. It is expected that allelectrical products manufactured under the particular government contract will betested at least once by the new tester before delivery to the customer. Approximately100 new test applications will be needed per year for the foreseeable future. Therefore,there is continued need for maintenance of existing test applications and developmentof new test applications. The PT3800 test applications are required to contain all thesame information as the PT3300 test applications and more, hence the need for arobust development tool.1.2Motivation for PACRATThe PT3300 test applications are nearly free form text with only basic structurerequired. A significant amount of variation is allowed in the syntax. A cumbersomeeditor is used along with a rudimentary syntax checker. Application development isperformed on an HP terminal.The development of the PT3800 tester provided an opportunity to improve the testapplication format and the means of creating it. The following issues specificallyneeded to be addressed in the new PT3800 system: AM refers to “automated media,” specifically test application source code.Eric BeanDefect Prevention and Detection in ATE Software6/39
Incomplete requirements – Not all syntax requirements for test applicationswere defined including the allowance of some undocumented commands.Incomplete or undocumented requirements are not uncommon problems insoftware development. This tool needed to have the test equipment’srequirements for a test application built-in, or at least access to them. Thatmeant that a mechanism for defining and capturing the requirements wasrequired, as well.Syntax and file structure variability – The variability in syntax and filestructure in test applications made the test applications more difficult tomaintain. Furthermore, it led to mistakes that the syntax checker did not alwaysdetect.Complex setup – The setup portion of the test application was complex andtedious. A high degree of complexity or tediousness provides greateropportunity for defect injection. The new tool needed to simplify many of themost complex tasks.Cryptic syntax – A somewhat cryptic syntax was used in the old testapplications. The new development tool needed to make the syntax less of anissue and generally provide an easy-to-use interface.Syntax and data validation – As mention above, the capabilities of the syntaxchecker on the PT3300 system were limited and only provided defect detectioncapabilities. The new application builder needed to have much morecomprehensive syntax and data validation built-in. Further more, mistakeproofing features were required.Evolving requirements – Because so many products are tested on the PT3300,the maintenance needs are high for its test applications. The same is true of thenew PT3800 tester. The system needed to be flexible and expandable to meetthese needs.Ultimately, all these issues result in opportunities for defect injection leading to lowerproductivity. All are common problems of software, in general, and software forautomated test equipment in particular. The PACRAT test application developmenttool addresses these issues by meeting several objectives surrounding defectprevention and defect detection.2Application Builder ObjectivesThe test application builder objectives regarding defect prevention and detection aredescribed in the subsections below. The achievement of these objectives is describedin greater detail in section 3.2.1Defect PreventionOne of Phil Crosby’s absolutes of quality management is that “the system of quality isprevention” (Crosby, 1984). Crosby further stated, “the error that does not existcannot be missed.” He put forth the idea that all defects are caused and that anythingthat is caused can be prevented. In fact, one of the key process areas of level 5“Optimizing” of the Software Capability Maturity Model is defect prevention (Paulk,1993).Eric BeanDefect Prevention and Detection in ATE Software7/39
In the development of software, including software for automated test equipment,defect prevention may involve development process activities such as the following: Planning defect prevention activitiesTracking defectsIdentifying the root causes of defectsSystematically eliminating causes of defectsJoint Application Development with the customerUse of formal methods for requirements analysis and designStructured coding methodsFormal test case constructionPeer reviews throughout the development process (since developers learn toavoid the kind of defects that reviews detect)Continuous process improvementEach of these process activities is important. However, this project sought to go a stepfurther by building into the development tool defect prevention and mistake proofingfeatures based on many of these process practices while addressing the issuesexperienced previously with test application development on the PT3300. The testapplication builder had the following objectives for defect prevention: Build tester requirements into the tool through the use of an applicationdatabaseSimplify and automate complex tasksReduce or eliminate syntax variabilityEliminate file structure variabilityEncourage the use of the development process2.2Defect DetectionStudies have shown that defect correction is less costly when the defect is detectedearly in the process. The longer a defect is allowed to ripple down through a processthe more costly it is to find and correct. Defect detection and removal practices usuallyinvolve one or more of the following: Peer reviews or inspectionsVarious types of software testing (unit, integration, acceptance, etc.)Automated error detection toolsThe goal with this test application builder was to sharply reduce the number of defectescapes during test application development. This meant building active defectdetection into the application builder. This was to be done in two primary ways: Data entry validation against the tester’s requirementsSource code analysis via comprehensive file validation against the tester’srequirements before test application is used2.3Other ObjectivesOther objectives of the test application builder included:Eric BeanDefect Prevention and Detection in ATE Software8/39
Providing for future expansion due to changing requirementsImproved maintainability of test application source codeReduced test application dev
Syntax and file structure variability – The variability in syntax and file structure in test applications made the test applications more difficult to maintain. Furthermore, it led to mistakes that the syntax checker did not always detect. Complex setup – The setup portion of the test application was complex and tedious.
Defect Analysis and Prevention Defect Analysis is the process of analyzing a defect to determine its root cause. Defect Prevention is the process of addressing root
25 Weather station defect Output 1,002 C R T 26 Block Input 1,002 C S 27 Wind sensor 1 defect Output 1.002 C R T 28 Wind sensor 2 defect Output 1.002 C R T 29 Wind sensor 3 defect Output 1.002 C R T 30 Wind sensor 4 defect Output 1.002 C R T 31 Wind direction defect Output 1.002 C R T 32 R
theoretical model and results of the experimental study have been compared to a single defect on the outer ring of bearing. The results of the theoretical model show good agreement with experimental vibration response. 2. Defect Frequencies The defect present in the bearing element tends to increase vibration energy at defect frequency.
3 - Defect detection in uniform thickness skin 1a,1b,1c,3a,3b,3c 4 - Defect detection in tapered skin 1a,1b,1c 5 - Inspection of bonded substructure 1a,1b,1c 6 - Inspection of co-cured substructure 2a,2b 7 - Defect detection around other aircraft elements 2a,2b PA Exercises Panels General C-Scan Inspection Procedure All panels 1 - PA .
Detection of defect/inhomogeneity in dielectric composites. Evaluation of defect size and properties. Production of high-resolution defect images. Detection and evaluation of properties of surface cracks, anomalies & perturbations (impact damage) in metals and graphite composites. Conductor & dielectric sheet surface profiling.
For FMEA, mandate and commitment as required by ISO31000 should be "Defect Prevention" as proposed by the previous case studyThe 2019 AIAG/ VDA FMEA handbook provide more lengthy explanation and example of . objective of FMEA such as defect free launch in section 1.2. However, the defect prevention should be highlighted as
interdental angular bone loss mesial to tooth#21 with a defect depth more than 4mm and a defect angle of 37o (Figure 1b). The treatment plan was to perform open flap debridement and employ combination of PRP and DFDBA matrix to fill the one wall defect. Initial phase-I therapy was done with scaling, root planning and oral hygiene instructions.
Misrun defect is a kind of incomplete casting defect, which causes the casting uncompleted. The edge of defect is round and smooth. When the metal is unable to fill the mould cavity completely and thus leaving unfilled portion called misrun. A cold shunt is called when two metal
