NAVAL POSTGRADUATE SCHOOL - DTIC
NAVALPOSTGRADUATESCHOOLMONTEREY, CALIFORNIATHESISASSESSMENT OF EXTERNAL RELIABILITY DATASOURCES AND RELIABILITY PREDICTIONS OFCOMPLEX SYSTEMS IN EARLY SYSTEM DESIGNbyJohn W. KosempelSeptember 2018Thesis Advisor:Second Reader:Bryan M. O'HalloranAnthony G. PollmanApproved for public release. Distribution is unlimited.
THIS PAGE INTENTIONALLY LEFT BLANK
Form Approved OMBNo. 0704-0188REPORT DOCUMENTATION PAGEPublic reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewinginstruction, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection ofinformation. Send comments regarding this burden estimate or any other aspect of this collection of information, includingsuggestions for reducing this burden, to Washington headquarters Services, Directorate for Information Operations and Reports, 1215Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302, and to the Office of Management and Budget, Paperwork ReductionProject (0704-0188) Washington, DC 20503.1. AGENCY USE ONLY(Leave blank)2. REPORT DATESeptember 20183. REPORT TYPE AND DATES COVEREDMaster's thesis4. TITLE AND SUBTITLEASSESSMENT OF EXTERNAL RELIABILITY DATA SOURCES ANDRELIABILITY PREDICTIONS OF COMPLEX SYSTEMS IN EARLYSYSTEM DESIGN5. FUNDING NUMBERS6. AUTHOR(S) John W. Kosempel7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)Naval Postgraduate SchoolMonterey, CA 93943-50009. SPONSORING / MONITORING AGENCY NAME(S) ANDADDRESS(ES)N/A8. PERFORMINGORGANIZATION REPORTNUMBER10. SPONSORING /MONITORING AGENCYREPORT NUMBER11. SUPPLEMENTARY NOTES The views expressed in this thesis are those of the author and do not reflect theofficial policy or position of the Department of Defense or the U.S. Government.12a. DISTRIBUTION / AVAILABILITY STATEMENTApproved for public release. Distribution is unlimited.12b. DISTRIBUTION CODEA13. ABSTRACT (maximum 200 words)Two common reliability prediction methods are the traditional method and physics of failure method.Each method requires accurate failure data in order to fully assess a system’s durability. This is particularlyimportant in early system design when historical design and relative failure rates are non-existent.Consequently, practitioners rely on the use of external reliability data sources such as MIL-HDBK-217F,especially when using the traditional reliability approach. Several other external reliability data sources areavailable to the practitioner, each with its own strengths and limitations. This thesis surveys the variousexternal data sources industries use in reliability predictions and assesses the completeness of the reliabilitydata sources. The thesis presents the inherent limitations of all external data sources along with furtherconsiderations on using the traditional reliability approach. Early system design offers practitioners asignificant amount of decision-making flexibility. This thesis further analyzes both reliability approachesand addresses when it is appropriate for a practitioner to use either approach or a combination of the twoapproaches. The author develops a reliability decision framework to aid practitioners in selecting thereliability prediction approach appropriate for the system.14. SUBJECT TERMSreliability, external data sources, systems engineering, reliability prediction methods,reliability modeling, complex systems, design for reliability, early system design, physics offailure, reliability decision framework, preliminary design15. NUMBER OFPAGES7117. SECURITYCLASSIFICATION OFREPORTUnclassified20. LIMITATION OFABSTRACT18. SECURITYCLASSIFICATION OF THISPAGEUnclassified19. SECURITYCLASSIFICATION OFABSTRACTUnclassified16. PRICE CODEUUStandard Form 298 (Rev. 2-89)Prescribed by ANSI Std. 239-18NSN 7540-01-280-5500i
THIS PAGE INTENTIONALLY LEFT BLANKii
Approved for public release. Distribution is unlimited.ASSESSMENT OF EXTERNAL RELIABILITY DATA SOURCES ANDRELIABILITY PREDICTIONS OF COMPLEX SYSTEMS IN EARLY SYSTEMDESIGNJohn W. KosempelCivilian, Department of the NavyBSEE, Temple University, 2008MS, Central Michigan University, 2012Submitted in partial fulfillment of therequirements for the degree ofMASTER OF SCIENCE IN SYSTEMS ENGINEERING MANAGEMENTfrom theNAVAL POSTGRADUATE SCHOOLSeptember 2018Approved by:Bryan M. O'HalloranAdvisorAnthony G. PollmanSecond ReaderRonald E. GiachettiChair, Department of Systems Engineeringiii
THIS PAGE INTENTIONALLY LEFT BLANKiv
ABSTRACTTwo common reliability prediction methods are the traditional method andphysics of failure method. Each method requires accurate failure data in order to fullyassess a system’s durability. This is particularly important in early system design whenhistorical design and relative failure rates are non-existent. Consequently, practitionersrely on the use of external reliability data sources such as MIL-HDBK-217F, especiallywhen using the traditional reliability approach. Several other external reliability datasources are available to the practitioner, each with its own strengths and limitations. Thisthesis surveys the various external data sources industries use in reliability predictionsand assesses the completeness of the reliability data sources. The thesis presents theinherent limitations of all external data sources along with further considerations on usingthe traditional reliability approach. Early system design offers practitioners a significantamount of decision-making flexibility. This thesis further analyzes both reliabilityapproaches and addresses when it is appropriate for a practitioner to use either approachor a combination of the two approaches. The author develops a reliability decisionframework to aid practitioners in selecting the reliability prediction approach appropriatefor the system.v
THIS PAGE INTENTIONALLY LEFT BLANKvi
TABLE OF CONTENTSI.INTRODUCTION.1II.AN ASSESSMENT OF EXTERNAL RELIABILITY DATASOURCES.3A.INTRODUCTION.3B.RELATED WORKS .4C.METHODOLOGY .51.External Data Sources .72.The Development of a Survey Framework for ExternalReliability Data Sources .133.The Assessment of External Reliability Data Sources .15D.CONCLUSION .18E.FUTURE WORK .18III.SELECTING THE CORRECT RELIABILITY APPROACH INEARLY SYSTEM DESIGN .19A.INTRODUCTION.19B.BACKGROUND AND RELATED WORK .20C.METHODOLOGY .221.Reliability Decision Framework .232.Traditional Reliability Approach .283.Physics of Failure Approach .30D.CASE STUDY .35E.DISCUSSION .37F.CONCLUSION .38G.FUTURE WORK .39IV.CONCLUSION .41V.FUTURE WORK .43LIST OF REFERENCES .45INITIAL DISTRIBUTION LIST .49vii
THIS PAGE INTENTIONALLY LEFT BLANKviii
LIST OF FIGURESFigure 1.The Relationship between Failure Data and Reliability Approaches .2Figure 2.A Network of Reliability Data Sources .12Figure 3.The Relationship of the RDF in the Preliminary Design Phase of theSystems Engineering Process .22Figure 4.A Decision Flowchart of Reliability Predictions .24Figure 5.Relevant APU Information Retrieved from the Functional Analysis .36ix
THIS PAGE INTENTIONALLY LEFT BLANKx
LIST OF TABLESTable 1.List of External Reliability Data Sources Surveyed in this Research .7Table 2.External Data Source Assessment Results .16Table 3.Common Failure Mechanisms for Electronic Devices .33xi
THIS PAGE INTENTIONALLY LEFT BLANKxii
LIST OF ACRONYMS AND ABBREVIATIONSAPUauxiliary power unitCAIcritical application itemCCAcircuit card assemblyCSIcritical safety itemDGADélégation Générale pour l’ArmementDMSMSdiminishing manufacturing sources and material shortagesEPRDelectronics parts reliability dataESSenvironmental stress screeningFMDfailure mode mechanism distributionFMECAfailure modes, effects, and criticality analysisGSMglobal system for mobile communicationHALThighly accelerated life testsHASAhighly accelerated stress auditHASShighly accelerated stress screenIEEEInstitute of Electrical and Electronics EngineersMTBFmean time between failureNCnon-criticalNPRDnon-electronic parts reliability dataNTTNippon Telegraph and TelephoneOEMoriginal equipment manufacturerPOFphysics of failureRACreliability analysis centerRDFreliability decision frameworkSAESociety of Automotive EngineersSTRIFEstress plus life testxiii
THIS PAGE INTENTIONALLY LEFT BLANKxiv
EXECUTIVE SUMMARYReliability predictions are a methodology for the estimation of an item’s ability tomeet the operational capabilities of the system and the specified reliability requirements.System reliability estimations are performed early in the design process to aid theevaluation of the design in terms of system requirements and to provide a basis forcontinued reliability improvements (Blanchard and Fabrycky 2011). Reliability predictionmethods can be categorized into two different approaches. These methods are thetraditional reliability prediction approach and the physics of failure approach. Thetraditional reliability approach is commonly used and MIL-HDBK-217F is the most widelyused source for predicting reliability of components (Varde 2010). All reliability predictionmethods rely on three critical areas: failure data, statistical modeling of the failure data,and the system’s reliability logic model. Failure data can be categorized into three types,field reliability data, test reliability data, and external data sources. Due to the limitedinformation available to the practitioner in the early design phase, the traditional reliabilityapproach is often constrained to using external data sources such as MIL-HDBK-217F.An assessment of the various common external data sources was conducted toevaluate the completeness of the reliability data sources. The result found that all externaldata sources are inherently limited. All external data sources can be considered derivativesof MIL-HDBK-217F and are found to be tailored toward a specific industry. The majorlimitations of external data sources include: constant failure rates and stress factors, the testand/or field environments are not known, the failure data is for generic component types,which does not account for the part quality, and the failure data is generally outdated. As aresult, the traditional reliability approach assesses one aspect of a failure and does notaccount for actual failure mechanisms.The physics of failure approach assesses how a system fails, identifies the rootcauses of failures, and takes into consideration different failure mechanisms. As a result,the physics of failure approach leads to a more robust reliability prediction. The failurexv
mechanisms are modeled based on the expected operational life-stress profile of thesystem. The physics of failure models take into consideration the cumulative wear andstress on the system as opposed to the nature of independent failures in the traditionalapproach. The primary limitations of the physics of failure approach is the amount of timeand additional costs required to assess the dominate failure mechanisms. Since the failuredata specific to certain failure mechanisms are not readily available to the practitioner fromsuppliers or external data sources, the physics of failure approach requires the use ofaccelerated life tests. Accelerated life testing of the system is critical to receiving accuratefailure rates pertaining to the identified failure mechanisms and determining the life-stressprofile of the failure.In the early system design process, the practitioner has great decision makingflexibility in terms of which reliability approach would best serve the system’s design. Areliability decision framework has been developed to assist the practitioner during thisprocess. Iterative reliability assessments are crucial in the design process to improve thesystem’s reliability. As a result, the reliability decision framework provides a focus on thereliability improvement and helps the practitioner in intelligently achieving theimprovement. The practitioner should consider four factors in deciding which reliabilityprediction method is appropriate for his system in addition to the cost and timeframefactors. These factors are the availability of relevant historical failure data, the level ofsystem complexity, the operational life requirement, and the criticality of the system. Thereliability decision framework utilizes these factors to guide the practitioner in selecting aneffective reliability approach for the system. The developed reliability decision frameworkpresented in Figure 1, applies to the beginning of the preliminary design phase in thesystems engineering process. The results further assist the practitioner in the allocation ofsystem requirements.xvi
Figure 1.A Reliability Decision FrameworkIn general, a physics of failure approach will provide the practitioner with anunderstanding of the root causes of system failure. This approach is more intensive thanthe traditional approach and yields a more robust reliability prediction and system design.The trade-off is the need on accelerated life test to obtain failure data and to develop lifestress profiles for specific failure mechanisms. The accelerated life tests will naturallyincrease the time and cost for the program. The traditional approach is not as accurate asthe physics of failure approach when using external data sources. The traditional reliabilityapproach is better suited for use when accurate historical failure data is available to thexvii
practitioner. Data from historical life tests may also be used in the traditional approach ifthe environment and stressors for the tests are known and relevant.ReferencesBlanchard, Benjamin S., and Wolter J. Fabrycky. 2011. Systems Engineering andAnalysis, 5th ed. Saddle River, NJ: Pearson Education Inc.Varde, P. V. 2010. “Physics-of-Failure Based Approach for Predicting Life andReliability of Electronics Components.” BARC Newsletter Mar.-Apr. (313): 38–46.xviii
ACKNOWLEDGMENTSSpecial thanks go to my very understanding wife and best friend, who havesupported me throughout this journey. I would also like to thank my advisor and mentor,Dr. O’Halloran, for the great deal of knowledge, advice, and encouragement he hasbestowed on me over the past few years.xix
THIS PAGE INTENTIONALLY LEFT BLANKxx
I.INTRODUCTIONUse of reliability predictions during early system design is a growing area ofinterest. Reliability predictions is a methodology for the estimation of an item’s ability tomeet the operational capabilities of the system and the specified reliability requirements.According to Blanchard and Fabrycky (2011), “A reliability prediction estimates theprobability that an item will perform its required functions during the mission.” Systemreliability estimations are performed early in the design process to aid the evaluation of thedesign in terms of system requirements and to provide a basis for continued reliabilityimprovements (Blanchard and Fabrycky 2011). Reliability predictions can be categorizedinto two different methods. These methods are the traditional reliability predictionapproach and the physics of failure approach.All reliability prediction methods can be broken down into three key factors: failuredata, statistical modeling, and the system’s reliability logic model. Of these three factors,the failure data offers the greatest area of concern that can drive the variability in reliabilitypredictions. Failure data can be collected through historical field data, accelerated life tests,or retrieved from external data sources. In early system design, practitioners are verylimited in the amount of data they have available to them. Often, historical data is notavailable and test data is infeasible to obtain due to limited development costs and stricttimeframes. As a result, it is common for practitioners to retrieve failure data from externalreliability sources. As shown in Figure 1, field and external reliability data are prevalent toa traditional reliability approach. Due to the strong need of modeling various failuremechanisms, test data is more prevalent in a physics of failure approach. Chapter II surveyscommon external reliability data sources available to practitioners and provides anassessment on the completeness of the reliability data sources. Considerations forpractitioners to use in early system design and a synthesis of the relationships amongvarious reliability data sources are also provided in Chapter II.1
Figure 1. The Relationship between Failure Data and Reliability ApproachesThe physics of failure approach seeks to understand the root causes of systemfailures. While this approach is not as common as the traditional approach, it is gainingpopularity in the community as it addresses some of the major issues with the traditionalapproach. Chapter III provides an overview of the physics of failure reliability approachand how it relates to the traditional approach. In addition, Chapter III presents a reliabilitydecision framework that addresses when it is appropriate for a practitioner to use atraditional or physics of failure approach.This thesis intends to aid practitioners in performing system reliability predictionsin the early stages of system design. The contributions of this thesis include a detailedassessment of common external reliability data sources, a synthesis of how variousreliability data sources connect to each other, and a reliability decision framework forpractitioners to utilize in early system design. These contributions address critical areas inthe reliability prediction process that result in great variability in reliability estimations. Ashighlighted in Figure 1, the contributions specifically aid the practitioner in the decisionmaking process with regard to the use of reliability prediction approaches and the use ofexternal data sources.2
I
NPRD non-electronic parts reliability data . NTT Nippon Telegraph and Telephone . OEM original equipment manufacturer . POF physics of failure . RAC reliability analysis center . RDF reliability decision framework . SAE Society of Automotive Engineers . STRIFE stress plus life test .
redstone test center . white sands test center . yuma proving ground . naval air warfare center aircraft division patuxent river . naval air warfare center weapons division china lake . naval air warfare center weapons division point mugu . naval surface warfare center dahlgren division . naval undersea warfare center division keyport . naval .
NDP 1, Naval Warfare, describes the ways naval forces accomplish their missions and execute their roles with joint and multinational teams; Naval Logistics explains how the support for those operations is planned, acquired, and integrated into the operations of naval expedi-tionary fo
NAVAL POSTGRADUATE SCHOOL Monterey, California THESIS Approved for public release; distribution is unlimited A STATISTICAL ANALYSIS OF THE PERFORMANCE OF NAVAL ACADEMY GRADUATES AT THE BASIC . Naval Academy, Marine Corps, Officer Accessions, Officer Recruiting, Officer Candidates School, .
Rick Allard, Matt Turner and David Hebert at the Naval Research Laboratory, Stennis Space Center, Andrew Roberts of the Naval Postgraduate School, Jean-Francois Lemieux and Frederic Dupont of Environment and Climate Change Canada, Tony Craig and his supporters at the National Center for Atmospheric Research, the Naval Postgraduate .
normally has five to seven years of naval officer experience, is still in his twenties, likely has only three years of . able to start this project and especially when I thought I would never be able to finish. vii . NPS Naval Postgraduate School . NROTC Naval Reserve Officer Training Corps . NWC Naval War College .
naval base coronado north island naval base coronado/ nosc north island; . naval postgraduate school nps monterey/nsa monterey; ca 6/15/15; x x; point mugu naval aws nb ventura county/ nosc ventura county ca; . u.s. coast guard academy ct; 6/15/15 1/1/19; x x; x guantanamo bay ns cub;
Naval Special Warfare (SEAL) Insignia Naval Parachutist Insignia Basic Parachutist Insignia Armed Forces Communications and Electronics Association Naval Intelligence Award for Academic Excellence at the Naval Postgraduate School Ross Perot Patriot Award - Dallas, Texas United
POSTGRADUATE RESEARCH STUDENT HANDBOOK 2013/14 The Graduate School. CONTENTS Section 1: Introduction 1 1 Professor Diane Houston (Dean of the Graduate School) 1 . 5 Faculty Directors of Graduate Studies 2 6 Postgraduate Students based at Brussels 2 7 Key Information 3 8 New Research Student’s Checklist 3 Section 2: Postgraduate Research at .
