The Value Of Verification And Validation
Session 12EThe Value of Verificationand ValidationBy:Carlos E DaSilvaQuality Assurance ManagerRockwell AutomationSensing, Safety and ConnectivityOctober 21, 2014ASQ NEQC 60th Conference, Springfield, Massachusetts
ObjectivesAt the conclusion of this session, you will be able to have a betterunderstand of:- Definitions of Verification and Validation for the Product LifeCycle- Why and when use Verification and Validation in the designphases- IEC Standards and V Model- What are the key tools used in the Verification and ValidationOctober 21, 2014ASQ NEQC 60th Conference, Springfield, Massachusetts2
DefinitionsWhat are your definitions for Verification and Validation?October 21, 2014ASQ NEQC 60th Conference, Springfield, Massachusetts3
The Basics, and more Prior 1700 AD:1.The first test during the building/assembly process to understand the output2.If the new design also have a new environmental (application), the tests weredone in the places where the experience base already existed3.During the building/assembly process, the design and assembly weaknessesand flaws were discovered and modifications to the system were made4.Sometimes a catastrophic failure of a monumental project would occur and theprocess would start over: occasionally after attending the funeral of theprevious chief designer and his apprentices (chief designers had skin in thegame)by William L Oberkampf and Christopher J Roy, Cambridge University PressOctober 21, 2014ASQ NEQC 60th Conference, Springfield, Massachusetts4
The Basics, and more After 1700 AD:1. Invention of Calculus (Newton and Leibniz)2. Initial steps of mathematical modeling of physics to obtain a simulation results3. Invention of Slide Rule (hand calculation) and increase of modeling andsimulation within Industrial Revolution, as a minor support role4. The second World War, and the Cold War pushed the needs for usage ofmodeling and simulation5. A major step in modeling and simulation was achieved in 1965 with thepublication of “Computer Experiments in Fluid Dynamics” by Harlow andFromm.6. The 3rd pillar in science was recognized to be modeling and simulation, asscientific computing, in conjunction with Theory and Experiment.7. The concepts of Design Verification and Product Validation raised upOctober 21, 2014ASQ NEQC 60th Conference, Springfield, Massachusetts5
DefinitionsVerification Act of reviewing, inspecting, testing, checking, auditing, orotherwise establishing and documenting whether items,processes, services or documents conform to specifiedrequirements (American National Standard 1978)Validation Confirmation by examination and through provision ofobjective evidence that the requirements for a specificintended use or application have been fulfilled (InternationalOrganization for Standardization - ISO 2008).October 21, 2014ASQ NEQC 60th Conference, Springfield, Massachusetts6
What Product Development process I have?Phase 1Phase 2Phase 3Phase 4Phase 5Other Guys!Currenteffortconceptproduct dev.releaseseriesprocess dev.DevelopmentorderSeries orderStart of ProductionVALIDATIONOctober 21, 2014ASQ NEQC 60th Conference, Springfield, Massachusetts7
Key elements to achieve success October 21, 2014 Customer Needs Time to Market Risk Mitigation Simulation Variation Reduction Cost / MarginASQ NEQC 60th Conference, Springfield, Massachusetts8
How the CEOs were looking the future“Companies today are facing daily challenges related to the key factors that havedriven global business success and supply chain performance. Published in1998 the ground breaking report “The Nation’s CEOs Look to the Future(1)”(Table A) surveyed more than 300 CEOs to identify the top five most importanttrends related to competitive advantage.”(1)MAJOR TRENDSCOREGlobalization -- becoming truly global companiesImproving knowledge management and the use of new information technologyCost and cycle time reductionImproving supply chains globallyManufacturing at multiple locations in many countriesManaging the use of more part-time, temporary and contract workersDeveloping new employee relationships based on performance and personal growth, nottenure and long-term commitmentImproving human resources and the hiring, training, retention &motivation of the talentImproving execution of company strategic plansDeveloping more appropriate strategic plansOngoing measurement/analysis of organizational processesDeveloping a consistent global corporate cultureOutsourcing of manufacturingCreating learning organizationsOutsourcing of white collar servicesManaging employee turnoverHiring the best talent in developing countries such as Russia, India, ChinaGiving more weight to long term goals and relatively less to quarterly earningsAppointing CEOs and top managers who are not U.S. citizens94%88%79%78%76%71%Variation &Risk69%68%68%64%60%56%55%52%41%41%23%19%9%from Global Competition with Global Competence by Carlos E DaSilva, Copyright 2007 SAE InternationalOctober 21, 2014ASQ NEQC 60th Conference, Springfield, Massachusetts9
Risk portComplexity factors Rapid technology change Shorter component lifecycles # and age of productsavailable for sale Ratio of “base” to “options”to “accessories” # of unique part numbersacross products # and type of raw materialspecifications & tolerances Environmental regulation and certificationSupply market availability and pricing# and location of suppliersManufacturing location, capacity and agility(volume & mix support)Complex supply and materials planningInventory stocking and availabilityImport / export complianceDistribution and transportation complexitiesQualityDemand variabilityGlobalizationLow cost competitorsSolutions vs. productsTechnology lifecyclemanagement Contracts and pricing OEM growth Longer design cycles Increased product cost Lower supply chain visibility Less accurate forecasts Lack of agility to customerdemand variation Longer lead times / poor service Lower customer satisfaction Less growthOctober 21, 2014ASQ NEQC 60th Conference, Springfield, Massachusetts10RISKRISKPotential ramifications
Holistic view – Adding Value Phase 1conceptPhase 2Phase 3product dev.Phase 4releaseseriesprocess dev.DevelopmentorderSeries orderVERIFICATIONOctober 21, 2014Phase 5Start of ProductionVALIDATIONVALIDATIONASQ NEQC 60th Conference, Springfield, Massachusetts11
Quality Assurance for Risk MitigationQuality Management - ProcessesQuality Assurance - VerificationQuality Control - ValidationASSEMBLY LEVELENGINEERING LEVELMANAGEMENT LEVELOutputOctober 21, 2014Effectiveness (action & results)InputEfficiency (competency in performance)ASQ NEQC 60th Conference, Springfield, Massachusetts12
How this works by Gary Powalisz, GE HealthcareOctober 21, 2014ASQ NEQC 60th Conference, Springfield, Massachusetts13
Definitions from the ISO 9001 – 2008 (Design)7.3.5 Design and development verification (modeling and simulation prototype)Verification shall be performed in accordance with planned arrangements (see 7.3.1) toensure that the design and development outputs have met the design and developmentinput requirements. Records of the results of the verification and any necessary actionsshall be maintained (see 4.2.4).7.3.6 Design and development validation (product process testing Pilot)Design and development validation shall be performed in accordance with plannedarrangements (see 7.3.1) to ensure that the resulting product is capable of meeting therequirements for the specified application or intended use, where known. Whereverpracticable, validation shall be completed prior to the delivery or implementation of theproduct. Records of the results of validation and any necessary actions shall bemaintained (see 4.2.4).October 21, 2014ASQ NEQC 60th Conference, Springfield, Massachusetts14
What are the path for V & VOctober 21, 2014ASQ NEQC 60th Conference, Springfield, Massachusetts15
Few concepts to be discussedRisk are things that can affect a system and cause a drop in Dependability. There arethree main terms that must be clearly understood: Fault: A fault (which is usually referred to as a bug for historic reasons) is a defect ina system. The presence of a fault in a system may or may not lead to a failure. Forinstance, although a system may contain a fault, its input and state conditions maynever cause this fault to be executed so that an error occurs; and thus that particularfault never exhibits as a failure. Error: An error is a discrepancy between the intended behavior of a system and itsactual behavior inside the system boundary. Errors occur at runtime when some partof the system enters an unexpected state due to the activation of a fault. Since errorsare generated from invalid states they are hard to observe without specialmechanisms, such as debuggers or debug output to logs. Failure: A failure is an instance in time when a system displays behavior that iscontrary to its specification. An error may not necessarily cause a failure, for instancean exception may be thrown by a system but this may be caught and handled usingfault tolerance techniques so the overall operation of the system will conform to thespecification.October 21, 2014ASQ NEQC 60th Conference, Springfield, Massachusetts16
Where the Risks areOctober 21, 2014ASQ NEQC 60th Conference, Springfield, Massachusetts17
Why Identify RisksCustomer DisasterReliable ProductsCustomer doesn’t WantCompetitive AdvantageDesign and Process OrientationOctober 21, 2014ASQ NEQC 60th Conference, Springfield, Massachusetts18
Summary of IEC 61508IEC 61508-1:2010 covers requirements for implementing variety of processrelated measures over a Product’s lifecycle. Clause 5 – Documentation ManagementClause 6 – Functional Safety Management– Define FS policy– Define FS Product Lifecycle Assign roles & responsibilities relative to each lifecycle phase Assure competency of individuals assigned FS responsibilities for each lifecyclephase Identify individuals / departments / organizations involved at each lifecycle phase– Define communication mechanisms– Define requirements for periodic AuditsCost: Incremental upfront effort!Opportunity: Improved Quality!Fault avoidance leading to lower backend costs!October 21, 2014ASQ NEQC 60th Conference, Springfield, Massachusetts19
Summary of IEC 61508 (cont’d)Other sections of IEC 61508:2010 IEC 61508, Part 2– HW Development Fault avoidance measures Fault control measures IEC 61508, Part 3– SW Development Fault avoidance measures Fault control measures Tool managementCost: Incremental upfront effort!Opportunity: Improved Quality!Fault avoidance leading to lower backend costs!October 21, 2014ASQ NEQC 60th Conference, Springfield, Massachusetts20
More definitions from the IEC 61508-4 (Safety Measures)3.8.1verificationconfirmation by examination and provision of objective evidence that the requirementshave been fulfilled[ISO 8402, definition 2.17, modified]NOTE In the context of this standard, verification is the activity of demonstrating for each phase ofthe relevant safety lifecycle (overall, E/E/PE system and software), by analysis, mathematicalreasoning and/or tests, that, for the specific inputs, the outputs meet in all respects the objectives andrequirements set for the specific phase.EXAMPLE Verification activities include– reviews on outputs (documents from all phases of the safety lifecycle) to ensure compliance withthe objectives and requirements of the phase, taking into account the specific inputs to that phase;– design reviews;– tests performed on the designed products to ensure that they perform according to theirspecification;– integration tests performed where different parts of a system are put together in a step-by-stepmanner and by the performance of environmental tests to ensure that all the parts work together inthe specified manner.October 21, 2014ASQ NEQC 60th Conference, Springfield, Massachusetts21
More definitions from the IEC 61508-4 (Safety Measures)(cont’d)3.8.2validationconfirmation by examination and provision of objective evidence that theparticular requirements for a specific intended use are fulfilled[ISO 8402, definition 2.18, modified]NOTE 1 In this standard there are three validation phases:– overall safety validation (see Figure 2 of IEC 61508-1);– E/E/PE system validation (see Figure 3 of IEC 61508-1);– software validation (see Figure 4 of IEC 61508-1).NOTE 2 Validation is the activity of demonstrating that the safety-relatedsystem under consideration, before or after installation, meets in all respectsthe safety requirements specification for that safety-related system.Therefore, for example, software validation means confirming by examinationand provision of objective evidence that the software satisfies the softwaresafety requirements specification.October 21, 2014ASQ NEQC 60th Conference, Springfield, Massachusetts22
V Model - traceability for Verification and Validation The V-model is not only another way of looking at a waterfallapproach. The main distinction from the waterfall method is thateach of the activities at each level of development (the left side ofthe V) is related to the relevant activity for verifying andvalidating the system (the right side of the V).October 21, 2014ASQ NEQC 60th Conference, Springfield, Massachusetts23
Verification Tools - Modeling and SimulationOctober 21, 2014ASQ NEQC 60th Conference, Springfield, Massachusetts24
Verification Tools – measuring performanceStandards for dependability and the benefits derived from them areoften taken for granted, their importance is not appreciated untilproblems arise.Dependability is a combination of: Availability: readiness for Design Intent Reliability: continuity of Design Intent Maintainability: easy to repairIEC 60300-1 Dependability management - Part 1: Dependability management systemsIEC 60300-2 Dependability management - Part 2: Guidelines for dependability managementIEC 61160 Design reviewOctober 21, 2014ASQ NEQC 60th Conference, Springfield, Massachusetts25
Verification Tools – continuity of Design IntentOctober 21, 2014ASQ NEQC 60th Conference, Springfield, Massachusetts26
Verification Tools – continuity of Design IntentOctober 21, 2014ASQ NEQC 60th Conference, Springfield, Massachusetts(cont’d)27
Verification Tools – continuity of Design IntentOctober 21, 2014ASQ NEQC 60th Conference, Springfield, Massachusetts(cont’d)28
Re-Validation to avoid surprises – Adding more Value Phase 1Phase 2conceptPhase 3Phase 4productdev.Phase 5releaseseriesprocess dev.DevelopmentorderOctober 21, 2014SeriesorderStart of ProductionVERIFICATIONVALIDATIONRe –VALIDATIONComponent & SystemComponent & SystemSystemASQ NEQC 60th Conference, Springfield, Massachusetts29
Thank youQuestions ?October 21, 2014ASQ NEQC 60th Conference, Springfield, Massachusetts30
Carlos E silvaCertified Manager of Quality/Organizational ExcellenceCertified Six Sigma Black BeltCertified Quality Auditor30 years of experience in Quality, Lean Manufacturing, and Organizational Excellencewith Electronics, Chemical and Automotive plants in South/North Americas and EuropeAuthor and co-author of publications, including:Leadership Through ISO/TS 16949MSBU - M/A COM & TYCO ELECTRONICS, Springfield, USA - October 11, 2003Global Competition with Global CompetenceSAE International - Detroit, USA - April, 2008Optimizing Process PerformanceManufacturing Executive Leadership Forum and Hannover Fair International.Hannover, Germany - April, 2009October 21, 2014ASQ NEQC 60th Conference, Springfield, Massachusetts31
IEC 60300-1 Dependability management - Part 1: Dependability management systems IEC 60300-2 Dependability management - Part 2: Guidelines for dependability management IEC 61160 Design review October 21, 2014 ASQ NEQC 60th Conference, Springfield, M
Silat is a combative art of self-defense and survival rooted from Matay archipelago. It was traced at thé early of Langkasuka Kingdom (2nd century CE) till thé reign of Melaka (Malaysia) Sultanate era (13th century). Silat has now evolved to become part of social culture and tradition with thé appearance of a fine physical and spiritual .
May 02, 2018 · D. Program Evaluation ͟The organization has provided a description of the framework for how each program will be evaluated. The framework should include all the elements below: ͟The evaluation methods are cost-effective for the organization ͟Quantitative and qualitative data is being collected (at Basics tier, data collection must have begun)
̶The leading indicator of employee engagement is based on the quality of the relationship between employee and supervisor Empower your managers! ̶Help them understand the impact on the organization ̶Share important changes, plan options, tasks, and deadlines ̶Provide key messages and talking points ̶Prepare them to answer employee questions
Dr. Sunita Bharatwal** Dr. Pawan Garga*** Abstract Customer satisfaction is derived from thè functionalities and values, a product or Service can provide. The current study aims to segregate thè dimensions of ordine Service quality and gather insights on its impact on web shopping. The trends of purchases have
On an exceptional basis, Member States may request UNESCO to provide thé candidates with access to thé platform so they can complète thé form by themselves. Thèse requests must be addressed to esd rize unesco. or by 15 A ril 2021 UNESCO will provide thé nomineewith accessto thé platform via their émail address.
Chính Văn.- Còn đức Thế tôn thì tuệ giác cực kỳ trong sạch 8: hiện hành bất nhị 9, đạt đến vô tướng 10, đứng vào chỗ đứng của các đức Thế tôn 11, thể hiện tính bình đẳng của các Ngài, đến chỗ không còn chướng ngại 12, giáo pháp không thể khuynh đảo, tâm thức không bị cản trở, cái được
Food outlets which focused on food quality, Service quality, environment and price factors, are thè valuable factors for food outlets to increase thè satisfaction level of customers and it will create a positive impact through word ofmouth. Keyword : Customer satisfaction, food quality, Service quality, physical environment off ood outlets .
More than words-extreme You send me flying -amy winehouse Weather with you -crowded house Moving on and getting over- john mayer Something got me started . Uptown funk-bruno mars Here comes thé sun-the beatles The long And winding road .
