Method Validation And Verification

Method validation andverificationW. W. WongSenior Accreditation OfficerHKAS28 April 20091

Method Validation What is it?When is it required?Why is it necessary?What are required?How much is adequate?How should it be done?2

Method Validation – what is it Definition – validation is the confirmationby examination and the provision ofobjective evidence that the particularrequirements for a specific intended use arefulfilled (ISO/IEC 17025:2005 cl. 5.4.5.1) A process of evaluating method performanceand demonstrating that it meets a particularrequirement3

Method Validation – what is it Specific intended use – why are youanalysing the samples and what are theresulting data used for Objective evidence – set of results fromvalidation studies Confirmation – compare informationobtained with customer’s requirements4

Method Validation – when is it required When is validation not required standard methods on condition that– used within their scope of applicability– without modifications (including QA planand reporting) Otherwise, required5

Method Validation – when is it required Non-standard methods Laboratory-developed methods6

Method Validation – when is it required Standard methods and validated methodswhich are:– used outside their intended scope (e.g.matrices, ranges, etc)– amplified or modified (e.g. analyticaltechniques, etc.)7

Method Validation – why is it necessary To meet accreditation requirement Ensure that the test method give “correct”results Objective evidence for defence againstchallenges Customers want to be assured of thecorrectness of results8

Method Validation – why is it necessarySamplesProficiency testingReference materialsValidated methodsQuality of test results9

Method Validation – what are required ISO/IEC 17025:2005 Section 5.4 Test andcalibration methods and method validation Selection of methods (cl. 5.4.2)Laboratory-developed methods or methodsadopted by the laboratory may also beused if they are appropriate for theintended use and if they are validated.10

Method Validation – what are requiredLaboratory-developed methods (cl. 5.4.3) introduction of test methods for its own use planned activity assigned to qualified personnel equipped with adequate resources plans updated effective communication(Process requirements)11

Method Validation – what are requiredNon-standard method (cl. 5.4.4) methods not covered by standard methods agree with customers include clear specifications of customer’srequirements and purpose of test validated appropriately before use(a laboratory-developed method is a non-standardmethod but the reverse may not be true)12

Method Validation – what are requiredValidation of methods (cl. 5.4.5.2) as extensive as is necessary record– results obtained– procedure used– statement that method is fit for intendeduse13

Method Validation – what are requiredValidation of methods (cl. 5.4.5.2 Note 1) May include procedure for sampling,handling and transportation14

Method Validation – what are requiredValidation of methods (cl. 5.4.5.2 Note 2) techniques for method performancedetermination include– Calibration using reference standards and ReferenceMaterials– Comparison of results achieved with other methods– Interlaboratory comparisons– Systematic assessment of the factors influencing the result– Assessment of uncertainty of results based on scientificunderstanding of the theoretical principles of the method andpractical experience15

Method Validation – what are requiredValidation of methods (cl. 5.4.5.2 Note3) When changes are made in thevalidated non-standard methods, theinfluence of such changes should bedocumented and, if appropriate, a newvalidation should be carried out.16

Method Validation – what are requiredValidation of methods (cl. 5.4.5.3 ) Range and accuracy of valuesobtained from validated methods,as assessed for the intended use,shall be relevant to the customers’need17

Method Validation – what are requiredValidation of methods (cl. 5.4.5.3 )Examples: Uncertainty of resultsDetection ityRobustness/cross-sensitivity18

Method Validation – what are requiredValidation of methods (cl. 5.4.5.3 Note 1 ) Validation includes– Specification of the requirements– Determination of characteristics ofmethod– Check requirement fulfilled by method– Statement on validity19

Method Validation – what are requiredValidation of methods (cl. 5.4.5.3 Note 2) Regular review as method developmentproceeds to verify customer needsfulfilled Modifications to development plan dueto change in requirements should beapproved and authorised20

Method Validation – what are requiredValidation of methods (cl. 5.4.5.3 Note 3) Validation is a balance betweencosts, risks and technicalpossibilities21

Method Validation – what are requiredValidation of methods(5.4 H HOKLAS Policy (c)) Requirements differ significantly fromone technical discipline to another Should commensurate with intended use Confirm fit for intended use22

Method Validation – what are requiredValidation of methods(5.4 H HOKLAS Policy (d)) HOKLAS classifies test methods into 3– standard method– standard method with modifications– in-house method23

Method Validation – what are requiredValidation of methods(5.4 H HOKLAS Policy (d)) standard method–Conforms exactly to the standard24

Method Validation – what are requiredValidation of methods(5.4 H HOKLAS Policy (d)) standard method with modifications– Differs from standard but deviations unlikelyto affect test results– Need supporting evidence, i.e. validationrequired– Modifications stated in scope of accreditationand test reports25

Method Validation – what are requiredValidation of methods(5.4 H HOKLAS Policy (d)) In-house method– Methods other than the above two– Validation required– Cannot make reference to otherstandard method, i.e. cannot claim“based on” a standard method26

Method Validation – what are requiredValidation of methods(5.4 H HOKLAS Policy (f)) Non-standard methods shall be fullydocumented and validated27

Method Validation – what are requiredValidation of methods(5.4 H HOKLAS Policy (g)) Shall have policy and procedure for design,development and subsequent validation of– in-house methods,– laboratory designed/developed methods,– standard methods used outside their intended scope– amplifications and modifications of standardmethods28

Method Validation – what are requiredHOKLAS Supplementary Criteria No.20 “Chemical Testing”, “ChineseMedicine”, “Construction Materials”,“Food”, Toys and Children’sProducts” – Chemical TestingSection 5.3 Validation of methods29

Method Validation – what are requiredHOKLAS SC No. 20 Section 5.3 Concentration range Sample matrices “more advanced” techniques maybe a deviation Conformation of identity30

Method Validation – what are requiredHOKLAS SC No. 20 Section 5.3 LoD Precision and trueness Applicable concentration range Applicable sample matrices31

Method Validation – what are requiredHOKLAS SC No. 20 Section 5.3 Method bias assessed by appropriate matrixCRMs– Levels of analytes– Matrix matches intended sample matrix– Uncertainty of assigned values suitable32

Method Validation – what are requiredHOKLAS SC No. 20 Section 5.3 Method performance characteristicsreview, and revised, regularly33

Method Validation – what are requiredHOKLAS SC No. 20 Section 5.3 For food analysis, the method validationrequired depends very much on theanalytes of interest and the matrices.Common food matrices include thoserich in protein, carbohydrate, oil,dietary fibre, liquid, etc.34

Method Validation – what are requiredHOKLAS SC No. 20 Section 5.3 If a method is to be accredited forgeneral food, satisfactory validationdata shall be obtained for at leastfive different food matrices with atleast three kinds of food for eachfood matrix.35

Method Validation – what are requiredHOKLAS SC No. 20 Section 5.3 The range of matrices shall also in linewith those listed in relevant regulations.Due consideration shall also be takenfor the food matrices with potentialinterferences, e.g. high chloride effecton the ICP-MS determination.36

Method Validation – what are requiredHOKLAS SC No. 20 Section 5.5 Estimation of uncertainty ofmeasurement37

Method Validation – how much is adequateDepends on the critically of the measurement The scope of the method38

Method Validation – how much is adequate How far can the light shine? How far can the validation data on somematrices be extended to other matrices? Need professional judgment based on technicalknowledge of the limitations of themethodology used.39

Method Validation – how much is adequate Example: General foodstuff Can never validate using all possible foods Classify food according to matrix from ananalytical chemistry point of view Main components of food – fat, protein,carbohydrate, (fibre, water) To obtain reliable data, 3 food types for eachmatrix Performance at specification limits should beavailable40

0%100%Method Validation – how much is adequate%FATINTERO%P0%0%10100%0%%CARBOHYDRATES41J. AOAC, 83 (2), 413 (2000)“The Referee”, AOAC Int’l, July 1993

Laboratory internal validation andverificationExisting informationFully validated standard methods(have been studied in a on(Secondary validation)Standard methods – amplifications and Validationmodifications e.g. new instrumentStandard methods – outside theirintended scopeValidationLaboratory – developed and nonstandard methodsValidation(Primary validation)42

Definition Validation is the confirmation by examination andprovision of objective evidence that the particularrequirements for a specific intended use are fulfilled(ISO/IEC 17025 clause 5.4.5.1) Verification – confirmation, through the provision ofobjective evidence, that specified requirements havebeen fulfilled (ISO 9000: 2005)Note: intended use vs specified requirements43

Method VerificationISO/IEC 17025 cl. 5.4.2 The laboratory shall confirm that it canproperly operate standard methods beforeintroducing the tests or calibrations. If thestandard method changes, the confirmationshall be repeated.44

Method Verification – what are requiredHOKLAS SC No. 20 Section 5.4Verification of methods A laboratory using standard methods has to confirmthat it has the ability to carry out thosemethods .Verification is usually carried out bycomparing the performance data obtained by thelaboratory when performing a standard methodwith those claimed by the same method.(note: specified requirements include the methodperformance of the std methods)45

Method verification Released early 2008 http://www.aoac.org/alacc guide 2008.pdf46

Method Verification importing a validated method show that laboratory can do it at its site demonstrate that laboratory can repeat themethod performance47

Method VerificationStandard methods shall be verified for:1. the equipment2. the requiredreagentsreferencematerials/standard,3. the environmental conditions4. testing staff member competence to performthe test5. capability to achieve the method performance48

How should it be done?The following are some examples of useful references Harmonised guidelines for single-laboratoryvalidation of method of analysis (IUPAC TechnicalReport), 2002 How to meet ISO 17025 Requirements for MethodVerification, AOAC, 2007 ISO/TS 21748:2004 Guidance for the use ofrepeatability, reproducibility and trueness estimatesin measurement uncertainty estimation49

How should it be done?Validation of methods (cl. 5.4.5.2 Note 2) techniques for method performancedetermination include–––––Calibration using reference stds and RMsComparison of results achieved with other methodsInterlaboratory comparisonsSystematic assessment of the factors influencing the resultAssessment of uncertainty of results based on scientificunderstanding of the theoretical principles of the method andpractical experience50