First Inter-laboratory Comparison Report First Inter-laboratory .

Figure 17. The z score for each single exchangeableK content determined using ammonium acetateas an extractant (K exch). The coloured linescorrespond to the limit of questionable (green) andunsatisfactory (red) results. 23Figure 26. Histograms presenting the distributionof all organic carbon content, determined usingthe Walkley and Black method (OC WB), overlaidby a model of normal distribution (bell curve). 34Figure 27. Histograms presenting the distributionof all organic carbon content, determined usingthe combustion method (OC Comb), overlaid by amodel of normal distribution (bell curve). 35Figure 18. The mean z score for the replicates ofpH measures determined on a 1:2.5 soil: watersuspension, the coloured lines correspond to thelimit of warning (green) and unsatisfactory (red)results 25Figure 28. Histograms presenting the distributionof all available P content, determined using theOlsen method (P Olsen), overlaid by a model ofnormal distribution (bell curve). 36Figure 19. The mean z score calculated from thereplicates of organic carbon content determinedusing the Walkley and Black method (OC WB).The coloured lines correspond to the limit ofquestionable (green) and unsatisfactory (red)results. 26Figure 29. Histograms presenting the distributionof all available P content, determined using theBray 1 method (P Bray.1) overlaid by a model ofnormal distribution (bell curve). 37Figure 20. The mean z score calculated from thereplicates of organic carbon content determinedusing combustion method (OC Comb). Thecoloured lines correspond to the limit ofquestionable (green) and unsatisfactory (red)results. 27Figure 30. Histograms presenting the distributionof all available P content, determined using theBray 2 method (P Bray.2) overlaid by a model ofnormal distribution (bell curve). 38Figure 31. Histograms presenting the distributionof all exchangeable K content determined usingammonium acetate as an extractant (K exch)overlaid by a model of normal distribution (bellcurve). 39Figure 21. The mean z score calculated from thereplicates of available P content determined usingthe Olsen method (P Olsen). The coloured linescorrespond to the limit of questionable (green) andunsatisfactory (red) results. 28Figure 32. Precision of each analytical methodestimated from the coefficient of variation (cv)between replicates from the same soil (955, 970, KI,LB from left to right). Note that a low cv indicates ahigh precision and vice versa. 41Figure 22. The mean z score calculated from thereplicates of available P content determined usingthe Bray 1 method (P Bray.1). The coloured linescorrespond to the limit of questionable (green) andunsatisfactory (red) results. 29Figure 33. Laboratories’ precision calculated fromvariations between analytical results obtained onthe replicates of the same soil samples (3 rep. for955 and 970, 4 rep. for KI and LB). Note that a lowcv indicates a high precision and vice versa. 43Figure 23. The mean z score calculated from thereplicates of available P content determined usingthe Bray 2 method (P Bray.2). The coloured linescorrespond to the limit of questionable (green) andunsatisfactory (red) results. 30Figure 34. Laboratories’ mean precision calculatedfrom the four coefficients of variation presentedon Figure 33; error bars represent the standarddeviation around the mean. Note that a low cvindicates a high precision and vice versa; shorterror bars indicates similar precision for thedifferent soil types and vice versa. 45Figure 24. The mean z score calculated from thereplicates of exchangeable K content determinedusing ammonium acetate as an extractant (Kexch). The coloured lines correspond to the limitof questionable (green) and unsatisfactory (red)results. 31Figure 25. Histograms of the distribution of allpH measures, determined on 1:2.5 soil: watersuspension, overlaid by a model of normaldistribution (bell curve). 33V

VIWikimedia

PrefaceReference Laboratories are tasked to establishNational Soil Laboratory Networks in order totransfer GLOSOLAN knowledge to the othernational laboratories that can spontaneouslyregister in the network. The first regional networklinked to GLOSOLAN was established in Asia inNovember 2017. The network was named afterthe already existing South-East Asian LaboratoryNetwork (SEALNET), which was launched in 2014by Mrs. Nopmanee Suvannang, at that time, Headof the Soil Analysis Laboratory and researcher fromthe Land Development Department of Thailand,and Dr. Christian Hartmann, researcher from theInstitut de Recherche pour le Développement –IRD, France.The Global Soil Laboratory Network (GLOSOLAN)was formally established under the frameworkof the Global Soil Partnership (GSP) in November2017, when its first meeting took place at FAOHeadquarters in Rome, Italy. GLOSOLAN’sobjectives are: (1) to strengthen the performance oflaboratories through use of standardized methodsand protocols, and (2) to harmonize soil analysismethods so that soil information is comparableand interpretable across laboratories, countriesand regions. In this context, GLOSOLAN plansto develop open access Standard OperatingProcedures and manuals on good laboratorypractices, execute regional and global proficiencytesting, and increase the overall performance oflaboratories through the organization of trainingsessions. By April 2019, over 220 laboratories fromall continents were registered in GLOSOLAN.During their first meeting, managers from 18National Reference Laboratories in Asia decidedto maintain the name “SEALNET” for their regionalnetwork and elected Dr. Jamyang (the laboratorymanager from The Soil and Plant AnalyticalLaboratory - SPAL, Bhutan) as Chair and Dr. Gina P.Nilo (laboratory manager from the Bureau of Soilsand Water Management - BSWM, Philippine) asvice-Chair. They agreed on the SEALNET work planfor the year 2018, which included the conductionof an independent assessment of the technicalperformance of SEALNET laboratories through aninter-laboratory comparison.The potential of GLOSOLAN is enormous. Byincreasing laboratory performance, the Networkwill support decision making at field and policylevels; support countries in reporting on theSustainable Development Goals (SDGs) and onother international commitments; contribute tothe development of international standards andindicators; contribute to the establishment of theGlobal Soil Information System (GLOSIS), which isanother priority activity of the GSP. GLOSOLAN willalso contribute to the development of harmonizedmethods for the assessment and monitoring ofdegraded lands, the impact of climate changeon lands, and other threats to soil functions, asidentified in the Status of the World Soil Resourcesreport. The Network has the potential to improvethe connection between soil chemistry, physicsand biology; contribute to and improve soilclassification and description; assist companiesmanufacturing laboratory equipment in improvingtheir products; expand the opportunities fortechnical and scientific cooperation; strengthenthe capability of extension services; identifyresearch needs; and increase investments in soilrelated research.This exercise was co-funded by the Global SoilPartnership (GSP) of the Food and AgricultureOrganization of the United Nations (FAO),L’Institut de Recherche pour le Développement(IRD, France), and the Land DevelopmentDepartment (LDD, Ministry of Agriculture andCooperatives, Thailand). I wish to express thegratitude of the GSP and FAO to all partnersinvolved and to Mrs. Nopmanee Suvannang andDr. Christian Hartmann, who led this initiativewith professionalism and on a voluntary basis. Ourgratitude also goes to the laboratory managerswho analysed the samples and provided data, andto the external reviewers who helped to ensurethe high quality of the analysis. It is our hope thatthe results and conclusions of this report willassist SEALNET laboratories in improving theirperformance, and inspire other RESOLANs andlaboratories to join GLOSOLAN.GLOSOLAN operates at the regional levelthrough its Regional Soil Laboratory Networks(RESOLANs) and at the national level throughNational Reference Laboratories identified bythe GSP national focal points. These NationalMr. Eduardo MansurDirector Land and Water DivisionVII

AcknowledgementsRecherche pour le Développement (IRD, France)and the Global Soil Partnership of FAO forfinancially supporting the execution of this interlaboratory comparison. Ultimately, the authorswish to thank the Indonesian Soil ResearchInstitute (ISRI) for hosting the First Regional SoilLaboratory Network for Asia (SEALNET) meetingand overall allowing this exercise to be conducted.The authors wish to express their gratitudeto all reviewers, whom provided constructivecomments and criticisms. A special thanks goes toDr. Philip MOODY and Mr. Robert DEHAYR fromthe Australasian Soil and Plant Analysis Council(ASPAC). Our gratitude also goes to the LandDevelopment Department, Ministry of Agricultureand Cooperatives of Thailand, the Institut deVIII

Definitions and terminologycoming from a different population or (ii) resultingof an error in measurement or in transcription[EURACHEM Guide, 1998].ACCURACY‘The closeness of agreement between a testresult and the accepted reference value’. Note:The term ‘accuracy,’ when applied to a set oftest results, involves a combination of randomcomponents and a common systematic error orbias component.‘A quantity referring to the differences between themean of a set of results or an individual result andthe value which is accepted as true or correct valuefor the quantity measured’ [EURACHEM Guide,1998].PRECISION‘The closeness of agreement between independenttest results obtained under stipulated conditions.’Note: Precision depends only on the distributionof random errors and does not relate to the truevalue or specified value. The measure of precisionis usually expressed in terms of imprecision andcomputed as a standard deviation of thetest results. ‘Independent test results’ meansresults obtained in a manner not influencedby any previous result on the same or similartest object. Quantitative measures of precisiondepend critically on the stipulated conditions.Repeatability and Reproducibility are particularsets of extreme conditions [ISO Guide 35]. ‘Ameasure for the reproducibility of measurementswithin a set that is of the scatter or dispersion ofa set about its central value’ [EURACHEM Guide,1998].ASSIGNED VALUEBest available estimate of the true value [UNODC,2009].CERTIFIED REFERENCE MATERIAL (CRM):Reference material one or more of whose propertyvalues are certified by a technical procedure,accompanied by or traceable to a certificateor other documentation which is issued by acertifying body [UNODC, 2009].PROFICIENCY TESTING (also called‘External QC’ or ‘inter laboratorycomparison’)A periodic assessment of the performance ofindividual laboratories and groups of laboratoriesthat is achieved by the distribution by anindependent testing body of typical materialsfor unsupervised analysis by the participants’[EURACHEM Guide, 1998].CONSENSUS VALUEValue produced by a group of experts or refereelaboratories using the best possible methods. It isan estimate of the true value [UNODC, 2009].ERROR (OF MEASUREMENT)‘The value of a result minus the true value’[EURACHEM Guide, 1998].INTERNAL QUALITY CONTROLSet of procedures undertaken by a laboratory forcontinuous monitoring of operations and resultsin order to decide whether the results are reliableenough to be released. Quality control of analyticaldata primarily monitors the batchwise trueness ofresults on quality control materials, and precisionon independent replicate analysis of test materials[UNODC, 2009].QUALITY CONTROLA set of activities or techniques whose purpose isto ensure that all quality requirements are beingmet. Simply put, it is examining “control” materialsof known substances along with patient samplesto monitor the accuracy and precision of thecomplete examination process [UNODC, 2009].REFERENCE MATERIAL (RM)Reference material, one or more of whoseproperty; are certified by a technical procedure,accompanied by, or treceable to, a certificate,or other documentation, which is issued by acertifying body [UNODC, 2009].OUTLIERSOutliers are extreme values, so far separatedfrom the other values that it suggests they are (i)IX

STANDARD DEVIATIONThis is a measure of how values are dispersedabout a mean in a distribution of values: Thestandard deviation ‘s’ for the whole population of ‘n’values is given by:ni 1σ interval. Uncertainty of measurement comprises,in general, many components. Some of thesecomponents may be evaluated from thestatistical distribution of the results of a seriesof measurements and can be characterised byexperimental standard deviations. The othercomponents which can also be characterisedby standard deviations, are evaluated fromassumed probability distributions based onexperience or other information. It is understoodthat the result of the measurement is the bestestimate of the value of the measurand andthat all components of uncertainty, includingthose arising from systematic effects, such ascomponents associated with corrections andreference standards, contribute to the dispersion’[EURACHEM Guide, 1998].(xi µ)2nIn practice we usually analyse a sample and notthe whole population. The standard deviation ‘s’ forthe sample is given by:s n(xi x)2n 1i 1[EURACHEM Guide, 1998]UNCERTAINTY (OF MEASUREMENT) i.e.MEASUREMENT UNCERTAINTY:‘Parameter associated with the result of ameasurement that characterises the dispersionof the values that could reasonably be attributedto the measurand. Note: The parameter maybe, for example, a standard deviation (or a givenmultiple of it), or the width of a confidenceZ scoreStandardized measure of performance, calculatedusing the participant result, assigned value andthe standard deviation for proficiency assessment[WHO, 2016].X

Executive summaryThis report presents the results of the analysisusing different figures to help laboratorymanagers and other non-specialist readersto perceive the different aspects of (i) thelaboratory performance evaluation, (ii) the wayto identify the technical problems in case of poorperformances and (iii) suggesting which solutionscan be proposed to improve the analyticalperformances. Overall, the variability aroundthe consensus value in nearly all laboratories wasvariable depending on the soil characteristic: itwas low for soil pH, medium and questionablefor organic carbon (OC), and generally highand often unsatisfactory for available P andexchangeable K. Because the laboratory’sprecision (intra-laboratory variability) was alsovariable depending on the soil characteristic, poorlaboratory performances were not related todifferences in the Standard Operating Procedures(SOPs) used. Otherwise, variability in precisioncould be related to (i) the lack of quality controlinside the laboratories, in particular the absenceof internal control samples, and (ii) the lack ofsufficient initial and ongoing professional trainingof the staff. The same observations apply tothe soil organic carbon results obtained by drycombustion. Even though this method generallyprovides better results than the oxidationmethod, the low performance of one NationalReference Laboratory confirmed that stafftraining and qualification is necessary to get highperformance.The first proficiency test of SEALNET wasorganized in 2018 with the purpose of assessingthe performance and inter-/intra-laboratoryvariability of 16 National Reference Laboratoriesin Asia. Testing soil samples were prepared withthe financial support of the International JointLaboratory ‘Impact of Rapid Land Use Changeon Soil Ecosystem Services (LMI-LUSES) from theInstitut de Recherche pour le Dévelopement (IRD,France) and the Land Development Department(LDD, Thailand), and shipped to participatinglaboratories by FAO’s Global Soil Partnership(GSP).Each laboratory received replicates of four soiltypes. In total, each laboratory was asked toanalyse 14 soil samples for soil pH (in soil waterratio1:2.5), organic carbon (using Walkley & Blackand/or dry combustion), available phosphorus(using Olsen and/or Bray 1 and/or Bray 2methods) and Exchangeable K (using NH4OAcmethod). Laboratories were not informed aboutthe nature of the samples, whose coding wasalso randomised at the purpose of preventinglaboratories to compare each others results.The GSP collected and anonymized laboratories’results before sending them to Ms. NopmaneeSuvannang and Mr. Christian Hartmann forthe statistical analysis. In order to estimatethe laboratory’s accuracy, the consensus valuewas calculated after identifying and excludingoutliers in the dataset. The variability betweenlaboratories was estimated from the standarddeviation around the consensus value whilethe performance of each laboratory wasestimated by calculating the commonly usedz-score. Robust statistic parameters (median,MADe) were calculated for information. Intralaboratory variability (precision) was estimated bycalculating the coefficient of variation around themean value of the replicates coming from a givensoil type for each laboratory.In conclusion, it is recommended that (i) alllaboratories should implement good laboratorypractices and quality control programmes,regardless to the adoption of SEALNET/GLOSOLAN SOPs; (ii) laboratories with a highperformance should be identified for the purposeof providing training to laboratories in need,and (iii) inter-laboratory comparisons shouldbe organized on a regular basis (at least twice ayear) for the purpose of monitoring laboratories’performance and of measuring the impact of stafftraining and SOPs implementation.XI

XIIWikimedia

1. IntroductionThe current report is presenting the details of: the procedures used to prepare and send thetest soil materials; the procedure of statistical analysis; the figures presenting the performance ofthe laboratories concerning accuracy andprecision; comments on the interpretation of theperformance addressed to the laboratorymanagers but also to the stakeholders thatuse the results for decision making; a conclusion on the performance andcomparability of the analytical results andfinally some recommendations for the futureinter-laboratory comparisons.Southeast Asia Laboratory Network (SEALNET)is the regional network of laboratories from thecountries of the Asian Soil Partnership (ASP), aregional section of the GSP. The objective ofPillar 5 of the GSP and of SEALNET is to helpsoil laboratories produce analytical resultsthat can be compared, wherever the soilsample was analysed inside the Region.For the GSP, the Asian Region consists of 24countries, each of which having to appoint its ownreference laboratory when participating in GSPand ASP activities. At the first SEALNET meetingorganised in November 2017 in Bogor (Indonesia),18 countries sent at least one representative oftheir national reference laboratory (see meetingreport on http://www.fao.org/3/I9063EN/i9063en.pdf). During discussions, it becameclear that, for the main soil characteristics,most of the laboratories used analyticalmethods based on the same chemical andphysical principles, but many differenceswere observed concerning the details of theanalytical procedures.Soil testingparameterTable 1. Agreed method endorsed during the first meeting oflaboratories’ managers in Bogor (Indonesia) in 2017 (* agreedmethod that was recommended).To estimate if the results coming fromdifferent laboratories could reasonably becompared, it appeared necessary to evaluatethe impact of these differences in analyticalprocedures on the final analytical result.Thus it was decided to organise an inter-laboratorycomparison by sending the same soil sub-samplesto all reference laboratories and letting themanalyse these samples according to agreedmethod (Table 1) following individual laboratoryprocedures for determination of soil pH, organiccarbon (OC), available P and exchangeable Kin order to assess their performance andcomparability on the basis of a statisticalanalysis of their results.MethodpH inwater1:2.5OCWalkley &Black*Avail POlsen P*Bray 2 PPreliminary results of the statistical andperformance analyses were presented inNovember 2018 during the second SEALNETmeeting in Bhopal (India) and during the secondGLOSOLAN meeting in Rome (Italy).1UnitAdjust the soil:water to 1:2.5and follow yourregular SOPNAFollow yourregular SOPand reportwhich methodDry combustion that you haveusedBray 1 PExch KNotedNH4OAc*percentFollow yourregular SOPand reportwhich methodthat you haveusedmg/kgUsed your regular SOPmg/kgorcmolc/kg

2. Testing material:preparation and sending2.2 Composition of the set of samples sentto each laboratoryFour soil samples coming from different locationsand having consequently different characteristicswere selected by the Central Laboratory of theLand Development Department (LDD, Ministryof Agriculture and Cooperatives, Thailand) andprepared with the technical and financial supportof the LMI-

5. Report on laboratory precision 42 5.1 Precision estimated from single cv values (Figure 33) 42 5.2 Precision estimated from mean-cv (Figure 34) 44 6. General discussion 46 6.1. Analysing the lab performance in terms of accuracy and precision 46 6.2. Analysing the performance and homogeneity of SEALNET 47 Conclusions and recommendations 49 .