Competency Assessment How, Who, What, When And Why
Competency AssessmentWho, What, When, Why andHowDonna D CastelloneMS, MASCP, MT(ASCP)SHQuality ManagerNew York Presbyterian/Columbia Medical CenterDcastell@nyp.org
New York Presbyterian-ColumbiaMedical CenterDcastell@nyp.org
Learning Objectives Define CompetencyExplain difference between training andcompetencyList the elements required for competencyassessmentDiscuss who can perform competencyassessments
Competency See Dooneone Teachone!
Most Commonly Cited Deficiencies 2016 College of American Pathologists. All rights reserved.7
Quality Management System QSE: Personnel“Obtaining and retaining an adequate number ofqualified, well trained, and competent laboratory staff toperform and manage the activities of the laboratory.”GP26-A4
Quality Management System The effectiveness of training and competence directlyaffects the laboratory’s ability to meet it’s quality goalsand objectives. Competence – ongoing assessment is required to verify jobperformance Must be documented and different from training
Why?Prevent: Errors Non-Standard work Complacency Not following SOP – short cuts developed by personnelCaused by: Loss of familiarity of processes or SOP’s Rotation through other labs Working in only one section of the lab
Who Requires Competency Assessment?Everyone! CMS AABB CAP FDA FACT Joint Commission Stateregulations
CLIA Considered a minimum set of guidelines for quality All clinical labs in the USA must meet these guidelines inorder to receive Medicare and Medicaid re-imbursement Most labs try to take beyond and have a QualityManagement System
Define Competency – The ability of personnel toapply their skill, knowledge and experienceto perform their laboratory duties correctly. Assessment – Is used to ensure that thelaboratory personnel are fulfilling theirduties as by federal regulation.Centers for Medicare and Medicaid Services
Proficiency Testing vs. Performance Appraisal Proficiency Testing (PT) Assesses the laboratory’s ability toperform accurate and reliable testing by the testing of unknownsamples sent to the lab by a CMS approved PT program. Performance appraisal evaluates general job/position performanceand is more subjective. May include competency Competency assessment ensures conformance to standardprocedures and is objective and binary (e.g. competence observedor not).
Why?One thing worse than training employees andlosing them is not training them and keepingthem.”“-Ed Metcalf
What Are the Requirements? GEN.55500 The competency of each person to perform his/her assigned duties is assessed. Evidence of Compliance: Records of competency assessment for new and existing employees reflecting the specific skillsassessed, the method of evaluation. A laboratory must evaluate and document the competency of all testing personnel for each testsystem. A TEST SYSTEM is the process that includes pre-analytic, analytic, and post-analytic steps usedto produce a test result or set of results. Documentation of these elements, including observation of test performance, results reporting,instrument maintenance, review of worksheets, recording QC, performance of PT, and demonstrationof taking appropriate corrective actions are examples of daily activities that can be used todemonstrate competency. If elements of competency are assessed by routine supervisory review, the competency procedure mustoutline how this routine review is used to evaluate competency. Competency assessment by routinesupervisory review may be documented by a checklist. For nonwaived test systems, six elements must be assessed annually. For waived test systems, thelaboratory may select which elements to assess.
Waived tests Waived tests require minimal technical competency and theyyield low risk in the event of an erroneous result. Any individual can perform a waived test in a physician officeor their home and no educational or professional training isrequired. If the test is performed in a physician’s office, the officemust apply for a CLIA certificate and the facility thatperforms the test must comply with all the manufacturerrecommendations. CLIA doesn’t demand any other regulations for waived tests.
Certificate of Waiver (CoW) Enrollin the CLIA program Pay biennial certificate fees Only perform tests categorized aswaived Not subject to routine inspections Must follow manufacturer’sinstructions
Issues with waived testing In the waived test environment, the methodology or the instrument used isvery good. It's in the preanalytic and postanalytic phases—for example, human errorsthat occur while the test sample is being gathered, or after the test has beenrun—that industry-wide, no matter what the test classification is, there areissues involved with quality. Most laboratories in the United States now are using waived testing devicesthat have pre-analytic and post-analytic problems These sites are generating problem data that are being used in the diagnosisand treatment of patients.
If my laboratory only performs waivedtesting, do I need written policies forassessing personnel competency? CLIA does not require policies for assessing personnel competency for waivedtesting. Even though CLIA has no specific requirements for personnel performingwaived testing, you need to ensure that patient testing results are correct toassist in making an accurate patient diagnosis. You will need to ensure that testing personnel are following allmanufacturers’ instructions. Testing personnel who are properly trained and performing the test correctlywill aid the physician/provider in making an accurate patient diagnosis. If your laboratory is accredited, you may need to consult your accreditingorganization’s standards.
Non waived tests Non-waived tests are defined as either moderate or high complexity. Unique educational and professional experience are required toperform non-waived tests. CLIA and the Food and Drug Administration(FDA) determines the complexity of the test, using specific criteria. Healthcare facilities that perform non-waived tests must obtain a CLIAcertificate and abide by CLIA regulations. In addition, the facility must be inspected periodically and provideproof that they comply with the CLIA quality requirements. All Laboratory Developed Tests (LDT), and non-FDA approved tests areclassified as high complexity tests and CLIA requirements are morerigorous for these kinds of tests.
What happens when you adjust how atest is performed. Devices used outside of the manufacturer’s requirementsare considered to be test modification/off label use.This is not a new CLIA regulation!Test modification/off label useAny change to a test system/device or manufacturer’sinstructions or intended use that affects the test’sperformance specifications for accuracy, precision,sensitivity or specificity.Modified tests become high complexity tests under CLIA
Competency RequirementsThe following 6 elements are the minimal requirements1.Direct observation (handling, preparation, processing, testing)2.Monitoring the recording and reporting of testing3.Review of intermediate test results or worksheets, QC, records, PT results, and PM records4. Direct observation of performance of instrument maintenance and function checks5. Assessment of problem solving skills, case studies, quizes6. Assessment of test performance using previously tested specimens, blind specimens andexternal proficiency samples
New York State: (in case you performtesting from NYS)7.NYS has added a 7th. Requirement of observationof compliance with safety protocols8. “Assessment of competency of any delegatedsupervisory functions” – recently added by NYS
Training vs. CompetencyTraining Occurs before patienttesting beginsUsually once unlessemployee fails successfuldemonstration of skill totrainer and retrainingrequireDoes not require use of sixcompetency assessmentelementsCompetency Occurs after independentpatient testing beginsDoes require use of sixcompetency assessmentelements for non-waivedtestingd 17
TrainingChecklistTRAINING Leukocyte Alkaline Phosphatase (LAP)NAME:DATE:TRAINER:Training completed: byI understand the procedure and feel confident to perform thetesting.Sign DateTrainingUnderstands the intended use of the test.Overview of principle of procedureSpecimen requirements: collection and storagePPE requirementsReagent preparationPull pending log and worksheetAdditional materials requiredReview of the SOPQuality control samplesPreparation of blood smearStorage and stability of smearsDrying the smearFixing the smearsStaining the smearsUsing the microscopeIdentifying the granulesScoring the samplesReference rangeRecording of resultsInterpretation of ResultsReporting results in CernerTraineeTrainer
New York-Presbyterian Hospital / Columbia University Medical CenterDEPARTMENT OF CLINICAL LABORATORIESANNUAL COMPETENCY ASSESSMENT CHECKLISTEmployee Name: Test EmployeeDept:LaboratoryYear:Policy: HR-9iCompetency Assessment is used to ensure that the laboratory personnel are fulfilling their duties as required by the department and federal regulation.Method of assessment:1) direct observation;2) observation for compliance with safety protocols;3) periodic review of work product for compliance with standard operating procedures and applicable workload limits;4) monitoring the recording and reporting of test results;5) direct observation of performance of instrument maintenance and function checks;6) assessment of test performance through testing of previously analyzed specimens, internal blind or external proficiency testing samples;7) assessment of problem solving skills.8) assessment of delegated functionsEvaluate the competency of staff for all tasks for which he/she is responsible. Indicate evaluation for each task as: YES (competent) or NO (not competent). Areas that the employee is notresponsible will be marked NA (not applicable). If the staff is not competent, a corrective or remedial action should be initiated and documented and filed with this document.1) Direct ObservationRoche Analyzer[ ] Check If Task Is Not ApplicableAssessedBy:Date:2) Safety Compliance3) Compliance w/ SOP5) InstrumentMaintenance4) Review test results6) Unknown/PT Samples7) Problem SolvingSpecimen requirements.Sample Handling and Stability[ X ] YES [ ] NO[ X ] YES [ ] NO[ X ] YES [ ] NO[ X ] YES [ ] NO[ X ] YES [ ] NA[ X ] YES [ ] NA[ X ] YES [ ] NO[ X ] YES [ ] NOProcessingMeasurement methodology[ X ] YES [ ] NO[ X ] YES [ ] NO[ X ] YES [ ] NO[ X ] YES [ ] NO[ X ] YES [ ] NA[ X ] YES [ ] NA[ X ] YES [ ] NO[ X ] YES [ ] NOQuality Control[ X ] YES [ ] NOReporting Results[ X ] YES [ ] NOReagent Handling and StorageMaintenance[ X ] YES [ ] NO[ X ] YES [ ] NO[ X ] YES [ ] NO[ X ] YES [ ] NO[ X ] YES [ ] NO[ X ] YES [ ] NO[ X ] YES [ ] NO[ X ] YES [ ] NA IfYES,date ofevent:Accession #:[ X ] YES [ ] NA[ X ] YES [ ] NA[ X ] YES [ ] NA[ X ] YES [ ] NA[ ] YES [ ] NA IfYES, date of eventscore:[ X ] YES [ ] NO[ X ] YES [ ] NO[ X ] YES [ ] NO[ X ] YES [ ] NODelegated QC Review[ X ] YES [ ] NADelegated Staff Training[ X ] YES [ ] NA[ X ] YES [ ] NADelegated Regulatory TasksDelegated SchedulingBy:Date:Specimen requirements.[ ] YES [ ] NO[ ] YES [ ] NOSample Handling and StabilityProcessingMeasurement methodologyQuality ControlReporting ResultsReagent Handling and StorageMaintenanceDelegated QC ReviewDelegated Staff TrainingDelegated Regulatory TasksDelegated Scheduling[[[[[[[[[[[[[[[[[[[[[1) Direct ObservationMiscellaneous TasksCentrifugesPipettesAtttestation and Signatures[ X ] YES [ ] NA5) InstrumentMaintenance[ ] YES [ ] NARadiometer ABL 800 Blood Gas Analyzer[ ] Check If Task Is Not ApplicableAssessed8) Delegated functionsTask/ElementOperationOperation] YES] YES] YES] YES] YES] YES] YES[[[[[[[] NO] NO] NO] NO] NO] NO] NO2) Safety Compliance[ ] YES [ ] NO3) Compliance w/ SOP] YES] YES] YES] YES] YES] YES] YES[[[[[[[] NO] NO] NO] NO] NO] NO] NO4) Review test results[ ] YES [ X ] NA IfYES, date of event:Accession #:] YES] YES] YES] YES] YES] YES] YES[[[[[[[] NA] NA] NA] NA] NA] NA] NO6) Unknown/PT Samples[ ] YES [ ] NA IfYES, date of eventscore:[[[[[[[] YES] YES] YES] YES] YES] YES] YES[[[[[[[] NO] NO] NO] NO] NO] NO] NO[ X ] YES[ X ] YES[ X ] YES[ X ] YES1) Direct Observation2) Safety Compliance3) Compliance w/ SOP4) Review test results[ X ] YES [ ] NO[ X ] YES [ ] NO[ X ] YES [ ] NO[ X ] YES [ ] NA[ X ] YES [ ] NO[ X ] YES [ ] NONANA5) InstrumentMaintenance[ X ] YES [ ] NA[ X ] YES [ ] NAEmployee Signature / Date8) Delegated functions[ ] YES [ ] NODateSupervisor Signature / Date[[[[] NA] NA] NA] NA6) Unknown/PT Samples7) Problem Solving8) Delegated functionsNANA[ X ] YES [ ] NO[ X ] YES [ ] NO[ X ] YES [ ] NO[ X ] YES [ ] NO"I have read the above competency assessment. I understand that by signing this statement, I agree that I have been trained in the areas noted in this assessment and I am approved to perform the tests/procedures independently. I also feel competent to perform this list of tests/procedures, and processes independently as indicated by the above assessment."Employee Signature7) Problem Solving
Requirements: Direct observation of routine patient testing -Watch employee perform routine work processes and proceduresincluding patient preparation, specimen handling, processing, andtesting.use of a checklist as guidance for evaluation and to document thatall required procedural steps are completed. Monitoring the recording and reporting of test results -Review worksheets, computer printouts, or manual test logs forresults or entries that are incorrect or incomplete.The worksheets, logs, and printouts should be compared to final testreports, evaluated turnaround times, and for compliance withpolicies and procedures.
Requirements Review of intermediate test results, records, and proficiency testingresults -Review testing documentation, to include test results or worksheets,documentation forms, QC logs, proficiency testing documentation,and other applicable documentation for completion of properpolicies and procedure. Documentation of review will includefollow up of corrective action related to problems in the laboratory. Direct observation of instrument maintenance and function checks -Watch employee perform instrument maintenance and functionchecks as compared to documentation for completeness andcorrectness of process and expected outcome.Record observations on checklist for documentation of inclusion andcompletion of all steps.
Requirements: Assessment of test performance through testing previously analyzedspecimens, internal blind testing samples or external proficiencytesting samples -Assessment of previously tested samples; internal blind samplesor external proficiency testing samples.Use of checklist for documentation of blind sample performance. Assessment of problem solving skills -Include review of problem logs, incident reports, and QC failures;Review corrective actions employed to resolve the problem.Personnel interviews (staff narrative) may also be included fordocumentation.Exams (include pre-analytical, analytical and post analytical)
In God we trust, all others must providedata The easiest way to document problem solving iswith a quiz It should cover the elements of specimenrequirements, sample handling and stability,processing, methodology, QC and reporting results Case studies work very well
Frequency At least semiannually during the first year the individualtests patient specimens At least annually thereafter unless test methodology orinstrumentation changes – then 6 months and annually Prior to reporting patient test results, the individual'sperformance must be reevaluated to include the use of thenew test methodology or instrumentation
What is the timeline?If you are hired on January 1 You are trained in hematology from January 2- March 2 And you are assessed competency on performing a CBC on March3rd When is your 6 month competency assessment performed:a. June 1 b. June 3c. September 3d. Anytime before the end of the year
Answer: C SeptemberNew employees Competency assessment separate from training Assessed twice in the first yearAnnual assessment Documentation for staff that work on all shifts Tests being evaluated Is there any distinction made for testing that may be providedon day shift vs after hours Special testing
Case studyAn employee goes out on maternity leave and isgone for a period of 6 months She comes back and immediately begins working onthe bench and reporting out results Is this acceptable? She has worked in her position previously for 10years
What do the regulations say? Not much There are currently no defined regulatory or accreditation requirements forthis situation It is a defined laboratory policy-whatever you state in your policy, you mustfollow NYP defines this as 3 months:1. Re-assess competency2. if there is a failure, person will be retrained3. then re-assessment of competency If an employee has minimal previous experience, or they are uncomfortableat their workstation, a full training and competency should be performed.
Case Study Inspector is conducting a tracer audit of a sample performedon April 10, 2019 Notes that the technologist that performed and signed off onthe testing is a relatively new employee Asked to see their files Competency for this procedure was assessed on April 29,2019- employee was trained on April 1, 2019 Is this within compliance
NO! CAP/CLIA requirementGen. 55500 competency assessment – the competency ofeach person to perform his/her assigned duties isassessed. This must be addressed following trainingbefore the person performs patient testing.College of American Pathologists – ref. CLIA elements
Case study In the Special Coagulation laboratory, the StarEvolution and the Sta-Max are used for testing The technologists are assessed for competency onfactor assays, AT testing, von Willebrand testing,mixing studies Since they are on the same platform is thisnecessary?
Yep! May I combine for competency purposes, all tests performed simultaneouslyon the same testing platform?As long as there are no unique aspects, problems or proceduresassociated with any test on the testing platform, all tests performedsimultaneously on the same testing platform may be combined. However, any test with unique aspects, problems or procedures withinthe same testing platform should be assessed separately to ensure thatstaff maintain their competency to report test results promptly,accurately and proficiently.So: Factor assays are clot basedAT is a chromogenic assayMixing study requires intervention; incubation; manual dilutionVon Willebrand testing is complex and immunoturbidimetric assayAll different methodologies but on the same platform!
Case study Inspector is reviewing competency assessment forunknown or proficiency testing sample It is filled out as: YesAugust 12, 2019100% score The inspector doesn’t want to accept this All the boxes are filled out What is wrong
If it’s not documented it’s not done! Insufficient information Information must be retrievable “Documentation of the event used for the assessment of the staff’stest performance must contain enough specific detail so that theevaluation can be substantiated” Should have PT name (challenge 1,2 or 3) date testing wasperformed/signed by tech and score. Or if a previously analyzed specimens were used- must indicate thedate and the result of both the original testing and testing performedby the staff member. If no supporting data is attached to the documentThis documentation shows the directive was fulfilled
What happens if it doesn’t match You cannot accept it as a fulfillment of that requirement Check for error due to sample instability Repeat the exercise If results still do not match what do you do? GEN.57000 Competency Corrective ActionIf an employee fails
GEN.55500 The competency of each person to perform his/her assigned duties is assessed. Evidence of Compliance: Records of competency assessment for new and existing employees reflecting the specific skills assessed, the method of evaluation. A laboratory must evaluate and document the competency of all testing personnel for each test
UNESCO EDUCATION SECTOR 5 Systems thinking competency Anticipatory competency Normative competency Strategic competency Collaboration competency Critical thinking Self-awareness competency Integrated problem-solving competency Promotion of key competencies for sustainability ESD enables all individuals to contribute to achieving the SDGs by equipping them
1 Introduction 1 1.1 Background of the Study 1 1.1.1 Definition and meaning of Competency 2 1.1.2 History and Evolution of Competencies 8 1.1.3 Competencies for Sales Staff 12 1.1.4 The Sales Competency Model 19 1.1.5 ASTD Competency Model 20 1.1.6 Competency Mapping 24 1.1.7 Model of Competency Mapping 27
Core Competency Area 5: Observation and Assessment 43 Core Competency Area 6: Diversity and Dual Language Learners 47 Core Competency Area 7: Family and Community Relationships 50 Core Competency Area 8: Health, Safety, and Nutrition 54 Core Competency Area 9: Professionalism and Ethics 59
1. The Competency Catalog serves as immediate reference and guide of HRMD to support the following activities: a) Recruitment and Selection Development of Competency-based assessment tools Development of Competency-based Job Descriptions b) Development of Competency-based Performance Management System by building a fair
possible resolutions for development of a viable Army-wide program including the Demonstration Competency Assessment Program (DCAP), which is a prototype for Army-wide competency assessment. The second phase extends the feasibility investigation through development of five Military Occupational Specialties (MOS) competency assessments as well
Competency Model, the Supervisory Competency Model, and now the General Employee Competency Model. Each of the competency models were developed specifically for public sector employees, and outline the specific skills and behaviors required for success on the job for the group of employees for which each model was developed. The finalized .
2.1 Competency Dictionary; in which competencies are described according to a specific competency model. The Competency dictionary is established according to the following format: Competency clusters: Represent a group of competencies that are naturally linked with one another as well as specific areas in HR.
COMPONENTS OF NURSING COMPETENCY In order to make nursing competency measurable, since it is a holistic and integrated concept, many research-ers seek to identify its main components. Matsutani et al.8 analyzed the definitions, attributes, elements and structure of nursing competency by reviewing articles in English about nursing competency.
