National Survey On EMS EPCR Usability - NAEMT

Factors Impacting Perceptions of ePCR UsabilityFor software and product designers to improve theusability of ePCRs, it’s helpful to understand if there arespecific factors that impact the user’s perception of usability.USABILITY SCORES ARE IMPACTED BY TYPE OF TRAINING.As described on the previous page, usability variedaccording to which brand of software was being used. Ananalysis of the survey results found two other traits thatpredicted a higher overall usability score and higher scores3.74.374.66No trainingClassroom instruction,no software interactionClassroom instruction,with software interactionacross the three domains.*Trait 1: Type of TrainingThe type of training that EMS professionals received on theirePCR systems predicted who found the software usable. EMS practitioners who received no training on theirePCR rated the ePCR usability much lower than thosewho had received training: 3.7 out of 7 Those who received interactive training had thehighest usability score: 4.66The low scores were consistent across the three domainsof usefulness, information quality and interface quality.Trait 2: Social Media ExperienceTHE IMPACT OF AGEIt might be logical to assume that younger people,who grew up with smart phones and social media andmay have never used a paper-based system, wouldrate ePCR usability the highest. But the findingsweren’t uniform across all domains. Younger users(ages 20 to 25 and 26 to 30) ranked usability higherthan an older age group (46 to 50) for usefulness andinformation quality, but there were no statisticallysignificant differences in interface quality oroverall usability.To determine if experience performing basic computertasks was related to usability perceptions, the survey askedrespondents about their comfort level using word processingprograms such as Microsoft Word, performing internetsearches and using social media such as Facebook and Twitter.One trait stood out. There was a clear relationship betweenself-reported expertise in using social media and the usabilityperceptions of ePCR software. Respondents that self-identified as “novice” in usingsocial media gave their ePCR’s the lowest usabilityscore: 4.08 Those who considered themselves social media“experts” rated their ePCR usability higher: 4.8“This is strong evidence that EMS professionals whoroutinely engage in electronic social media platforms find ePCRTIP: DEVELOP IN-HOUSE EXPERTS.Every EMS agency needs ePCR users who havean interest in data and technology and becomein-house experts who can train others in using thesystem, assist at go-live, maintain a relationship withthe vendor, share system updates, educate otherpersonnel about why good data collection is important,answer questions about the ePCR and assist withtroubleshooting.These ePCR “champions” shouldn’t be expectedto do this work in addition to all of their otherresponsibilities, but should have time set aside forhelping others, especially when there are new hires orduring periods when system changes are occurring.software more usable because they are more familiar with howsoftware works and have more skills at using software for theirneeds than those who do not,” Dr. Garza said.Respondents with strong word processing skills rankedePCR usefulness and information quality, but word processingwasn’t a predictor of scores for interface or overall usability.* As with the software vendor scores, training and social media were statisticallysignificant independent of other traits, meaning that other factors such as numberof years in EMS, number of ePCRs used and the amount of time spent using theePCR system were controlled for. These traits should be viewed as valid issues thatimpact how useful EMS practitioners find ePCRs, and could be areas of focus forimproving ePCR usability.2017 National Survey on EMS ePCR Usability NAEMT.ORG7

!ht8yaMsneponoiatrtsigReSave the DateOctOber 16-20, 2017Las Vegas, NVNorth America’s largest and most internationally attended EMSconference and trade show, EMS World Expo, is hosted in partnership withthe National Association of Emergency Medical Technicians (NAEMT).EMS World Expo sets the standard in EMS education, offering the trainingEMS professionals need to do their jobs today, with the progressivecurriculum and technology that provides solutions for tomorrow.eMsWOrLdexpO.cOM

Results – ePCR TrainingTYPES OF TRAININGAdequately training personnelbecause of too much training, but I knowon ePCR technology can improveof a number that have occurred becauseproductivity, boost the accuracy ofof too little,” Jason Mitchell, MD, directorwith software interactioninformation collected and ensure userfor the Center for Health Informationsatisfaction with the software system.Technology (IT) at the American25% Instruction from a field“I have not been made aware ofany EHR [electronic health record]Academy of Family Physicians wastraining officerquoted as saying in Medical Economics .3implementation program that failedWithout adequate training, productivity can decline as practitionersspend extra time doing administrativeThe survey found that 87% ofrespondents had some trainingon their ePCR system, while13% had none.34% Classroom instructiontasks and staring at computer screensinstead of providing patient care.The survey found that 87% ofrespondents had some training on theirePCR system, while 13% had none.23% Both classroominstruction with software interactionand instruction from a field trainingofficer10% Classroom instructionon using the ePCR software with nointeraction with the softwareAndrea Downing Peck. “EHR implementation: Trainingpays dividends.” Medical Economics, July 25, 2013.3TIP: INCLUDE SOFTWARE INTERACTION AS PART OF EPCR TRAINING.Software interaction indicates respondents actually got to practice usingthe software as part of training. An analysis of the survey data by Dr. Garzaand his team found software interaction, whether in a classroom or with afield training officer, was positively associated with greater usability scores.“It’s important for providers to be able to interact with the software sothat they feel more comfortable with it once they go out to the field, and theywill feel like the software is more usable,” Garza said.2017 National Survey on EMS ePCR Usability NAEMT.ORG9

Measuring ePCR ProficiencyLet’s say an EMS agency providesePCR training to EMS personnel prior tosending them out in the field to use thesoftware. How can you tell if personnelhave understood the training and havean agreed upon level of proficiency ininteracting with the software?The survey found 38% ofparticipants said they had to be“certified” to use an ePCR softwareprogram before they were allowed toactively use the software in the field.The fact that so few EMSagencies have designated aspecific level of proficiencyfor ePRC use indicates thatTIP: FIND OUT WHO YOURTECHNOPHOBES ARE AND OFFEREXTRA TRAINING.not enough emphasis isbeing placed on quality datacollection, Dr. Garza said.The fact that so few EMS agencieshave designated a specific level ofproficiency for ePRC use indicates thatThe survey left open tointerpretation what constitutescertification. Often, it’s having tonot enough emphasis is being placed onquality data collection, Dr. Garza said.“We place a lot of importance onachieve a certain score on a test orclinical care, as we should, and wedemonstrate they can complete certainmake sure everybody is trained totasks after a training class.a certain standard through testingand certification,” he said. “When youstart getting into other EMS tasks itquickly falls off. What these numberstell me is we’re not paying as muchattention to the data. If you have arobust quality improvement program,you will be using data day in and dayout to make improvements. By lookingat your data every day, you have abetter understanding of where yourshortcomings are. It’s a bigger issuethan just the data entry. We have toestablish a culture of quality on theback end that is keeping an eye on thedata and making sure EMS personnelare capturing good data.”10NAEMT.ORG 2017 National Survey on EMS ePCR UsabilityAccording to survey responses,EMS practitioners described varyinglevels of experience with basiccomputer skills.59%described themselvesas “experienced” with using wordprocessing programs51%as experienced withInternet searches43%were experienced withsocial mediaComfort with basic computerskills was associated with higherscores on ePCR usability, accordingto an analysis of the survey data.But what about those staffmembers who are very inexperienced with basic computer skills?Identifying them and offering extratraining will help avoid productivityand accuracy issues with your ePCR.

ConclusionSpiraling healthcare costs andthe massive financial and quality oflife burden of chronic disease in theAmerican population gave rise tothe healthcare reform movement. Akey aspect of reform is transformingthe healthcare system from one thatincentivizes providing more and morehealthcare services (volume) to onethat encourages providing healthcareservices with proven value.To reach its potential as an integralpart of the healthcare system, EMS alsomust show value.Payment reform – to allow EMS to getpaid for patient care rather than just patient transportation – requires that EMSgather hard numbers to show that thework that EMTs and paramedics do on adaily basis is money well spent. To build aThis survey looked specifically at theEMS practitioners may also becase for support from payers for innova-usability of the software used to collecthelped through courses that boost theirtive services such as mobile integratedthe data, which has the potential tooverall familiarity with using computers,healthcare and community paramedicine,impact accuracy, timeliness, reliabilitysuch as community college courses intreat-and-refer and alternative destinationand other domains.word processing or typing.Findings demonstrate need forimprovementEducation on the importanceof dataprograms, EMS needs good data as well.Factors that impact quality ofdata collectedThe findings of the survey demonstrateTraining should of course includeThere are many factors that impact datathat the six most commonly used ePCRthe nuts and bolts on how to use thequality, including accuracy, timeliness,brands earned an average score ofePCR system to capture quality data.completeness, relevance, reliability4.5 out of 7 for usability – fair, butBut ePCR training and EMS educationand consistency. Initiatives such asnot great.overall needs to include not just thethe National EMS Information SystemSoftware vendors need to work with“how” but the “why” – why collecting(NEMSIS) Data Dictionary address theEMS practitioners on improving theaccurate, timely, complete, relevant,consistency component by developinguser experience. To measure usability,reliable and consistent data mattersa common vocabulary of terms that allePCR vendors should engage the EMSin a healthcare system increasinglyEMS agencies can use for data collection,community with usability studies asexpected, and rewarded, forallowing results to be compared acrossnew software is developed to improveshowing value.regions, states and nationally.their products for the end-user.Rather than seeing data collection asEMS agencies also have a rolea chore or another box to check, EMSCompass Project (which developedin improving usability perceptions,practitioners need to understand theEMS clinical performance measures)namely through enhanced training. EMSrole of data in improving operations,and the MIH Measures Developmentpractitioners should offer interactiveensuring limited resources areGroup (which developed outcomestraining on ePCR systems prior tospent wisely, enhancing patient andmeasures for MIH-CP programs),use in the field, and perhaps periodicpractitioner safety, and in improvinghave addressed components such asrefreshers to ensure system updates orthe quality of care for patients.relevance and completeness.upgrades are understood and utilized.Other groups, such as the EMS2017 National Survey on EMS ePCR Usability NAEMT.ORG11

ABOUT NAEMTFormed in 1975 and more than 55,000 members strong, the National Association ofEmergency Medical Technicians (NAEMT) is the only national association representingthe professional interests of all emergency and mobile healthcare practitioners,including emergency medical technicians, advanced emergency medical technicians,emergency medical responders, paramedics, advanced practice paramedics, criticalcare paramedics, flight paramedics, community paramedics, and mobile integratedhealthcare practitioners. NAEMT members work in all sectors of EMS, includinggovernment agencies, fire departments, hospital-based ambulance services, privatecompanies, industrial and special operations settings, and in the military.NAEMT.ORGNATIONAL ASSOCIATION OFEMERGENCY MEDICAL EMT

specific factors that impact the user's perception of usability. As described on the previous page, usability varied according to which brand of software was being used. An analysis of the survey results found two other traits that predicted a higher overall usability score and higher scores across the three domains.* Trait 1: Type of Training