New Benchmarks In Higher Education: Student Engagement In .
New Benchmarks in Higher Education:Student Engagement in Online LearningCHIN CHOO ROBINSONHALLETT HULLINGERORAL ROBERTS UNIVERSITYTULSA, OKLAHOMAABSTRACT. The increase in the adoption of Internet-related technologies foronline learning has been accompanied bya parallel, but separate, demand for greateraccountability in higher education. Measures of student engagement offer valuableindicators of educational quality, yet havebeen limited to use in on-campus settings.The authors used key engagement dimensions that the National Survey of StudentEngagement (NSSE) defined to measurestudent engagement in online courses from3 universities. Online students were modestly engaged in selected NSSE dimensionsand had a pattern of engagement that differed from on-campus students.Keywords: online education, online learning, student engagementCopyright 2008 Heldref PublicationsThere is no change in higher education more sweeping than thetransformation brought about by theadvent of the Internet and Web. Maeroff(2003) maintained that developmentsin online learning are not “just a fad”but a “sea change” (p. 2). The amalgamation of knowledge and technologypermits higher education to providelearning anytime, anyplace, and to anyone (Aggarwal & Bento, 2000; Maeroff;Pittinsky, 2003).Concurrent with the phenomenalgrowth in online learning, stakeholders in education continue to demandgreater accountability and evidence ofeffectiveness in teaching (Wilbur, 1998).Research in this area tends to focus onwhether online learning is as effective asface-to-face learning in achieving learning outcomes. Studies on the effectiveness of online learning fall into threebroad categories: (a) students’ outcomes,focused on test scores and grades; (b)student attitudes about learning; and (c)overall student satisfaction with onlinelearning. Findings largely support theview that the learning outcomes of students online are similar to those in faceto-face settings (Palloff & Pratt, 2001).Bucy (2003) noted the volume ofresearch that focuses on comparisonsbetween traditional and online coursesand concluded:Rather than using research to help replicate what is done in the traditional class-room, researchers should focus on identifying what is done well in the onlinelearning environment. Research shoulddetermine whether they (the students)are learning what we intended them tolearn—NOT whether they are learningthe same as in traditional methods. (p. 7)Objective judgment of the student’sknowledge and learning is still important for all stakeholders, including thelearner, the instructor, and the educational institution (Valenti, Cucchiarelli,& Panti, 2001). However, the evaluation of online learning needs to gobeyond these measures and consider thequality of the learning experience as awhole. Measures of student engagementoffer such an evaluation.Student engagement pertains to thetime and physical energy that studentsexpend on activities in their academicexperience (Jacobi, Astin, Ayala, 1987;Kuh, 2003). Engagement pertains to theefforts of the student to study a subject, practice, obtain feedback, analyze,and solve problems (Kuh, 2003). TheNational Survey of Student Engagement(NSSE) measures dimensions of engagement on the basis of the widely citedSeven Principles of Good Practice inUndergraduate Education by Chickeringand Gamson (Kuh, 2001). Though theNSSE was created for on-campus education, the Seven Principles of Good Practice in Undergraduate Education servesas its foundation, and the principles havebeen widely applied to online learning.November/December 2008101
Chickering and Erhmann (1996) published guidelines on how online education can be done in ways consistentwith these principles. Weiss, Knowlton,and Speck (2000) discussed the useof technology to facilitate the aims ofthe seven principles. Palloff and Pratt(2001) maintained that a sound, effective online course must abide by thesesame principles. Likewise, the findingsof Thurmond, Wambach, and Connors(2002) indicated that the principles ofgood practice in education that Chickering and Erhmann described apply to thevirtual classroom.Much as application of the principlesof good practice has been extended to theonline setting, selected dimensions of theNSSE show promise for investigatingstudent engagement in online learning. Inthe present study, we measured the levelof student engagement in online learningin undergraduate education using specific dimensions of engagement consideredto be effective educational practices:level of academic challenge, active andcollaborative learning, student–facultyinteraction, and enriching educationalexperiences (Kuh, 2001). That measurement was a first step toward the broaderapplication of student engagement, fromthe traditional classroom to the onlinelearning environment.SAMPLE AND METHODIn our survey research, students wererequired to complete an online selfadministered questionnaire relating totheir experiences as students in onlinelearning environments. The instrumentwas a modified version of the wellestablished NSSE survey, the widelyaccepted research initiative to assessstudent learning through measuring thelevel of engagement. First launched in2000, NSSE was built on indexes ofeffective educational practice that camefrom the Seven Principles of Good Practice in Undergraduate Education (Kuh,2001). The NSSE has enjoyed year-toyear growth in participation, from 276colleges and universities in 2000 to morethan 600 in 2007, with typical samplesizes of more than 100,000 students.NSSE represents a definite departurefrom the traditional approach to defining collegiate quality. Instead of test102Journal of Education for Businessscores, student engagement focuses onthe level of academic effort and quality of active and collaborative learning.Instead of indicating what resources areavailable to the student, student engagement indicates what the student doeswith the resources that are available. Itis used to decipher the type and depth ofinteraction between the faculty and student. Facilitating, mentoring, and moderating are techniques promoting studentengagement. Rather than measuring aconcern about the reputation of the college, the measure of student engagementfocuses on the quality of enrichment inthe overall educational experience (Indiana University Center for PostsecondaryResearch [IUCPR], 2003).NSSE was built on five benchmarks:level of academic challenge, active andcollaborative learning, student interactionwith faculty members, enriching educational experience, and a supportive campus environment. A supportive campusenvironment, as defined by NSSE, wasdeemed not applicable to the study. Modification of the NSSE instrument involvedanalyzing the applicability of eachresearch question to the online learningenvironment. This was done on the basisof a review of literature regarding thedeployment of Internet technology in theteaching–learning process and with careful attention to the guidelines by Chickering and Ehrmann (1996) in Implementingthe Seven Principles of Good Practice inUndergraduate EducationThe population of interest was undergraduate students from Oklahoma StateUniversity of Tulsa, Oklahoma; CapellaUniversity in Minneapolis, Minnesota;and Northeastern State University ofTahlequah, Oklahoma. All the studentswere enrolled in at least one totally onlinecourse. Our approach was a multicourseand multiuniversity strategy, which provided statistical benefits. Arbaugh andHiltz (2005) affirmed the following:Multi-course studies would provide severalmethodological benefits for ALN (Asynchronous Learning Networks) research.Two direct benefits would be increasedexternal validity and statistical power.Because ALN research has been historically reliant on many studies based onindividual courses, many of their findingsmay reflect idiosyncrasies of the instructor rather than provide accurate prescriptions for best practice in web-based coursedelivery. Multi-course studies increase thelikelihood that these instructor-uniquecharacteristics can be controlled for, thereby allowing for increased generalizabilityof findings. (p. 91)Using multicourse, multi-institutionsamples helped us to increase the sample size, providing a more completepicture, and increased external validityand statistical power. It contributed tothe generalization of findings (Arbaugh& Hiltz, 2005).A total of 225 students participated.Of these, 24 responses were incomplete.Findings were drawn from the remaining 201 respondents.RESULTSParticipants included 86 men (43%)and 115 women (57%). The typical profile of the respondent was White (84%)and less than 25 years old (85%). Morethan half (54%) of the participants wereupperclassmen; 21% were freshmen;and 25% were sophomores. The overall engagement score was 5.65 on a10-point scale. We used a variety ofdifferent scales (such as Likert-type andSemantic Differential) ranging from 4to 7 points in the questionnaire. Scoreswere converted to a 10-point numericscale to arrive at a global score. In allfour benchmarks—level of academicchallenge, student–faculty interaction,active and collaborative learning, andenriching educational experience—theengagement scores ranged from 5.13 to5.87. The 2003 NSSE survey reportedthat on a 10-point scale derived fromengagement scores, most colleges fallsomewhere between 4 and 6. NSSE considered these indexes of modest levels ofstudent engagement (IUCPR, 2003).Although the intent of the survey wasnot to focus on comparisons with oncampus engagement, the 2006 NSSEscores were used as a benchmark toillustrate the depth of engagement online(see Table 1). Online students reportedhigher levels of engagement than bothfreshmen and senior on-campus students on each of the four benchmarks.Seniors from the top 10% of NSSErespondent institutions reported higherlevels of engagement in the level of academic challenge, active and collaborative learning, and enriching educational
TABLE 1. National Survey of Student Engagement (NSSE) Benchmark Scores (2006): Online Versus On-CampusNSSE 80392,468.00IndexNSSE Top .69— 201.4714.691.2104.69— 2055.941.2055.94— 359,895.00Index0.8890.8751.0070.9510.951—Note. ACL active and collaborative learning; EEE enriching educational experience; LAC level of academic challenge; SFI student–faculty interaction. Bold are aggregate scores derived from averaging the individual NSSE engagement benchmark scores.experience. There was no differencein the level of student–faculty interaction. Compared with the freshmenfrom these highly engaging institutions, online freshmen respondents hadgreater student–faculty interaction and amore enriching educational experience.However, the level of academic challenge scored lower with online students.(IUCPR, 2006a; IUCPR, 2006b.)A summary report of the engagement factors follows. This report provides an understanding of the type anddepth of the engagement experiencein the online environment as it pertains to each key benchmark area (seeTables 2 and 3). Findings concludewith a report on subgroup distinctionsto highlight incidences where specificgroups delineated by demographics(gender, age) and academic (grade performance, major) factors appeared tobe comparatively more engaged.Level of Academic ChallengeLevel of academic challenge is whether students are putting forth enough academic effort, such as that spent studying,reading, writing, and preparing for class(Kuh, 2003). This benchmark measuredengagement in academic rigor in areasof mental activities, type and quantity ofhomework, evaluations, and academicskill development.Development of mental capacities tothink at different levels has traditionallytaken center stage in academic curricula(Barakzai & Fraser, 2005; Notar, Wilson,& Montgomery, 2005). All five levelsof mental activities—namely memorization, analysis, synthesis, making judg-ment, and application—were prevalentin the online learning environment. Students were most engaged in analyticalwork. More than half of the respondentswere at least often engaged in this way,whereas almost all respondents (96%)reported at least some engagement.This was followed by course work thatrequired the application of theories andconcepts to solve problems. Of respondents, 91% reported some, quite a bit,or very much emphasis. Memorization,synthesis, and making judgment enjoyedalmost equal emphasis. Synthesizing andorganizing ideas and information showedthe lowest incidence (38%) of high levelsof involvement.The availability of technology servesto increase the opportunities to stimulate higher order levels of thinking.First, asynchronous networks allow thelearner more time to think criticallyand reflectively, stimulating analysis,synthesis, judgment, and application.Second, multichannel communicationthrough visuals, print, and virtual experiences promote thinking. Third, learning communities that are inevitable inthe online classrooms are really communities of inquiry to advance mentalthinking (Chickering & Ehrmann, 1996;Conrad & Donaldson, 2004; Lorenzo& Moore, 2002). Duderstadt, Atkins,and Houweling (2002) affirmed, “Whenimplemented through active, inquirybased learning pedagogies, online learning can stimulate students to use higherorder skills such as problem solving,collaboration, and stimulation” (p. 75).In a typical week, the online studenthad to work on one to three problem setsthat took more than 1 hr to complete andanother one to three problem sets thatrequired less than 1 hr of work. Eachcourse typically required the student toread between one and three textbooks andbetween one and six articles. The majority of the students had to write betweenone and six reports. The latter requirement varied greatly. Whether the quantity of reading and writing assignmentsled to higher levels of learning dependson whether the materials assigned led thestudent to think and conceive persuasivearguments (IUCPR, 2003).The online classroom presented challenging standards and expectations.Often, the majority of the students had towork harder than they thought they couldto meet academic expectations. Tests andevaluations were even more demanding. The vast majority (more than 80%)of the students were challenged to dotheir best. This was an exemplary levelof engagement. When high expectations are communicated, students rise tomeet them (Reynolds, 1995). Since theInternet serves as the major platform forcommunication in the online classroom,evaluation criteria are usually published.This makes a more explicit communication of standards and expectations tospur the student’s performance to greaterheights (Chickering & Ehrmann, 1996).With the exception of speaking skills,online learning made at least some contribution to the capabilities of the students to write clearly, think critically,and analyze quantitative problems.Almost half of the students reportedthat there was very little emphasis onspeaking skills. Development of speaking skills was definitely not a strengthof the online mode of education.November/December 2008103
TABLE 2. Frequency Distribution in Percentages for Engagement Factors (n 201)BenchmarkLevel of academic challengeMental activitiesMemorizing facts, ideas, or methodsAnalyzing an idea, experience, or theorySynthesizing and organizing ideas, or experienceMaking judgments about the value of information,arguments, or methodsApplying theories or conceptsExpectations and evaluationsWorked harder than you thought you couldSkill developmentWrite clearly and effectivelySpeak clearly and effectivelyThink critically and analyticallyAnalyze quantitative problemsStudent–faculty interactionDiscussed ideas from readings or class notesDiscussed grades or assignmentsReceived prompt feedbackDiscussed career plansActive and collaborative learningWorked with other studentsTutored or taught other studentsMade a class presentation onlineVisited online library resources to meet class assignmentsWork effectively with othersEnriching educational experienceTechnology competencyUsed computer technology to analyze dataDeveloped a Web page or multimedia presentationUse computing and information technologyLife enrichmentRegular communication with other students onmatters unrelated to the courseLife enrichmentVisited online library resources, not related to classassignmentsLearn effectively on your ownWork enrichmentParticipated in online class discussionsAcquire job or work-related knowledge and skillsSolve complex real-world problemsVery littleor NeverSome orSometimesQuite a bitor OftenVery muchor Very 8339.8045.7721.3925.8722.3920.906.478.46TABLE 3. Level of Academic Challenge for Homework in Percentages (n 201)FactorNone1–34–67–10Number of assigned textbooksNumber of written papers or reports of five pages or fewerNumber of written papers or reports of six pages or moreNumber of problem sets that take more than 1 hrNumber of problem sets that take less than 1 nts were highly engaged in theareas of critical and analytical thinking. Almost 90% reported at least somelevel of involvement. This was a goodreport because the ideal curriculum104Journal of Education for Businessis one that encourages the learner tothink critically and reflect on the content with the view to apply it to dailylife (Chickering & Ehrmann, 1996).Critical thinking is a manifestation of 101.4921.8912.943.981.99an engaged environment (Conrad &Donaldson, 2004). Three quarters ofthe respondents reported the benefit ofat least some development in writingskills and quantitative analysis. Online
learning served to promote basic academic skill development, with theexception of speaking skills.Student–Faculty InteractionStudent–faculty interaction relatesto the nature and frequency of contactthat students have with their faculty.Contact includes faculty feedback anddiscussion of grades and assignments,ideas, careers, and collaborative projects (Kuh, 2003). Faculty feedback wasthe most frequent type of interactionbetween the student and instructor. Thelevel of contact was exemplary. Withthe exception of 4%, all students indicated they received feedback at leastsometimes. As many as 66% indicatedthat the level of feedback was often orvery often. Technology can be somewhat credited for this outcome becauseit provides various avenues for feedback(Chickering & Ehrmann, 1996). Promptfeedback is particularly vital for theonline student who can otherwise suffer effects of isolation and detachment(Schwartz & White, 2000).Grades and assignments were popular topics of discussion. Approximately40% of the students interacted with thefaculty on these issues often or veryoften and approximately 90% had thesediscussions at least sometimes. Onlinediscussions on readings and class noteswere at a modest level. Only a quarterof the students interacted often or veryoften on these matters. In all, 56% indicated that they sometimes had thesediscussions. How much student–faculty contact is deemed to be optimaldepends on the issue that is at the centerof the interaction. Both (a) communication about grades and assignmentsand (b) discussion of career mattersrequire occasional interaction. However, feedback and discussion of idea
illustrate the depth of engagement online (see Table 1). Online students reported higher levels of engagement than both freshmen and senior on-campus stu-dents on each of the four benchmarks. Seniors from the top 10% of NSSE respondent institutions reported higher levels of engagement in the level of aca-demic challenge, active and collabora-
Network Attached Storage VMs on NAS . 2/15/2013 Virtual Machine Workloads: The Case for New Benchmarks for NAS 10 VM-NAS I/O Stack . GPFS, WAFL, ZFS GPFS, WAFL, ZFS NAS Appliance Application NFS /SMB Current NAS Benchmarks New NAS Benchmarks Physical Machine
The Council on Higher Education (CHE) is an independent body established by the Higher Education Act, No. 101 of 1997. The CHE is the Quality Council for Higher Education. It advises the Minister of Higher Education and Training on all higher education issues and is responsible for quality assurance and promotion through the Higher Education .
The Canadian Language Benchmarks (CLB) standard is a descriptive scale of language ability in English as a Second Language (ESL) written as 12 benchmarks or reference points along a continuum from basic to advanced. The CLB standard reflects the progression of the knowledge and skills that underlie basic, intermediate and advanced ability
The Center for Internet Security (CIS) has published benchmarks for Microsoft products and services including the Microsoft Azure and Microsoft 365 Foundations Benchmarks, the Windows 10 Benchmark, and the Windows Server 2016 Benchmark. CIS benchmarks are internationally recognized as security standards for defending IT systems and data against
Education Scotland Guidance on using Benchmarks for Assessment March 2017 Education Scotland’s Curriculum for Excellence (CfE) Statement for Practitioners (Aug 2016) stated that the two key resources which support practitioners to plan learning, teaching and assessment are: Experiences and Outcomes Benchmarks
SAT College and Career Readiness Benchmarks 75% likelihood of earning at least a C in a first-semester, credit-bearing college course in a related subject Set at section level Grade-level benchmarks are based on expected student growth toward the SAT Benchmarks:
Benchmarks for literacy and numeracy should be used to support teachers’ professional judgement of achievement of a level. In other curriculum areas, Benchmarks support teachers and other practitioners to understand standards and identify children’s and young people’s next steps in learning. Evidence of progress and achievement will
Love Map Questionnaire (1) By giving honest answer to the following questions, you will get a sense of the quality of your current love maps. For the most accurate reading of how your marriage is doing on the first principle, both of you should complete the following. 1. I can name my partner’s best friend. T or F 2. I can tell you what stresses my partner is currently facing. T or F 3. I .
