Use And Users Of Electronic Library Resources
Use and Users of ElectronicLibrary Resources:An Overview and Analysis ofRecent Research Studiesby Carol TenopirWith the assistance of Brenda Hitchcock and Ashley PillowAugust 2003Council on Library and Information ResourcesWashington, D.C.
iiAbout the AuthorsCarol Tenopir is a professor at the School of Information Sciences at the University of Tennessee,Knoxville. She is the author of four books, including, most recently, Towards Electronic Journals: Realities forScientists, Librarians and Publishers, coauthored with Donald W. King (Washington D.C.: Special LibrariesAssociation, 2000). She has published more than 200 journal articles, is a frequent speaker at professionalconferences, and since 1983 has written the “Online Databases” column for Library Journal. She is therecipient of the 2002 American Society for Information Science & Technology, Research Award (forlifetime achievement in research). Ms. Tenopir holds a doctoral degree in Library and Information Sciencefrom the University of Illinois.Brenda Hitchcock has a B.S. from the University of Connecticut and an M.S. from Michigan StateUniversity. Currently, she is working on a master’s degree in the School of Information Science at theUniversity of Tennessee and she plans to become a school media specialist.Ashley Pillow is a graduate student in the School of Information Sciences at the University of Tennessee.Her interests include information policy, electronic journals, and information access and retrieval.Published by:Council on Library and Information Resources1755 Massachusetts Avenue, NW, Suite 500Washington, DC 20036Web site at http://www.clir.orgCopyright 2003 by the Council on Library and Information Resources. No part of this publication may be reproduced or transcribedin any form without permission of the publisher. Requests for reproduction should be submitted to the Director of Communicationsat the Council on Library and Information Resources.
iiiContentsExecutive Summary.iv1. Overview . 11.1 Introduction. 11.2 Report Outline. . 22. Tier 1 Studies. 32.1 Overview . 32.1.1 SuperJournal. 42.1.2 Digital Library Federation/Council on Library andInformation Resources/Outsell . 42.1.3 HighWire/eJUSt. 42.1.4 Pew Internet and American Life (OCLC/Harris, andUrban Libraries Council). 42.1.5 OhioLINK . 52.1.6 Tenopir and King Studies . 52.1.7 LibQUAL . 52.1.8 JSTOR. 62.2 Participants. 62.3 Methods . 62.4 Tier 1 Analysis. 112.4.1 SuperJournal. 112.4.2 DLF/CLIR/Outsell. 132.4.3 HighWire eJUSt . 142.4.4 Pew Studies/OCLC-Harris/Urban Libraries Council . 162.4.5 OhioLINK . 192.4.6 Tenopir and King. . 202.4.7 LibQUAL . 212.4.8 JSTOR. 233. Tier 2 Analysis . 253.1 Differences in Behavior or Preferences that can be Explained byDifferences Among Users . 283.2 Information Seeking Behavior and Preferences. 313.3 Perceived Advantages of Electronic Resources and Preferences. 353.4 Problems or Concerns with Electronic Resources . 363.5 Library Policies and Financial Concerns. 393.6 Summary of Tier 2 . 424. Reviews of the Literature and Methods. 435. Conclusions . 446. Bibliography . 476.1 TIER 1 . 476.1.1 SuperJournal . 476.1.2 DLF/CLIR/Outsell. 476.1.3 eJUST/HighWire. 486.1.4 Pew/OCLC-Harris Survey/Urban Library Council . 486.1.5 OhioLINK . 486.1.6 Tenopir and King . 496.1.7 LibQUAL . 516.1.8 JSTOR. 546.2 TIER 2 . 556.3 Methodology and Literature Reviews. 64
ivExecutive SummaryIn the last several years, many research studies have focused on how people use electronic resources or on their feelings about electronic and printresources in the library. These usage studies draw many conclusions aboutthe behavior and preferences of library users, although sometimes the conclusions are contradictory or unclear. This report for the Council on Libraryand Information Resources (CLIR) summarizes and analyzes more than200 recent research publications that focus on the use of electronic libraryresources and were published between 1995 and 2003. Eight major ongoing studies (each with multiple publications) are identified as Tier 1 studiesand are analyzed in detail, while about 100 smaller-scale studies are classified as Tier 2 studies and are examined together.The studies use a variety of research methods, including observation,surveys, interviews, experiments, and transaction log analysis. Some surveys or interviews ask questions about preference, including how users feelabout the library or about specific media; others ask questions that provideinformation on user behavior. Observations, experiments, and logs alsoshow what users do, but do not always reveal preferences or motivations.Each of these methods allows different types of conclusions and it is onlywhen they are taken together that we can get a full picture of what usersactually do, why they do it, what they would prefer, and what they arelikely to do in the future.The Tier 1 and Tier 2 studies make several valid conclusions that shedlight on user behavior with electronic resources. They include the following: Both faculty and students use and like electronic resources and mostreadily adopt them if the sources are perceived as convenient, relevant,and time saving to their natural workflow. Experts in different subject disciplines (work fields) have different usage patterns and preferences for print or electronic. There is no oneright solution for services or system design for every subject discipline. Print is still used for some reading and is part of research in almostevery discipline. It is considered important in certain disciplines, especially in the humanities. Print remains the most popular medium for books; e-book use is still inthe very early stages. Most e-journal users still print out articles that are judged useful—so aprinting format such as PDF is popular. Subject experts use hyperlinks to view related articles; students’ use ofhyperlinks is less clear. Browsing a small number of core journals is important (in print or electronic forms), especially for subject experts and for current awarenesssearching.
v Searching by topic in an article database is important for all other purposes.Users will read articles from a wide variety of journal titles and sourcesif available to them, although most of the readings come from relativelyfew journals.Personal subscriptions to journals continue to decrease, so users relymore on electronic subscriptions subsidized by the library and on the Internet.Most journal article readings are of articles within their first year of publication, but a sizeable minority of readings come from materials that areolder than one year.College and high school students use the Internet more than the libraryfor research, and many believe they are more expert at searching thantheir teachers.Students exercise some quality judgments about materials they retrievefrom the Internet, but those quality judgments may not exactly match faculty members’ criteria for quality.
vi
Use and Users of Electronic Lbrary ResourcesI. Overview1.1 IntroductionLibraries of all sizes and types are embracing digital collections,although most libraries will continue to offer both print anddigital collections for many years to come. New purchasesand purchases of journals, magazines, and abstracting and indexingservices are heavily weighted toward digital, while digital books (ebooks) are only beginning to become a presence in library collections.Libraries prefer digital collections for many reasons, including,but not limited to, the following: digital journals can be linked fromand to indexing and abstracting databases; access can be from theuser’s home, office, or dormitory whether or not the physical libraryis open; the library can get usage statistics that are not available forprint collections; and digital collections save space and are relativelyeasy to maintain. When total processing and space costs are takeninto account, electronic collections may also result in some overallreductions in library costs (Montgomery and King 2002).Such a dramatic switch from print collections to digital collections has an impact on library users and users’ perceptions of thelibrary. Many researchers have attempted to predict or measure thatimpact through surveys, transaction log analysis, and other researchtechniques. Librarians would like to be able to use the informationand conclusions generated by the many research studies, especiallybecause it is time consuming to conduct good research on their ownand because the best measures of impact come after decisions are already made and collections are converted. Unfortunately, the conclusions of various studies sometimes seem contradictory, and it may bedifficult to judge which research studies offer valid and reliable findings. The opinion literature outnumbers the research literature, and itmay be a challenge to distinguish fact from opinion.The purpose of this report for the Council on Library and Information Resources is to help librarians identify reliable research studies, to provide a synopsis of the good studies, and to present an analysis of conclusions. A subtitle of the report might well be the same asthe CLIR symposium held March 28, 2003, “What Are Users TellingUs?” Or, “What do user studies tell us about how and why library1
2Carol Tenopirconstituents of all types use digital library resources and are likelyto use them in the future?” The goal of this report is to provide information that librarians can use to make important decisions aboutcollections, services, and product design. Also relevant to this topicis CLIR’s January 2002 report “Usage and Usability Assessment: Library Practices and Concerns” by Denise Troll Covey.Although this introduction refers to the resources as digital resources or digital libraries, the less precise, but more commonly usedterms electronic resources or electronic libraries will be used throughoutas synonyms.1.2 Report OutlineHundreds of recent publications focus on how users interact withor how they feel about electronic library resources. It is important,therefore, to state clear parameters of what is included (and what hasbeen excluded) in this report. Only publications or reports of studiesthat meet the following parameters are included and analyzed: Studies must focus on the use of both electronic resources and libraries (electronic resources through the library, in addition to thelibrary, or in comparison with the library). Studies that are mostlyabout the Internet, but include a substantive section on libraries (the Pew studies, for example) or those that are mostly aboutlibraries in general, but include a substantive section on digitallibraries (the LibQUAL studies, for example) are also included.Internet use studies that do not focus on libraries are excluded. Studies or surveys that focus solely on librarians, library staff, library Web sites, or publishers are excluded; only those that studylibrary patrons are included. Studies that are limited to the behavior of authors rather thanreaders are excluded. Only research studies are included. Opinion pieces or descriptionsof how a library converted their print collections to digital collections are excluded. A wide variety of research methods are covered (including surveys, transaction log analysis, experimental). Because differentkinds of research methods allow different types of conclusions tobe drawn, this report describes the research method used in studies and what types of conclusions made by the researchers arevalid in accordance with the method. Studies are restricted to those conducted since 1995, or a post-Webworld. Some studies compare recent findings with past studies(for example, the Tenopir and King studies), so they may addresshow usage patterns have changed with the advent of electronicresources, but the main focus remains user behavior in an increasingly digital age. Poorly conducted research from which valid conclusions cannotbe drawn is excluded.
Use and Users of Electronic Lbrary ResourcesApplying the foregoing parameters resulted in a pool of morethan 200 individual research publications. Some publications describe different phases or parts of large, and often ongoing, researchprojects. A further distinction was made to separate these large orongoing studies from the more limited studies and to describe eachmajor study as a whole, rather than as separate publication parts.This led to a distinction between “Tier 1” and “Tier 2” studies.Tier 1 studies are those major studies that have many publications, sometimes by many different authors. The studies involvehundreds or thousands of subjects over multiple workplaces, workroles, or subject disciplines. Many important conclusions can bedrawn from each of these studies and they are typically widely reported and discussed in the library community. Each Tier 1 study isactually a group of studies conducted by a research team. Tier 1 studies are discussed in the greatest detail since they may use multiplemethods and provide, at times, complex findings.The designation as a Tier 1 study was intentionally highly selective. Only eight user studies (actually, groups of studies) weredesignated as Tier 1 studies, but they represent nearly 100 individualarticles or reports. Additionally, nearly that many other publicationsare designated Tier 2 studies. Tier 2 studies are not of lesser qualitythan Tier 1 studies; they are just typically smaller in scale or are onetime projects. Tier 2 studies may involve only dozens or hundredsof subjects. They may focus on a single workplace (for example, asingle college campus). They provide valuable insights into libraryuser behavior, but are best taken together as a whole to reach generalconclusions.In addition to Tier 1 and Tier 2 studies, selected related materialsare briefly described and are included in the bibliography. These include bibliographies of writings about users of digital library materials and several important methods papers.2. Tier 1 Studies2.1 OverviewEight groups of studies were identified as “Tier 1” or major recentresearch studies on how people use electronic library resources.Tier 1 studies are (in no particular order):1. SuperJournal2. Digital Library Federation/Council on Library and InformationResources/Outsell (DLF/CLIR/Outsell)3. HighWire/eJUSt4. Pew Internet and American Life (with comparison to OCLC/Harris and Urban Libraries Council)5. OhioLINK6. Tenopir and King studies7. LibQUAL 8. JSTOR studies3
4Carol TenopirA synopsis of each is given first, followed by an analysis of themethods used, participants included, levels of conclusions, and findings for each group. In the bibliography, all of the publications thatreport on each study are listed together by the study group name.2.1.1 SuperJournalThe SuperJournal project is a group of studies of e-journal use thatbegan in 1995 in the United Kingdom in response to the information explosion and limited budgets. The researchers use a variety ofresearch methods, including log file analysis, surveys, interviews,and focus groups, to study how academic users interact with e-journals and what features they value. Academic scientists and socialscientists were studied, including both faculty and students in Britishuniversities.2.1.2 Digital Library Federation/Council on Library andInformation Resources/Outsell (DLF/CLIR/Outsell)Outsell, Inc., conducted a survey of information use for the DigitalLibrary Federation and Council on Library and Information Resources in the fall of 2001 and early winter of 2002. Some 3,234 faculty,graduate students, and undergraduate students across seven subjectdisciplines at private and public doctoral research universities andleading liberal arts colleges were interviewed over the telephone.They were asked about their use and preferences for both print andelectronic resources from the library.2.1.3 HighWire/eJUStThe Stanford E-Journal Users Study (e-JUSt), published by HighWirePress, used a variety of methods to gain insights into the use of electronic journals, including qualitative user surveys, transaction loganalysis, and an ethnographic study of scholarly e-journal usage. Thequalitative user surveys were done online with participants takenfrom subscribers to HighWire’s medical and scientific journal Tableof Contents service. The participants included graduate students,faculty members, and clinicians from universities, hospitals, andgovernment and academic research institutes from 99 countries. Thestudies were conducted between November 2000 and August 2002.2.1.4 Pew Internet and American Life (also OCLC/Harris,and Urban Libraries Council)The Pew Internet and American Life Project conducted two studiesabout how students use the Internet. In the “Internet Goes to College,” 2,054 college students at two- and four-year public and privatecolleges completed surveys. In addition, graduate student researchers observed the behavior of college students at Chicago area colleges and universities. In the other Pew Internet and American LifeProject, “The Digital Disconnect: The Widening Gap between Internet Savvy Students and their Schools,” middle and high school students were studied between November 2001 and March 2002. About200 students wrote essays in which they expressed how they and
Use and Users of Electronic Lbrary Resourcestheir friends used the Internet for school and how they might use itin the future. Both these studies included how the students view thelibrary.OCLC/Harris and the Urban Libraries Council conducted similar surveys comparing library and Internet use by students and thepublic respectively. In the OCLC/Harris study, 1,050 participantswere surveyed between December 11, 2001, and January 1, 2002. Inthe Urban Libraries Council study, 3,097 participants were surveyedby telephone between March and April 2000.2.1.5 OhioLINKThe Ohio Library and Information Network is a consortium ofOhio’s college and university libraries and the State Library of Ohio.The consortium serves in excess of 500,000 students, faculty, and staffat more than 80 institutions of higher learning. OhioLINK’s Electronic Journal Center makes electronic articles and journals availableto OhioLINK members. Transaction log analysis is used to measurethe number of articles users download from the Electronic JournalCenter. This program, begun in April 1998, is ongoing.2.1.6 Tenopir and King StudiesThe Tenopir and King research studies are a series of surveys of morethan 16,000 scientists, engineers, medical professionals, and socialscientists in university and non-university research settings. Thesurveys measure reading and authorship patterns of these subjectexperts through critical incident, demographic, and usage questions. Information-seeking behaviors, amount of reading, purposesof reading, and source of readings are all measured. Recent studieshave focused on how reading patterns have changed over time withthe adoption of e-journals and what role library-provided journalsplay in overall reading patterns. These ongoing experiments beganin 1977.2.1.7 LibQUAL LibQUAL , conducted by the Association of Research Libraries(ARL) in conjunction with Texas A & M University, surveyed students, faculty, and staff at various community colleges, four-yearcolleges, and health science schools in the United States as well asthe New York Public Library and Smithsonian Institution during thespring of 2002. More than 70,000 faculty, staff, and students relatedhow often they used the physical and electronic libraries. Furthermore, they answered questions about their library’s level of servicethat they found minimally acceptable, the level they perceived, andthe level they desired. The results are presented by status of respondent and type of institution. Only those few questions that focus ondesired levels for print and electronic collections and services arerelevant and reported here.5
6Carol Tenopir2.1.8 JSTORThe JSTOR system provides electronic archives of back issues ofscholarly journals. JSTOR uses log analysis of both viewed andprinted articles to characterize use of its materials. In addition, someJSTOR subscribing libraries have analyzed their use of the JSTORjournals within their specific library environment. In the fall of 2000,JSTOR surveyed more than 4,000 academic users of the collection inhumanities, social sciences, and economics to discover usage patterns and preferences of university faculty.2.2 ParticipantsEach of the eight Tier 1 studies examined a variety of participants,with college and university students and faculty members the mostoften studied, followed by practitioners and other subject experts inscience, engineering, health, and social sciences. Table 2.1 summarizes the main participants included in each study.StudyTable 2.1. Tier 1: ParticipantsParticipantsSuperJournalStudents and facultyDLF/CLIR/OutsellStudents and facultyHighWire/eJUStScholars and cliniciansPew/OCLC-Harris/Urban Libraries CouncilMiddle, high, and college students/general publicOhioLINKOhioLINK usersTenopir and KingScientists and social scientists(academic and non-academic)LibQUAL Library users at institutions of highereducation (students and faculty)JSTORJSTOR users (mostly faculty)2.3 MethodsThe method or methods used in a research study determine whattypes of conclusions can be drawn about the sampled participantsand what findings can be generalized to the population as a whole.Wang (1999) provides an overview of methods for user behavioralresearch. An extension of her categorization of methods is used hereto describe Tier 1 studies. Tier 1 studies use one or more of the following methods: surveying users interviewing users (including focus groups) observing users through experiments observing users in natural settings (including keeping journals) transaction log analysis (included under “observing users” inWang 1999)
Use and Users of Electronic Lbrary ResourcesCovey (2002) also categorizes usage studies to help librariansdesign the most appropriate studies for the type of information theyhope to gather. Covey’s categories of research studies are similar toWang’s and include the following: surveys (questionnaires) focus groups user protocols (experiments and observations are both includedhere) other (heuristic evaluations, paper prototypes and scenarios, andcard-sorting tests) transaction log analysisTable 2.2 summarizes the methods used by the Tier 1 studies.Several use multiple methods for different phases of their projects;others rely on a single method.StudyTable 2.2. Tier 1: Methods UsedMethodsSuperJournalLogs/surveys/focus groups/ Surveys/interviews/logsPew/OCLC-Harris/Urban LibrariesCouncilSurveys/observation/focus groups/journal keepingOhioLINKLogsTenopir and KingSurveys/critical incidentLibQUAL SurveysJSTORLogsSurveys of users are typically done by sending a questionnaireby e-mail, the Web, or paper mail to a randomly selected percentage of the population under study. Tenopir and King, for example,survey samples of university faculty, members of professional organizations such as the American Astronomical Society, and scientistsin companies and government laboratories. LibQUAL librariessurvey students and faculty within their own university communityfor comparison with other LibQUAL libraries. Conclusions basedon the responses are generalized to the whole using appropriate statistical tests. Care in selecting samples and a reasonable return rateare necessary to draw valid conclusions.Almost all of the studies reported here that use surveys followthese basic precepts of sampling and analysis, but the types of conclusions that can be drawn vary by the types of questions that areasked. Among the Tier 1 studies that use surveys, the main distinctions in types of questions asked can be characterized as follows:1. preference (focusing on what people want or think about a particular service; e.g., LibQUAL , Pew)2. reported behavior (focusing on what people say they do in general; e.g., DLF/CLIR/Outsell, HighWire/eJUSt)3. critical incident questions (focusing on what people say they7
8Carol Tenopirdo in regard to a specific instance or reading; e.g., Tenopir andKing).1Table 2.3 shows methods in more depth by looking at what typesof questions were asked.StudyTable 2.3. Tier 1: Types of QuestionsType of QuestionsSuperJournalPreference and reported behaviorDLF/CLIR/OutsellPreference and reported behaviorHighWire/eJUStPreference and reported behaviorPew/OCLC-Harris/Urban Libraries CouncilPreference and behavior-reported and observedOhioLINKLog analysisTenopir and KingCritical incident, preference and reported behaviorLibQUAL Preference and reported behaviorJSTORPreference and reported behavior and log analysisTogether, the categories of participants and the methods used(as outlined in Tables 2.2 and 2.3) determine at which of three levelsvalid conclusions can be drawn: the “user level,” that is, what do individuals or groups of individuals such as social science faculty say they do or prefer; the “group level,” that is, what do groups of users at an institutiondo, without demographic differentiation; or the “readings or incident level,” that is, what do specific users orgroups of users do or prefer about a specific type of informationor reading (see Table 2.4).StudyTable 2.4. Tier 1: Conclusion LevelConclusionsSuperJournalUser levelDLF/CLIR/OutsellUser levelHighWire/eJUStUser levelPew/OCLC-Harris/Urban Libraries CouncilUser levelOhioLINKGroup levelTenopir and KingUser and reading levelsLibQUAL User levelJSTORUser and group levelSuperJournal, for example, uses transaction logs, surveys withquestions about preferences and behavior, focus groups, and interviews to study faculty and graduate students. Demographicinformation is known for each user. These multiple methods allowconclusions to be drawn at the user level for both behavior andSee Urquhart et al. (2003) for a description of the critical incident technique ininformation research.1
Use and Users of Electronic Lbrary Resourcespreferences (what specific types of users do and what they prefer).JSTOR uses transaction logs separate from survey questions. DLF/CLIR/Outsell used interviews to gather information on what userssay they prefer and say they do in general. Demogra
the behavior and preferences of library users, although sometimes the con-clusions are contradictory or unclear. This report for the Council on Library and Information Resources (CLIR) summarizes and analyzes more than 200 recent research publications that focus on the use of electronic library resources and were published between 1995 and 2003.
s r e s Ul a c o L —To configure users in the local database using the Users Local Users and Users Local Groups pages. For information on configuring local users and groups, refer to Configuring Local Users and Configuring Local Groups. RADIUS—If you have more than 1,000 users or want to add an extra layer of security for
69. electronic woodpecker 55 70. fish caller 55 71. electronic raindrops 56 72. pencil lead organ 56 73. electronic motorcycle 57 74. machine gun pulse detector 57 75. electronic siren 58 76. chirping bird 58 77. electronic cat 59 78. electronic bird 59 79. "horror movie" sound effect 60 80. electronic organ 60 81. soundmachinei 61 82.
e-Commerce Act but not quite as far as the definition of a qualified electronic signature under eIDAS. The advanced electronic signature based on qualified certificate aims to achieve a higher level of security. 5 ELECTRONIC SEALS Electronic seals (the "e-seals") are available only to legal persons, such as corporate entities, to ensure
Electronic Devices used in Electronic Smoking Systems: Devices commonly known as electronic cigarettes are devices designed to heat electronic liquids or tobacco products as an alternative to smoking aerosol products, which may contain or not contain nicotine, inhaled through the mouth. Page 5 of 15 .
Quick Start: Manage Users and Partner Visibility Microsoft Volume Licensing 2 When you go to the Manage Access section, you will see two tabs named Manage Users and Manage Partner Visibility. The Manage Users tab is where you can search for a user, invite new users, and assign or remove roles to manage your users' permissions. The Manage Partner Visibility is where customers can allow their .
2.13 Get child objects of a OU & container 2.14 Move object to another OU 2.15 Change properties on multiple users, reset password on multiple users, delete all users in particular OU 2.16 Find users who haven't logged in for 7 days and find users who haven't changed the password in the last 7 days. 2.17 Select timestamp attributes on users located in specific OU
Honeywell users and OEMs are eligible to become members of Honeywell Users Group EMEA conference. In addition, the 2022 Honeywell Users Group EMEA Steering Committee can waive the above eligibility rule to accommodate special cases. Users who represent various process automation industries make up the Honeywell Users Group (HUG)
Policies and Procedures - Team Creation and Access Control 3 5. Policies & Procedures -Electronic Document Management 5 6. Policies & Procedures - Electronic Signatures 6 7. References 8 8. SOP History 8. SOP: Use of Florence for Electronic Records and Electronic Signatures Version: 2 Effective Date: 1-December-2020 Page 2 of 8
