eJUSt - Stanford University Libraries and Highwire Press
About eJUSt
Research Methodology
Research Team
Research Findings
Expert Workshops
Privacy Policy
Announcements
Presentations
Frequently Asked Questions
  Research Findings

E-Journal User Study
Report of First Survey
March 2002


Table of Contents

Introduction
Part I. Frequency of E-Journal Usage
Part II. Journal Format Preferences
Part III. Perceived Impacts of E-journals on Personal Research Practice and on Scientific Scholarship in General
III.A. Perceived Impacts of E-journals on Personal Research Practice
III.B. Perceived Impacts on Scientific Scholarship in General
Conclusions and General Discussion

REFERENCES

APPENDIX I: SAMPLE METHODOLOGY
I.1. List of Participating Societies
I.2. Sampling Methods

APPENDIX II: MODELS AND RESULTS
II. 1. Econometric Models for Analyses
II. 2. How to Read the Regression Tables
II. 3. Tables

APPENDIX III: SCHOLARS AND CLINICIANS COMMENT ON E-JOURNALS

FOOTNOTES


Acrobat PDF versions available:     Report    //     Tables 1-8    //     Tables 9-11



INTRODUCTION

Online access to scholarly scientific content is growing. Electronic journals (e-journals), in particular, are an increasingly important resource for scholars and play a significant role in scholarly communications. Understanding the evolution of scholarly practices with e-journals — how e-journals are being used, the reasons scholars use or don't use e-journals, and the perceived benefits and disadvantages of them — should help in the development of better online journal services and tools. A deeper understanding of usage patterns and sources of value may benefit libraries, publishers, scholars, aggregators, and software developers.

In the first survey for the E-Journal User Study project,1 we investigated patterns of scholarly practice among both users and non-users of electronic journals in the life sciences. Specifically, we examined possible determinants of three dependent variables representing three aspects of scholarly practice: (1) frequency of e-journal usage, (2) preferences and reasons for using e-journals versus printed journals,2 and (3) perceptions of the impacts of e-journal use, both on personal research practice and on scientific scholarship in general.

Factors Tested

At any particular moment, a wide range of factors may influence scholars in their journal-related practices, preferences, and perceptions. These factors range from basic demographics and available resources to more complex information goals, as well as from the life stage of the scholar to the particular task at hand (such as reading, paper writing, or grant writing).

From among the many factors possible, we chose in this first survey to investigate the effects of demographics (specifically age, occupation, gender, career experience, scientific field, and country of residence); access to resources (defined here as [1] the presence or absence of specific information technologies at home and/or at work and [2] personal and institutional journal subscriptions); frequency of collegial communications (through various methods, including in person and by e-mail or telephone); and publishing effort and success (based on the number of journal articles submitted and the number accepted for publication). We tested these factors using ordered probit and logistic regression analyses to determine which were significantly linked to our three sets of dependent variables. In addition, we used our first primary dependent variable — frequency of e-journal usage — as an independent variable in the analyses to see if it influenced the other two sets of dependent variables (preferences and perceptions).

The Survey Sample: Descriptive Characteristics

We drew our sample from the rosters of American (and one British) scholarly societies. (See Appendix I for the details of our sampling methodology.)

About 30% of respondents were female and 70% male; average and median ages were 47 and 48, respectively. About 20% of respondents were medical doctors (MDs) and allied health professionals. Approximately 60% of respondents stated their fields as biological sciences, 34% as health sciences, 4% as agricultural sciences, and 2% as other research fields. Approximately 60% of respondents were working at academic institutions, and the rest worked at nonacademic institutions such as private companies, government agencies, and hospitals or were retired. The full list of occupational categories included tenured faculty (26% of the sample), MDs and allied health professionals (20%), graduate students and postdoctoral researchers (12%), untenured faculty from academic institutions (11%), researchers in academic institutions (11%), researchers from the private sector (11%), researchers from government agencies (6%), retirees (3%), and others (1%). The average job experience of respondents was about ten years.

Survey respondents represented 99 countries: 77% resided in the United States or Canada; 11% were from Europe (excluding the United Kingdom); 5% from Asia; 3% from the United Kingdom; 2.5% from Latin America; 1.7% from Australia; and 0.3% from Africa.

Survey respondents used the Internet both for work and for leisure. On average, they used it ten hours per week, excluding time spent working with e-mail. In general, respondents in our sample were relatively familiar with the Internet. (See Appendix II for detailed topline tables of these sample characteristics.)

Communications Context

Electronic journals exist within a larger context of communications tools. In general, sample respondents actively used various communications methods. We hoped to examine the relationship between the frequency of using specific modes of collegial communications (one-on-one in person, one-on-one by e-mail, phone, fax, regular mail, and conferences) and our dependent variables. Because nearly all of our respondents had an extremely high frequency of collegial communications, however, testing the effects of variation in that frequency was difficult. Of our respondents, 86% had e-mailed colleagues about scientific matters the week before the survey, and 61% had e-mailed the day before. Seventy-two percent had made phone calls to colleagues the week before, and 38% the day before. Seventy-four percent had held informal face-to-face meetings with colleagues the week before, and 42% the day before. Use of fax, use of regular mail, and conference attendance were somewhat less frequent, but even within these methods, 64% of the sample had communicated with colleagues by fax in the past month, 55% had communicated by regular mail in the past month, and 68% had attended a conference in the past month.

Because of this relative lack of variation, together with a lack of consistent direction of correlations between the communications variables and the other variables in the survey, we had difficulty determining the direction of causality for those correlations. We need further quantitative and qualitative research to explore these links more fully. Nevertheless, we did find that those who communicate with colleagues more frequently tend to use e-journals more frequently. A positive correlation does exist between high collegial communications 3 and frequent e-journal usage, then, although the direction of causality is unclear. This suggests that e-journals may be closely intertwined with scholarly communications in many ways.

Although scholars may be similar in overall levels of communications frequency, their ways of communicating with each other and their reasons for doing so are diverse. E-journals could support this diversity if the diversity were better understood.


PART I. FREQUENCY OF E-JOURNAL USAGE

Our survey results show that e-journal usage is now an important part of the scholarly routine. E-journals are becoming essential for scholars who want to communicate to and share information with the broader scientific community. If providers (such as society publishers, libraries, and aggregators) are to improve their scholarly and technological offerings, they must better understand the factors that affect the frequency of e-journal usage.

Basic Statistics on Frequency of E-Journal Usage

Our "frequency of use" questions focused on how often respondents used e-journals to retrieve, read, and/or download full-text articles. We asked for this information in two ways: (1) What was their average frequency over the past year ("How often [based on the past year] do you retrieve, read, or download full-text, peer-reviewed journal articles online or through the Internet?") and (2) what was their most recent usage ("When did you last retrieve, read, or download a full-text, peer-reviewed journal article online or through the Internet?"). More than half of the sample (64%) said that they had used e-journals for this purpose on a weekly basis or more frequently on average over the past year; 18% said daily. Thirty-seven percent of our sample had used e-journals for this purpose either the same day they took the survey or during the previous day, and an additional 33% had done so "within the last week"; in all, 70% of the sample had used e-journals to access full-text articles within the week before the survey. Thus the majority of scholars, doctors, graduate students, and health professionals surveyed said they use journals regularly — weekly if not daily.

We chose to use most recent usage as our proxy indicator for e-journal usage frequency, because we think that average frequency over the past year is harder to recall accurately. Studies have demonstrated deterioration of recall accuracy over as short a period as six weeks (Smith, Jobe, and Mingay 1991; Pearson, Ross, and Dawes 1992). 4

Factors Tested as Predictors of Frequency of E-Journal Usage

What drives this general pattern of e-journal usage frequency? Unsolicited comments from survey respondents (see Appendix III) and our qualitative interviews suggest that many factors may change a scholar's pattern of using e-journals for a particular period of time. These include the stage of the research cycle (beginning, middle, or end), the research task (such as searching for clinical material or writing a review article), or the scholar's particular information need (such as keeping track of the latest published articles on a particular subfield or scanning daily news). Despite these shifts during the research cycle, however, more stable factors are more likely to shape the bigger patterns of frequency and intensity of e-journal usage more dramatically and over a longer period of time. These factors include institutional and demographic characteristics, access to resources, and scholarly effort of the individual.

We used "ordered probit" regression to test the hypothesis that a combination of demographic factors, Internet familiarity, basic information technology (IT) resources, institutional access to journals, personal subscription access to journals, and scholarly effort can significantly explain the probability of higher e-journal usage frequency. Ordered probit is a statistical technique that allows the use of a dependent variable that has ordered categories (in this case, frequency of e-journal usage). 5

Key Findings from the Regressions: Factors Related to Frequency of E-Journal Usage

According to our findings, younger scholars are more likely to be frequent e-journal users than are older scholars, and those with higher Internet usage are more likely to be frequent users than are those with lower Internet usage. Biologists are more likely to be frequent users than are other life scientists, and non-MDs are more likely than are MDs. Access to institutional journal subscriptions increases frequency of e-journal usage, as does access to a trio of basic IT tools (computer, printer, and the Internet) both at work and at home. Respondents with very few (zero or one) or many (more than five) personal journal subscriptions are more likely to be frequent users than are those with an average number of subscriptions. Those with an average level of scholarly effort (around three papers submitted in the past year) are more likely to be frequent users than are those with very low or very high levels of effort.

Country of residence (United States or Canada versus all other countries) had no statistically significant effect on e-journal usage when all other variables were controlled. Thus, although the first-order correlation shows that scholars from other countries use e-journals slightly more often than do those from the United States or Canada, differences in other control variables likely explain this. Gender also showed no effect: males were no more likely than females to use e-journals frequently, nor was there any first-order correlation between gender and usage.

To explore the most interesting statistical findings more thoroughly, we have clustered findings into three sets of factors: demographics, resources (tools and subscriptions), and scholarly effort and/or success. We report descriptive statistics for each set of factors when appropriate and then explain the link to frequency of e-journal usage.

Demographics

As noted in the introduction, our random sample of scholarly society members was skewed toward men (71% male), possibly because of their higher incidence in some of the life sciences, and toward residents of the United States and Canada (77%), most likely because all sampled societies were based in the United States (except for one British society). Average age was 47. The sample was also skewed toward non-MDs, with only 20% of respondents being MDs and allied health professionals. It was somewhat more evenly divided by field, however, with 60% being in biology and 40% in other fields.

  • The older the respondent, the less frequently he or she used e-journals, ceteris paribus.6 Senior scholars have often developed personal, tried-and-true methods for navigating, archiving, and communicating content. In addition, these scholars may have more resources, staff, and agents to scout and retrieve information for them. As explained in the qualitative report, convenience is a relative term; people practice what they know works for them.

  • Medical doctors used e-journals less frequently than others — at least to retrieve, download, and read full-text articles. Qualitative research suggested that MDs may have less time for e-journal research during the day; that some health care professionals lack dedicated terminals to facilitate usage; and that MDs may use e-journals in other ways — to access treatment protocols rather than full-text articles, for example.

  • Biologists used e-journals more frequently than did other life scientists (health and agricultural scientists in particular). Further data collection is needed to explain this finding. It is possible that biology is a faster-moving field or that it is more competitive in publications and grants than other fields are 7 and that biologists thus need to use information more urgently. It is also possible that more e-journals (or more with useful features) are currently available to biologists than to health and agricultural scientists. If this second possibility is true, scientific content online may be skewed toward the biological sciences, encouraging more frequent use.

Resources

Information Tools

Most of our sample respondents reported having many information resources, both at home and at work. Many lagged only in adoption of DSL (broadband transmission): 57% had DSL only at work, 3% only at home, and 23% at both home and at work; 17% did not have access at either location. Fewer than 2% lacked any of the other tools we asked about — personal computers, Internet access, printers, e-mail, or fax.

Seventy-three percent of respondents reported that they had access to PCs, printers, and the Internet — the key trio of tools for using e-journals — both at work and at home. Access to these three key IT resources varied by country of residence. Scholars from both the United States and Canada reported significantly (95% confidence interval) better access to the combination of PCs, printers, and the Internet than did researchers from other countries across Europe, Asia, Australia, and Latin America; 77% in the United States and Canada had access to all three at work and at home, compared to 63% in other countries combined.

Journal Subscriptions

Access to journals — both online and traditional paper — through personal and institutional subscriptions may also be a key factor that shapes frequency of e-journal usage. 8 The majority of respondents reported relying more on institutional subscriptions than on personal subscriptions for their research. A large majority (79%) reported accessing most journals (paper and electronic) through institutional subscriptions. Institutions, through their libraries, still provide the bulk of the literature that scholars need for their research. Respondents access a smaller subset of scientific journals in other ways, such as through personal subscriptions, colleagues' copies, and free online access. Variations across national borders complicate the story of institutional access, however. Respondents from the United States and Canada were more likely than respondents from other countries to report that they access most scientific journals through institutional journal subscriptions. Scholars from other countries were more likely to report that they do not access most of their scientific journals through institutional subscriptions. Within the United States and Canada, access to scientific journals did not differ by type of institution.

Although institutions played a key role in providing access to scientific journals (in any format), 72% of the sample respondents also reported having one to five personal journal subscriptions, with an average of 3.6. Only 7% said they had no personal subscriptions; just more than 20% reported having more than five. Country was not significant in the likelihood that a scholar had personal journal subscriptions. The numbers of personal subscriptions for the many respondents living outside the United States and Canada were not significantly different than those for the researchers inside the United States and Canada. This may be in part because study solicitations were drawn from membership lists of scientific societies (most based in the United States), and all respondents were thus members of at least one professional society and most likely received at least one journal subscription as part of that membership. (See sampling methodology, Appendix I.2.)

Older respondents tended to have more personal subscriptions than did younger respondents, and medical doctors out-subscribed research scientists. If fewer discretionary funds prohibit having multiple personal subscriptions, age and occupation may simply be a proxy for access to funds. (We did not measure financial resources in this survey and thus cannot control for this factor.)

Access to different resources appears to shape frequency of e-journal usage in different ways.

  • Access to information technology tools shaped usage frequency. Having a trio of key IT tools (a PC, a printer, and Internet access) both at work and at home boosts e-journal usage. Broadband access (such as DSL) at home also appears to encourage the use of e-journals. 9 The direction of causality is not clear here, however: Does having DSL at home increase e-journal usage, or are frequent e-journal users more frustrated with slow download times and therefore motivated to get DSL at home?

  • Scholars with access to institutional journal subscriptions used e-journals more frequently. One of the primary functions of the institution is to provide information in a central repository that surpasses the resources of any individual. We can infer from this that e-journal usage will continue to grow as institutions increase subscriptions to electronic editions.

  • Scholars with very few or very many subscriptions used e-journals more frequently than did those with an average number of subscriptions (about four). The relationship between number of personal subscriptions and frequency of use was actually curvilinear (modeled by the inclusion of a "squared" term, [SUBS]2). Perhaps those who have no personal subscriptions take advantage of "free" features such as abstracts, and those who have lots of personal subscriptions are more intensively engaged scholars and use all journals, including e-journals, more frequently. E-journals may also help scholars manage multiple personal subscriptions by facilitating content tracking and by alerting scholars to new articles.

Publishing Effort

We tested scholarly effort — as measured by the number of papers submitted in the past year 10 — as a predictor of e-journal usage frequency. The average number of paper submissions was about three per year. About 75% of respondents submitted one to five papers per year.

  • Scholars who submitted either very few or very many papers used e-journals less frequently than did those with a more average number of submissions. This was unexpected. We speculate that those who submit lots of papers are more senior researchers with multiple coauthors to help them search, retrieve, and read articles or who have already read more articles prepress as peer reviewers. Those who do not submit any papers at all may simply be less intensively engaged in scholarship and use all journals, including e-journals, less frequently.

Discussion

The findings suggest that a variety of contextual factors — including demographic characteristics, IT tools, journal access, and scholarly effort — determine frequency of e-journal usage. They also suggest that expectations of tools, as well as limitations and features of the environment, determine habits and routines for using technological tools. Older scholars, for example, tended to use e-journals less, whereas younger ones used them more. This may be related to formative practices developed early in the scholar's career. Younger scholars developed such practices with personal computers and Internet tools widely available in college and graduate school. Older scholars did not. Particular fields, such as biology, may have distinct rates of innovation and discovery different than those in other fields, which may influence frequency of use.

Our qualitative research indicates that the value of e-journals emerges from interdependence with other tools and media, not in isolation. Other resources that help increase the value of e-journals may support their use in a particular way. Scholars with high-speed color printers or broadband online services (such as DSL), for example, may use e-journals in different ways than do scholars without access to such resources. Institutional resources, such as lab assistants and department staff, can also be important for retrieving content.


Part II. Journal Format Preferences

Retrieving full-text articles is an important information-gathering activity for all scholars. Understanding why scholars prefer (or do not prefer) to retrieve articles using electronic journals rather than printed editions is key to overcoming barriers to their use and developing more user-friendly interfaces and content environments.

Basic Statistics on Journal Format Preferences

We asked respondents directly whether they preferred "online methods over print copy to retrieve full-text journal articles" and why. Topline data show that 75% of respondents said they preferred online methods for retrieval. (See Appendix II, Table 3.) We asked about three possible reasons for having such a preference: (1) Online articles are free, (2) they eliminate photocopying costs, and (3) they are convenient, allowing users to avoid trips to the library. All three reasons were popular, with more than 75% of respondents agreeing to each. Most of the sample (99%) agreed that convenience was a reason for favoring online methods for retrieval. The fact that online journals are "free to me" was the second most common reason, with 85% agreeing. Seventy-seven percent agreed that avoidance of photocopying costs was a reason for their preference. We asked the quarter of the sample who did not prefer online methods for retrieval why not, suggesting three possibilities: (1) PDF technology is troublesome, (2) online content is limited, and (3) printed journal versions are more portable. Portability was the most popular reason for favoring printed over online editions, with 86% agreeing. Limited content (lack of back issues online) was the next most popular reason, with 74% agreeing. Poor PDF technology was a reason for 62% of the sample.

These responses are reported conditionally. Only the 75% who said they favored online journals over printed editions were asked to respond to the first three motivations and were included in the final percentages reported here. Correspondingly, only the 25% who said they did not favor online journals were asked to respond to the last three motivations and were included in the reported percentages. Interestingly, however, most of those (5,466) who favored online journals (9,254) answered both sets of questions. This could be interpreted in at least two ways: (1) Perhaps respondents did not strongly prefer either online or printed editions for article retrieval or (2) perhaps they liked or disliked both but for different reasons and wanted to register their objections to (or approval of) all the mentioned motivations.

A closer look at the preferences data (see Appendix II, Table 9) reveals that as the frequency of e-journal usage to access articles increased, the respondent was more likely to agree that limited availability of back-issue content was a reason for disfavoring online editions. This might indicate that heavy users, who rely on e-journals more than other scholars do, have greater needs and expectations about archived materials. More frequent users were also less likely to complain about lack of portability.11

Factors Tested as Predictors of Journal Format Preferences

We ran several regression analyses (see Appendix II, Table 8 and Table 9) to try to find explanations for scholars' preferences for online methods or printed journals for retrieval. We looked at demographic factors (age, gender, occupation, job experience, field of research, and country of residence), Internet familiarity (measured as hours online per week other than for e-mail use), personal subscription behavior (based on number of personal subscriptions), publishing success (based on number of papers accepted in the past year), and e-journal usage frequency. We put these factors into the regression equations to see how well they predicted journal format preference (Table 8) and reasons for favoring or disfavoring e-journals (Table 9).

Key Findings from the Regressions: Factors Related to Journal Format Preference

All factors analyzed as possible predictors of journal format preference were statistically significant, and the results were as we expected. Younger scholars, non-MDs, biologists, women, frequent Internet users, users with less job experience, users with fewer personal subscriptions, users with more published papers, and more frequent e-journal users were all more likely than their counterparts who don't share these characteristics to prefer online methods for retrieving full-text journals. Conversely, older scholars, MDs, nonbiologists, men, users from countries other than the United States or Canada, users with less Internet familiarity, users with more job experience, users with more personal subscriptions, users with fewer published papers, and less frequent e-journal users are all less likely than their counterparts who don't share these characteristics to prefer online methods for retrieving full-text articles.

The reasons given for journal format preference varied only by demographic characteristics, personal subscription behavior, and frequency of e-journal usage.

Demographics

  • Older scholars and MDs were less likely to prefer online formats for article retrieval. Older scholars and medical doctors tend to use online journals less frequently and may have other established and effective routines for retrieving articles. They may use online methods for retrieval but do not necessarily prefer them over other methods, such as looking up articles from printed journals in their own collection or in a colleagues' or asking a coauthor, office staff person, or lab assistant to retrieve an article for them.

  • U.S. and Canadian respondents preferred e-journals over printed editions more than did respondents from other countries. Whereas scholars from all countries use e-journals equally frequently (ceteris paribus), those from the United States and Canada are more likely to prefer e-journals over printed editions. More research is needed to understand the sources and effects of these differences.

  • Biologists were more likely to prefer electronic journal formats than were nonbiologists. Biologists who disfavored e-journals were also more likely to cite slow PDF technology as a reason for avoiding using e-journals, ceteris paribus, although biologists were among the most frequent e-journal users. Patterns of scientific discovery and documentation in biology may demand faster, more efficient retrieval and access to content (perhaps particularly in certain stages of research) than those in other fields. E-journals may in fact help scholars in fields that are rapidly changing and assimilating new ideas, evidence, and techniques but may simultaneously be perceived as losing some of their time-saving potential because of slow PDF technology. Further research is needed to more fully understand how biologists' preferences relate to their research practices.

Resources

  • Frequent Internet users were more likely than infrequent users to favor e-journals over printed editions. Frequent users were also more likely to disfavor e-journals because of limited online access to back issues, ceteris paribus. Greater online experience may expose frequent users to more limitations of the online environment, including lack of archived content. Such limitations may have greater impact on the day-to-day practices of frequent users.

  • Scholars with very few or very many personal subscriptions were more likely to prefer online formats. The number of subscriptions had curvilinear effects. Those with few personal subscriptions may be getting free online access through their institutions, making online methods not just preferable but necessary. Those with many subscriptions may find searching through multiple journals and retrieving needed articles and content more efficient online.

Publishing Success

  • Scholars with very few or very many published papers were less likely to prefer e-journals for article retrieval. Those in the middle were more likely to prefer them.

Usage Frequency

  • Frequent e-journal users were much more likely to prefer e-journals over printed editions for retrieving articles. Through frequent usage, scholars may develop successful and effective practices for using online methods. Knowing how to use online tools such as search engines, particular Web sites, and other advanced features may help make e-journals a preferred method for retrieval.
  • Frequent users were more likely than infrequent users to say that being "free to me" was an important advantage of e-journals. Frequent users were also less likely to cite avoidance of photocopying costs as a reason for favoring e-journals. They were no more or less likely than infrequent users to cite convenience as a reason for preferring one format over another. Again, frequent users may have developed personal practices that include the effective use of both printed and electronic editions of journal articles to accomplish their work. What is convenient depends on the context of the work and the user.
Discussion

These findings suggest, as do our qualitative conclusions, that preferences for journal format and the reasons behind those preferences are complex and have multiple drivers. There may be no blueprint to predict choices for using electronic journals over printed journals; rather, changing conditional reasons, contextual factors, and stages of research may affect such choices. Findings also suggest that e-journal users are developing strategies for working around the limitations of both electronic and printed journals and any contextual variables, such as access points or infrastructure (in factors such as work at home, institutional arrangements, or job descriptions). Even when e-journals are weak in one area or another, users develop ways to use them that add value. These users may use other available resources to make up for any weaknesses. With such strategies, scholars develop a pattern of idiosyncratic, contextual usage.

Another significant finding was in the high number of responses citing "portability" as a reason for preferring printed editions over online versions. Those who do not prefer e-journals to retrieve articles cite portability as a reason for preferring print editions. Among those who prefer electronic editions, however, about 25% also say portability is a reason they like printed editions. In other words, those who have e-journal access and printers may still think of their printed editions as more "portable" than a printout of a bunch of articles. This suggests that portability may mean different things to different people. Our qualitative work suggests that most e-journal users print out electronic articles when they want to engage in intensive study reading. The concept of "portability" for some scholars may cover the portability of content (the ease with which they can assemble a bundle of related high-quality articles), not just the physical portability of the reading medium (the ease of carrying paper versus carrying a computer). From the present data, it is impossible to determine for which respondents this might be the case. The question of formats and portability requires further research.


Part III. Perceived Impacts of E-journals on Personal Research Practice and on Scientific Scholarship in General

Electronic journals affect both personal scholarly practice and scholarship in general. To improve the usefulness of e-journals and to get a sense of what their future might be, we must understand whether and how scholars perceive the influence of e-journals both on their own work and on scientific scholarship in general.


A. Perceived Impacts of E-journals on Personal Research Practice

We asked respondents if they felt that e-journals had affected their research activities or productivity in general. We then followed up with specific questions about the concrete ways in which e-journals were having an impact on their personal practice — by shortening retrieval time, allowing them to access more experimental information, increasing the number of papers they published, helping them with archiving, increasing their extradisciplinary reading, and enabling them to exchange more articles with colleagues.

Basic Statistics on Perceived Impacts on Personal Research Practice

Although 60% of all respondents felt that e-journals did affect their research activities or productivity, nearly 40% felt that e-journals did not. (See Appendix II, Table 7.)

The 60% who did see an impact on their personal practice reported faster exposure to a broader literature. The most common reported impacts were on everyday information-retrieval practices: 98% agreed that e-journals shortened retrieval time, 86% that they enabled scholars to obtain more experimental information, 71% that e-journals increased the number of papers they read outside their discipline, and 71% that e-journals enabled them to exchange more journal articles with colleagues. Scholars did not agree as widely that e-journals improved two other aspects of their personal practice — ability to publish papers (42% agreed) and ability to organize information in personal archives (52% agreed). In other words, respondents did not necessarily link faster retrieval time, increased exposure to outside literature, and increased scholarly exchange of information with a rise in their ability to publish more papers or organize information better.

These topline findings varied by country and occupation. Respondents from the United States and Canada were less likely than others to say that their productivity improved because they accessed good papers faster. Better library infrastructures in the United States and Canada may be responsible for this. MDs and biologists were more likely to report improved organization as a result of e-journal usage. These scholars may have more acute needs: MDs may need to access information quickly for treatment of patients and biologists to cope with the huge amount of new research in their field.

Factors Tested as Predictors of Perceived Impacts on Personal Research Practice

To explain these reported impacts on personal practice, we ran logistic regression analyses. We tested our hypothesis that certain factors could in part explain scholars' perceptions about the impacts of e-journals on their research activities. These factors are demographics (age, gender, occupation, job experience, field of research, and country of residence), Internet familiarity, personal subscriptions, publication success (number of papers accepted in the past year), and frequency of e-journal usage to retrieve full-text journal articles. We tested all of these factors in logistic regression equations to see how well they predicted scholars' perceptions of (1) the overall impact of e-journals on their personal research practice and (2) impacts on specific areas of their practice.

Key Findings from the Regressions

Factors Related to Perceived Impacts on Personal Research Practice

Scholars who tended to agree that e-journals had an overall impact on their personal research practice were younger and were more frequent users of e-journals. They were also more likely to have either very few or very many personal subscriptions and an average number of published articles. Conversely, older, less frequent e-journal users were less likely to agree that e-journals had an impact on their own research, as were users with an average number of personal subscriptions and with either very few or very many publications. Overall, demographic variables and frequency of e-journal usage seem to have the strongest influence on perceptions of the impacts of e-journals on personal practice.

Demographics

Country was a significant factor for nearly all reported impacts. Respondents from the United States or Canada were

  • more likely to agree that e-journals improved their ability to obtain experimental process information online and
  • less likely to agree that e-journals helped them publish more papers, read papers outside their primary discipline, exchange more articles with colleagues, and become more organized.

Age had an independent effect (controlling for all other contextual factors) on only one reported impact: older scholars were less likely to say that e-journals increased the number of papers they read outside their primary discipline.

Job experience (years from final degree, independent of age) very slightly reduced the likelihood of becoming organized and of publishing more papers based on e-journal usage but had no other effects. Field of study had only one effect: Biologists were more likely than non-biologists to agree that they "became more organized in archiving papers by using e-journals, creating [their] own mini library" as a result of e-journal usage, possibly because of the greater volume of publications available to biologists online.

Resources

Frequency of Internet usage had very significant but small effects, increasing the likelihood of all reported impacts except shortening retrieval time/reducing the number of library visits.

The number of personal subscriptions was again curvilinear.

  • Respondents with very many (more than five) and very few (less than one) personal subscriptions tended to believe that e-journals improved the processes of obtaining experimental process information and exchanging articles with colleagues.

Publishing Success

The number of papers published had little if any effect on perceived impacts of journals.

Usage Frequency

E-journal usage frequency had three strong effects on the specific perceived impacts of e-journals. Compared with less frequent users, heavy e-journal users tended to be

  • more likely to agree that e-journals increased the number of journal articles they published,
  • more likely to agree that e-journals made it easy for them to obtain experimental information, and
  • less likely to agree that they read more outside their primary disciplines as a result of e-journal use.

Frequent users may simply be more information hungry in general and use all methods of obtaining content more aggressively than do infrequent e-journal users. They may also have more distinctive patterns of usage and rely on e-journals intensely for certain practices (such as tracking down experimental information) but less intensely for other practices (such as browsing).


B. Perceived Impacts on Scientific Scholarship in General

We also asked scientists and practitioners about their perceptions of the impacts of e-journals on scholarship in a broad sense. We wanted to see if scholars felt that e-journals had impacts only on their own practice and did not significantly change the status quo of engaging in research — or if indeed these respondents perceived broader, more fundamental changes in the way scientific research is conducted.

Basic Statistics on Perceived Impacts on Scientific Scholarship in General

In many ways, respondents perceived impacts of e-journals on general scholarship just as they perceived impacts on their own individual practices; they believed that e-journals make a broader literature more quickly available to scholars. Two-thirds of respondents (66%) agreed to some extent that e-journal usage increases scholarly productivity (measured in number of grants, patents, articles, and papers). Related to this, 92% indicated that e-journals simplify and speed up current awareness of recent research. Two-thirds (67%) believed that e-journals increase exposure to non-peer-reviewed papers.

Strongly negative perceptions of e-journals did exist but were minimal. A full quarter of respondents (26%) agreed that e-journal usage decreases the rigor and quality of literature searches. Even more striking, one-third of respondents (34%) agreed that the unfriendly interfaces of e-journals waste users' time. Particularly older users, MDs, users living outside of the United States and Canada, and less frequent users of e-journals perceive interfaces negatively, according to regression results. (See Appendix II.3, Table 11, Column 5.)

Finally, respondents reported that they believe that e-journals have value beyond providing full-text articles. Eighty-seven percent agreed that e-journals provide other valuable services or features, such as editorial news, hyperlinking, peer reviews, and alerts or notification services. Frequent users have increased demands on and expectations for e-journals. As institutional access becomes more widespread, other users may increase their expectations as well.

Factors Tested as Predictors of Perceived Impacts on Scientific Scholarship in General

We used logistic regression analyses to test whether demographic factors (age, gender, occupation, job experience, field of research, and country of residence), Internet familiarity, personal subscriptions, number of papers accepted, or frequency of e-journal usage were associated with any of the impacts on scholarship in general reported by respondents.

Key Findings from the Regressions: Factors Related to Perceived Impacts on Scientific Scholarship in General

Scientists and practitioners varied in their perceptions of the impacts of e-journals on scholarly practice in general based primarily on age, country of residence, number of subscriptions, occupation, and frequency of e-journal usage. (See Appendix II, Table 11.)

Demographics

Age had a consistent and statistically significant effect on four perceptions of the impacts of e-journals on scholarly practice, independent of all other contextual factors.

  • The older the respondent, the more he or she was likely to report that e-journal usage decreases the quality/rigor of research literature searches. Older scholars may think that scholars using e-journals limit themselves to reading only what is available online, rather than going to a library or retrieving a full list of articles found through other means.

  • The older the respondent, the more he or she was likely to report that unfriendly interfaces of e-journals waste users' time. Older users may have more difficulty with these interfaces than do younger scholars.

  • The older the respondent, the less likely he or she was to report that e-journal usage increases scholarly productivity. Older respondents may resist seeing this new tool as essential to productivity, possibly because of more experience being productive prior to the introduction of e-journals.

  • The older the respondent, the less likely he or she was to report that he or she would do without or go to the library rather than pay any amount for online access. Older scholars might be even more willing than younger scholars to pay for online access if their experience of e-journal interfaces and usefulness could be improved, or they might be less comfortable doing without articles and have different definitions of a comprehensive search.
  • s

Being a doctor had three different effects on perceptions of e-journal impact:

  • MDs were more likely to agree that unfriendly interfaces of e-journals waste users' time. MDs may have more difficulty with these interfaces than do non-MDs, or they may be more time-pressed. This issue will be explored further in the third survey of this study.

  • MDs were less likely to agree that increased online retrieval is related to decreased offline retrieval. Medical practitioners might be less likely to depend on online retrievals than non-MDs. Doctors in teaching hospitals (60% of our sample of MDs) may have more established and convenient methods for offline retrieval, such as document retrieval services. An alternative interpretation is that doctors are looking for proven techniques and protocols, which might not always be the latest scientific developments found in current e-journal archives.

  • MDs were more likely to agree that they would do without or go to the library rather than pay any amount for online access. MDs may be more resistant to paying for e-journal access, possibly because they have better access to offline retrieval methods.

Country of residence showed moderately strong effects on nearly all perceived impacts, controlling for all other contextual factors. Residents of the United States and Canada tended to disagree with both positive and negative statements about the impacts of e-journals. Specifically:

  • U.S. and Canadian scholars were less likely to agree that e-journals waste users' time due to unfriendly interfaces. These scholars may have less difficulty with these interfaces (possibly because they have more up-to-date computer technology and higher bandwidth), or they may be less time-pressed.

  • U.S. and Canadian scholars were less likely to agree that e-journal usage increases scholarly productivity. These scholars may be more reluctant to see this tool as influencing productivity, possibly because they have had better access to it for a longer period of time and now take online access for granted more than do scholars in other countries. Or this may reflect the fact that the U.S. proportion of the world's growing scientific article output has decreased since the late 1980s, a trend predicted to continue (NSF Science and Engineering Indicators 2000). 12 Respondents may also have had in mind some measure of productivity different than what the study used (based on the number of articles published).

  • U.S. and Canadian scholars were less likely to agree that e-journal usage quickly improves awareness of recent research. These scholars may have more reliable offline resources, making any improvement in awareness through e-journals less dramatic for them. U.S. printed journals take longer (sometimes much longer) to reach subscribers outside of North America than to reach those in North America, for example. More than 85% of respondents, regardless of country of residence, agreed that e-journal usage quickly improves awareness of recent research, however.

  • U.S. and Canadian scholars were less likely to agree that e-journals provide other valuable features besides full-text articles. These scholars may have less interest in such features than do scholars in other regions, possibly because scholars in other regions may have less access and fewer options to full-text articles and thus may focus more on getting the most out of available features. (The tools come bundled, so they use what is there.)

  • U.S. and Canadian scholars were more likely to agree that they would do without or go to the library rather than pay any amount for online access. Scholars in other regions may be more willing to pay, possibly because they have less adequate access to offline or free institutional online retrieval methods. U.S. and Canadian scholars may also be more adamantly against paying extra for what they know is available for free elsewhere in other formats. They won't necessarily pay for convenience despite their desire for it. Convenience may not increase value enough to merit paying for it.

More job experience (years since final degree) had negative and statistically significant effects on two perceptions, ceteris paribus.

  • Scholars with more job experience were less likely to agree that e-journal usage increases scholarly productivity. As with age, scholars with job experience prior to the introduction of e-journals may have more experience being productive without them.

  • Scholars with more job experience were less likely to agree that e-journals provide other valuable features besides full-text articles. These scholars may be more likely to have already-established offline equivalents for these features.

Gender and field had little effect on any of the reported impacts of e-journals on scholarship in general.

Resources

Respondents with more personal subscriptions were more likely to agree that the unfriendly interfaces of e-journals waste users' time and less likely to agree that e-journal usage makes current awareness of recent research easy and fast.

Usage Frequency

Usage frequency had a very strong effect on four of the perceived impacts:

  • Frequent e-journal users were more likely to agree that e-journal usage increases scholarly productivity. Frequent use may help scholars get more out of e-journals.

  • Frequent e-journal users were more likely to agree that online searching increases exposure to non-peer-reviewed papers. Frequent users may dig deeper into the literature and be hungrier for all content, regardless of its source.

  • Frequent e-journal users were more likely to agree that e-journals provide other valuable features besides full-text articles. Scholars who use e-journals frequently may have more time to explore all available features.

  • Frequent e-journal users were less likely to agree that the more they use online retrieval, the less they obtain offline. Frequent users may be committed to both online and offline retrieval. They may have developed very specific practices that include both online and offline methods of retrieval, recognizing the need for both to support their research needs and scholarly goals.
Discussion

Based on the results from all the regressions on perceptions of scholarly impact, survey respondents from the United States and Canada were more likely than respondents from other countries to see e-journals as convenient tools that help daily routines (as opposed to a fundamental, transformative infrastructure that affects the quality of research and practice). Because these scholars have been exposed to e-journals for a longer time and through a better computing infrastructure, they may be more likely to use e-journals as a regular tool within their communications toolkit in their day-to-day practice.

The regression analyses indicate that frequent users benefit more from e-journals and are more committed to both online and offline retrieval than less frequent users. Frequent users are thus likely to demand more and better services from online journals. An increased usage frequency may help increase demand for features other than full-text article retrieval.

Interpretation of the basic statistic describing the sample majority's belief that e-journals "affect their research practice and productivity" is limited. First, 40% of the sample did not agree with this; second, we don't know from this question whether the perceived impact is positive or negative and in what ways. The detailed questions about the impacts of e-journals on personal practice and on scholarship in general are a bit more revealing. Reported impacts on personal practice seem to suggest that life science scholars believe that their universe of literature is expanding and becoming more diversified and that e-journals are helping to leverage that change. This shows up in concrete ways: access to more experimental information, more collegial exchange of information around journal articles, and more reading outside of a user's discipline. Practically, e-journals have made retrieval more efficient, allowing scholars to do more searching and retrieving and to take greater advantage of this broader literature. Links to increased productivity in terms of publishing are not clear from our results. The impacts seem to fall more centrally on the effectiveness and breadth of information gathering and on communications around that information.

Impacts on general scholarship mirrored impacts on personal practice. While there was some sense that e-journals decrease the rigor of searching (26% agreed), e-journals were perceived to increase exposure to non-peer-reviewed content and to increase and speed access to recent research, changing the flow and circulation of scholarly content and the kind of content. As our qualitative interviews indicate, this does in fact change the way that scholars perceive the practice of science in general. As the interviews suggest, this increased exposure could make science more open to diverse interpretations and raise the level of scrutiny.

The one difference in perception was in productivity. Although respondents felt that e-journals had not necessarily increased their own personal productivity through publishing, they felt that e-journals had increased scholarly productivity in general (the number of papers, grants, and articles produced). Scholars thus seem to experience the abundance of content, as facilitated by e-journals and online retrieval methods, yet they do not personally experience that productivity in publishing. They experience it more in their abilities to search and retrieve the content produced by others. Overall, the perceptions of impacts reported by doctors suggest that e-journal publishers and providers need to conduct further research into differences in the clinical environment versus the research lab.

Additional Findings from the Study

We asked a few questions in the survey to obtain additional information that we did not intend to use as factors in the above regressions but that we hoped would add to our basic findings about the perceptions of scholars in the life sciences. Specifically, we asked about criteria for choosing journals to which to submit publications and about the future of the journal and personal subscriptions.

Publishing

On average, respondents had submitted 3.2 papers (of which 2.7 papers were accepted and/or published) to journals in the past year. Existence of an electronic version was not perceived as an important factor in selecting journals for publication. A mere 3% of respondents considered the hosting of an online journal as a primary factor in their decision to submit papers. Journal reputation was rated highest as a reason for selecting a journal for publication: More than 50% of respondents strongly agreed that reputation and prestige of a journal are the most important criteria for selecting a journal for publication. Other reasons included the fact that a journal has a large/diverse readership (26%), likelihood of article acceptance (19%), and rapid peer review (12%).

The Future of the Journal

The journal will remain an important medium for publication over the next decade, according to survey respondents. Few thought that the journal as a form of publishing would disappear in the next five to ten years. Only 5% strongly agreed that the journal form would become obsolete. Sixty percent strongly agreed that journals would continue to be an essential tool for scientists' communication. More than 90% of respondents believed that journals are and will continue to be primary communications tools in the scientific community.

Most of the respondents (68%) agreed that personal journal subscriptions would become less important to them over time. This may be explained by an assumption (or a hope) that more journal article content will be available free to them through institutional subscriptions. Whereas 43% of respondents strongly agreed that more content would become available to them free through institutional subscriptions, fewer (27%) strongly agreed that personal subscriptions would become less important to them.


Conclusions and General Discussion

Scientists and practitioners believe that journals are and will continue to be primary tools in scholarly communications. They also believe that the burgeoning of e-journals has affected their research routines — shortening library visits, saving time on information retrieval, and facilitating communications with colleagues. They tend to disagree, however, on whether e-journals improve the quality of their research, the number of publications they access, and their organization of articles for future reference.

Furthermore, those who use e-journals do not consider the availability of journals online to be a top criterion for selecting a journal in which to publish their articles. This is significant in light of the current debate (and related pledges and petitions) about providing research findings globally and without charge within a short time after initial publication. It is difficult to reconcile promises to boycott publication in journals that do not make content free online with our finding that scholars generally do not consider the simple existence of online versions to be worth their consideration in deciding where to submit their papers.

Given that publishing articles is an extremely important part of scholarly communications, these findings suggest either that e-journals are still in the early stages of technology adoption, or that most scholars simply perceive e-journals as another Internet-based product that they find useful now but that other technology can replace in the near future.

The study found that users demand more features and services online; the more they use e-journals, the more features and services they demand. Especially biologists have a very high demand for e-journals and online features other than full-text retrieval.

The cost-and-benefit estimation of research library transition from printed editions to electronic journals has been debated; that is, do/will electronic journals save money for libraries? One study (Chen et al. 2001) demonstrated that subscriptions to e-journals made subscriptions to traditional printed journal editions more expensive. Chen et al. evaluate the transition cost based on direct subscription costs without considering the value to users. Our study suggests that cost-and-benefit analysis of library transition should count perceived use values by scientists. The use value could also be measured by how much less it costs scientists and others to obtain information online through libraries than it would cost them if libraries did not provide online access to this information (Tenopir and King 2000). E-Journal User Study will use other surveys and current journal pricing data to investigate this.

We conclude that e-journals provide a variety of features for full-text article and information retrieval and thereby enable scientists and practitioners to increase the efficiency and effectiveness of their research and practice routines. Whether e-journals increase the quality or productivity of research in the life sciences, however, is still in question: They seem to make a difference for some scholars but not for others.



REFERENCES

  1. Chen, F.L., Wrynn, P., Ehrman, F.L., Rieke, J.L., and French, H.E. Electronic journal access: How does it affect print subscription price? Bulletin of the Medical Library Association 2001; 89, 4:363-71.

  2. National Science Foundation. Science and Engineering Indicators 1996, Washington, D.C.

  3. National Science Foundation. Science and Engineering Indicators 2000, Washington, D.C.

  4. Pearson, R., Ross, M., and Dawes, R. Personal recall and the limits of retrospective questions in surveys. In Questions about questions: Inquiries into the cognitive bases of surveys. J. Tanur, ed. New York: Russell Sage Foundation, 1992.

  5. Smith, A.F., Jobe, J.B., and Mingay, D.J. Retrieval from memory of dietary information. Applied Cognitive Psychology 1991; 5:269-296.

  6. Tenopir, C., and King, D.W. Towards electronic journals. Washington, DC: Special Libraries Association Publishing, 2000.

Please share your reactions to the Survey Findings by filling out our simple form. Your feedback is a valuable component of the Electronic Journal User Study.


Last updated: 03-29-02





Home § About § Methodology § Workshops § Findings § Researchers § Privacy § Announcements § Presentations § FAQs



eJUSt home page Stanford University Libraries HighWire Press