Evaluating Journal Aggregators: A Preliminary Investigation Using EBSCO Online.
Robert Heyer-Gray [rheyer@ucdavis.edu]
Physical Sciences & Engineering Library
University of California
Davis, CA 95616
Introduction
I initially began exploring the usefulness/usability of EBSCO Online [http://www-us.ebsco.com/online/] in March 2000. At that time, the database contained 141 journals listed as “On Subscription” for UC Davis. I reviewed our holdings again in November 2000 and once more in May 2001. 100 additional titles have been added to EBSCO Online for UC Davis since April 2000, and I would imagine that as more of the UC Davis subscriptions are transferred to EBSCO, that number would continue to expand.
I approached EBSCO Online with a few questions in mind:
1. Is there anything unique to EBSCO Online? Or perhaps more precisely, are they providing access to publications (content) not already made available by other publisher agreements for UC Davis (UCD) and/or the California Digital Library (CDL)?
2. How did the holdings for title mounted on EBSCO Online compare to our print subscription in terms of coverage, completeness, and currency?
3. Can records (for both journals and individual articles) be linked to directly from an online catalog and/or A&I database?
4. What are the full-text formats?
5. How useful is EBSCO Online as an A&I/Full-Text database?
Why UC Davis is Using EBSCO Online
EBSCO Online provides access to full-text articles for journals to which the UCD Library subscribes through EBSCO Information Services. The UCD Library switched to EBSCO as its primary supplier of journals in late 1999. On a title-by-title basis since that time, EBSCO has been adding electronic access to those journals to which UCD has a print subscription.
In January 2001 the CDL entered into an agreement with EBSCO whereby EBSCO will negotiate on behalf of the CDL with publishers to secure licenses for electronic access to journals. EBSCO will be responsible for obtaining the information needed to allow access for the CDL to new electronic journal subscriptions and for working with the CDL to ensure that all authorized sites within the University of California system receive access. Because of this agreement, I would expect to see the number of publications accessible through EBSCO Online to grow substantially and to include journals for which we have no print equivalent.
Methods
Based on IP verification, EBSCO Online is able to determine to which publications UCD subscribes. Titles subscribed to are further defined as “Viewable on EBSCO Online” [denoted by the following image: 
] or “On Subscription” [denoted by the following image: 
]. For the purpose of investigating the usefulness of the EBSCO Online system, I eliminated those titles listed solely as “On Subscription” since those titles linked directly to the publisher’s site and are only provided as a convenience to the subscriber. For example:
Aggressive Behavior
Publisher: Wiley-Liss Inc
ISSN: 0096-140X
URL: http://www-us.ebsco.com/online/direct.asp?JournalID=102284
Subject: Psychology. Parapsychology. Occult
only provides a URL link to an intermediate page at EBSCO that than allows you to connect to the publisher’s page.
Those titles also listed as “Viewable on EBSCO Online” provide not only a URL link to the journals, but also an abstract, a URL link down to the article level, and the possibility of creating alerts for the arrival of new issues:
Administration & Society
Publisher: Sage Periodicals Press
ISSN: 0095-3997
URL: http://www-us.ebsco.com/online/direct.asp?JournalID=101606
Subject: Collections and General Works
You are subscribed to Volumes and Issues in bold.
Volume 33
Number 2 (May 1, 2001)
Number 1 (March 1, 2001)
Volume 32
Volume 31
Volume 30
In other words, these are the titles for which EBSCO maintains some form of control/responsibility or for which there is some form of value added (table of contents, article level linking, etc.). I must confess that in my initial pass through the EBSCO Online database, I took the phrase “Viewable on EBSCO Online” to mean that all titles listed were actually mounted on the EBSCO servers. It was not until my second pass through the database in November 2000 that I realized that “Viewable on EBSCO Online” was a combination of titles not only mounted on the EBSCO servers, but also on Catchword servers (via an arrangement with EBSCO) and on various publishers’ servers.
Because the manual check-in of our paper copies and the mounting of the electronic format are moving targets, I made every effort to complete my evaluations as quickly as possible. Certainly at the individual title level, I made sure that the electronic holdings were checked against the paper holdings within, at most, minutes of each other.
On my first pass through EBSCO Online, I was looking for unique titles (titles for which we had no other online access) including coverage and currency, whether or not a link could be made to the records (and at what level), full-text format issues, and usability of the system as an Abstracting and Indexing database. In subsequent reviews of EBSCO Online, I focused with more detail on the completeness of the holdings (missing issues) and with finer detail on the currency of the issues.
Findings
First Pass Through the Database
I was pleasantly surprised, even in my first pass through the EBSCO Online database, to find that a fair number of journals were included as “On Subscription” for UC Davis for which we did not have any kind of local or consortial publisher agreements. 98 of the 141 titles matching the criteria of “Journal is on Subscription” were also designated as “Viewable on EBSCO Online.” 16 of those 98 titles were also already available through publishers’ agreements (AIP, Springer, American Physiological Society, and SIAM titles). I included these 16 titles in the review process and in subsequent passes through the database because these titles do receive some form of value-added processing at EBSCO. That gave us access to 82 unique titles for which there was no local (UCD) or consortial (CDL) agreement in place.
| Online vs. Print Receipts | |||||||
| April 2000 | |||||||
|
Behinda |
% Behind |
Sameb |
% Same |
Aheadc |
% Ahead |
Total |
|
| Viewable on EBSCO Online |
38 |
38.78 |
35 |
35.71 |
25 |
25.51 |
98 |
a. Behind refers to the number of EBSCO Online titles behind print receipts.
b. Same refers to the number of titles where print and EBSCO Online receipt were the same.
c. Ahead refers to the number of EBSCO Online titles ahead of print receipts.
| Detailed Online vs. Print Receipts | |||||
| April 2000 | |||||
| Publisher |
# of Titles |
Behind |
Same |
Ahead |
Totals |
| Sage |
42 |
17 |
22 |
3 |
42 |
| Blackwell |
14 |
4 |
5 |
5 |
14 |
| Lawrence Erlbaum |
8 |
7 |
1 |
8 |
|
| Amer. Physiological Society |
7 |
7 |
7 |
||
| MIT |
5 |
4 |
1 |
5 |
|
| Taylor & Francis |
5 |
3 |
2 |
5 |
|
| Springer |
5 |
4 |
1 |
5 |
|
| AIP |
3 |
3 |
3 |
||
| Oxford |
3 |
1 |
2 |
3 |
|
| Cambridge |
2 |
1 |
1 |
2 |
|
| Society for Applied Spectroscopy |
1 |
1 |
1 |
||
| American Society of Agronomy |
1 |
1 |
1 |
||
| SIAM |
1 |
1 |
1 |
||
| Univ. of Chicago |
1 |
1 |
1 |
||
| Total |
98 |
38 |
35 |
25 |
98 |
Most of the publications had 1-2 years of full-text available online. While I made no note of the actual number of issues ahead or behind on EBSCO Online, I did make a note that most publications that were behind tended to lag by a single issue. The notable exceptions were the Lawrence Erlbaum publications, which had a significant lag in online receipts. Almost 40% of the titles were more current in print than online! This was not at all what I had hoped to see. This was, however, only a single snapshot and of only a small number of publications.
Second Pass Through the Database
On my second pass through the database in November 2000, 195 titles were now listed as “On Subscription” for UCD. Of these, 133 were also “Viewable on EBSCO Online.” 17 of these titles were already available through publishers’ agreements (American Physiological Society, Springer, AIP, SIAM, and Wiley). That gave us access to 117 unique titles for which there was no local (UCD) or consortial (CDL) agreement in place. Only 132 of the 133 titles were compared, since one of the Blackwell titles was only available online.
| Online vs. Print Receipts | |||||||
| November 2000 | |||||||
|
Behind |
% Behind |
Same |
% Same |
Ahead |
% Ahead |
Total |
|
| Publishers Servers |
10 |
20.83 |
18 |
37.50 |
20 |
41.67 |
48 |
| Ebsco Servers |
18 |
32.14 |
30 |
53.57 |
8 |
14.29 |
56 |
| Catchword/Ebsco Servers |
5 |
17.86 |
17 |
60.71 |
6 |
21.43 |
28 |
| Total |
33 |
25.00 |
65 |
49.24 |
34 |
25.76 |
132 |
| Detailed Online vs. Print Receipts | ||||||||||
| November 2000 | ||||||||||
| Publisher |
Server Location |
# of Titles |
Behind 3+ |
Behind 2 |
Behind 1 |
Same |
Ahead 1 |
Ahead 2 |
Ahead 3+ |
Totals |
| Sage | Ebsco |
51 |
1 |
1 |
12 |
29 |
8 |
51 |
||
| Blackwell1 | Publisher |
23 |
5 |
12 |
5 |
22 |
||||
| Lawrence Erlbaum | Catchword |
13 |
9 |
4 |
13 |
|||||
| MIT | Catchword |
7 |
3 |
4 |
7 |
|||||
| Amer. Physiological Society | Publisher |
7 |
7 |
7 |
||||||
| Taylor & Francis | Catchword |
5 |
1 |
3 |
1 |
5 |
||||
| Springer | Publisher |
5 |
3 |
2 |
5 |
|||||
| Cambridge | Publisher |
4 |
2 |
1 |
1 |
4 |
||||
| Corwin | Ebsco |
3 |
3 |
3 |
||||||
| AIP | Publisher |
3 |
3 |
3 |
||||||
| Carfax | Catchword |
2 |
1 |
1 |
2 |
|||||
| Oxford | Publisher |
2 |
1 |
1 |
2 |
|||||
| MCB | Ebsco |
1 |
1 |
1 |
||||||
| Amer. Assoc. for Adult Continuing Ed. | Ebsco |
1 |
1 |
1 |
||||||
| Society for Applied Spectroscopy | Catchword |
1 |
1 |
1 |
||||||
| American Society of Agronomy | Publisher |
1 |
1 |
1 |
||||||
| Assoc. of American Geographers | Publisher |
1 |
1 |
1 |
||||||
| SIAM | Publisher |
1 |
1 |
1 |
||||||
| Georg Thieme Verlag | Publisher |
1 |
1 |
1 |
||||||
| Wiley | Publisher |
1 |
1 |
1 |
||||||
| Total |
133 |
2 |
3 |
28 |
65 |
25 |
1 |
8 |
132 |
|
| 1 Blackwell provides online access to a single title for which UCD has no print subscription. | ||||||||||
| The number of total titles, 133, differs from the 132 used to determine currency of receipts. | ||||||||||
25% of the titles were still more current in print than online, however this is an improvement over the nearly 40% from the April 2000 review of the database. Just over 32% of those titles actually mounted on the EBSCO servers were behind our paper receipts. Most of the titles that were either ahead or behind the print receipts did so by only a single issue.
For this second pass through the database, I also looked for gaps (missing issues) in the electronic journals available through EBSCO Online and compared those to the issues missing for our print receipts. For determining missing print receipts, if our electronic access started with 1998, I only compared print to online starting with 1998. An issue was determined to be missing online if there was a gap between the first issue available online and the last issue available online. If the online receipt was an issue behind the print receipt, this was not considered to be a missing issue. The same held true for the print copy as well.
Missing from EBSCO Online:
Assoc. of American Geographers: Annals of the Association of American Geographers: v.89(4)
Blackwell: American Journal of Agricultural Economics: v.82(2)
Blackwell: Journal of Regional Science: v.37(2), v.37(3), v.37(4), v.40(1)
Blackwell: Modern Language Journal: v.84(2)
Blackwell: Political Psychology: v.20(1), v.20(4)
MIT: Journal of Economics and Management Strategy: v.7(1), v.8(2), v.8(3), v.8(4)
MIT: TDR: The Drama Review: v.42(4)
Sage: Compensation and Benefits Review: v.32(1)
Sage: Journal of Sport and Social Issues: v.24(1)
SIAM: SIAM Journal on Mathematical Analysis: v.27(3)
Missing in Print:
Blackwell: American Journal of Economics and Sociology: v.59(1)
Blackwell: Business and Society Review: v.105(1)
Blackwell: Political Psychology: v.21(1)
17 issues were missing online, while for the same period of online access only 3 issues were missing in print. 10 of the missing online issues were titles from publishers’ sites (Assoc. of American Geographers, Blackwell, and SIAM), 5 were titles from Catchword (MIT) and 2 were from EBSCO (Sage).
Third Pass Through the Database
On my third pass through the database in May 2001, 241 titles were now listed as “On Subscription” for UCD. Of these, 148 were also “Viewable on EBSCO Online.” 18 of these titles were already available through publishers’ agreements (American Physiological Society, Springer, AIP, SIAM, and Wiley). That gave us access to 130 unique titles for which there was no local (UCD) or consortial (CDL) agreement in place. Only 146 of the 148 titles were compared for this review, since one of the Blackwell titles and one of the American Physiological Society titles were only available online.
| Online vs. Print Receipts | |||||||
| May 2001 | |||||||
|
Behind |
% Behind |
Same |
% Same |
Ahead |
% Ahead |
Total |
|
| Publishers Servers |
4 |
7.55 |
22 |
41.51 |
27 |
50.94 |
53 |
| Ebsco Servers |
18 |
31.03 |
35 |
60.34 |
5 |
8.62 |
58 |
| Catchword/Ebsco Servers |
7 |
20.00 |
22 |
62.86 |
6 |
17.14 |
35 |
| Total |
29 |
19.86 |
79 |
54.11 |
38 |
26.03 |
146 |
| Detailed Online vs. Print Receipts | ||||||||||
| May 2001 | ||||||||||
| Publisher |
Server Location |
# of Titles |
Behind 3+ |
Behind 2 |
Behind 1 |
Same |
Ahead 1 |
Ahead 2 |
Ahead 3+ |
Totals |
| Sage | Ebsco |
52 |
1 |
1 |
14 |
31 |
5 |
52 |
||
| Blackwell1 | Publisher |
30 |
1 |
16 |
11 |
1 |
29 |
|||
| Lawrence Erlbaum | Catchword |
13 |
1 |
1 |
11 |
13 |
||||
| MIT | Catchword |
9 |
3 |
6 |
9 |
|||||
| Amer. Physiological Society1 | Publisher |
8 |
7 |
7 |
||||||
| Taylor & Francis | Catchword |
5 |
3 |
2 |
5 |
|||||
| Springer | Publisher |
5 |
1 |
4 |
5 |
|||||
| Cambridge | Publisher |
4 |
2 |
1 |
1 |
4 |
||||
| Corwin | Ebsco |
3 |
1 |
2 |
3 |
|||||
| Carfax | Catchword |
3 |
1 |
2 |
3 |
|||||
| Psychonomic Society | Catchword |
3 |
2 |
1 |
3 |
|||||
| AIP | Publisher |
3 |
1 |
2 |
3 |
|||||
| MCB | Ebsco |
2 |
1 |
1 |
2 |
|||||
| Oxford | Publisher |
2 |
2 |
2 |
||||||
| Amer. Assoc. for Adult Continuing Ed. | Ebsco |
1 |
1 |
1 |
||||||
| Guilford | Catchword |
1 |
1 |
1 |
||||||
| Society for Applied Spectroscopy | Catchword |
1 |
1 |
1 |
||||||
| Assoc. of American Geographers | Publisher |
1 |
1 |
1 |
||||||
| SIAM | Publisher |
1 |
1 |
1 |
||||||
| Wiley | Publisher |
1 |
1 |
1 |
||||||
| Total |
148 |
4 |
2 |
23 |
79 |
36 |
1 |
1 |
146 |
|
| 1 Both Blackwell and American Physiological Society provide online access to a single title for which UCD has no print subscription. | ||||||||||
| The number of total titles, 148, differs from the 146 used to determine currency of receipts. | ||||||||||
Slightly less than 20% of the titles were still more current in print than online. With each look at the database there seems to have been an improvement in terms of currency. However, those titles mounted on the EBSCO servers again had an alarmingly high percentage (at just over 31% compared to just over 32% in November 2000) that were behind our paper receipts. The most marked improvement were those titles mounted on the publishers� servers with a drop to just over 7% behind the paper receipts.
Again for this third pass through the database, I also looked for gaps (missing issues) in the electronic journals available through EBSCO Online and compared those to the issues missing for our print receipts.
Missing from EBSCO Online:
Blackwell: American Journal of Economics and Sociology: v.59(5)
Blackwell: Business and Society Review: v.101, v.103, v.105(3)
Blackwell: Journal of Finance: v.56(1)
Blackwell: Journal of Regional Science: v.37(2), v.37(3), v.37(4), v.40(1)
Blackwell: Philosophical Forum: v.31(1), v.31(3-4)
Blackwell: Political Psychology: v.20(1), v.20(4), v.21(3)
Psychonomic Society: Pyschonomic Bulletin & Review: v.7(4)
Sage: Compensation and Benefits Review: v.32(1)
Springer: Bulletin of Environmental Contamination and Toxicology: v.65(5), v.65(6)
Taylor & Francis: Society and Natural Resources: v.14(1)
Missing in Print:
Blackwell: American Journal of Economics and Sociology: v.59(1)
Blackwell: Business and Society Review: v.105(1)
Cambridge: Econometric Theory: v.17(1)
Sage: Compensation and Benefits Review: v.32(4)
19 issues were missing online, while for the same period of online access, only 4 issues were missing in print. 16 of the missing online issues were titles from publishers’ sites (Blackwell and Springer), 2 were titles from Catchword (Psychonomic Society and Taylor & Francis) and 1 was from EBSCO (Sage). 6 of the 16 issues missing from the publishers� sites were missing in the November 2000 review.
Linking
EBSCO does have the capacity for linking directly to journals:
URL: http://www-us.ebsco.com/online/direct.asp?JournalID=101606
is a direct link to the journal Administration & Society
and apparently directly to articles as well:
Citizens Versus the New Public Manager: The Problem of Mutual Empowerment
Full Text: Local PDF 58 KB
Issue: Volume 32, Number 1 March 1, 2000
URL: http://www-us.ebsco.com/online/direct.asp?ArticleID=FBVRL18XVTF3LGG1B4EX
Each journal and article listed as “Viewable on EBSCO Online” seems to have unique journal and article identifiers that would allow for linkage.
Full-Text Formats
The most common format for full text content on EBSCO Online is PDF (Portable Document Format from Adobe Acrobat).
Another frequent format for full-text is RealPage/Catchword (Taylor & Francis for example). RealPage requires a special browser provided by the Catchword company. Catchword now also usually includes a PDF format of the articles as well.
Some full-text articles apparently now also require IBM’s techexplorer plug-in for displaying expressions and documents containing mathematical and scientific markup.
In any event, the full-text formats above should pose no technical or software problems for the average user. All viewers/plug-ins are freely available from the respective companies.
Usefulness as A&I/Full-Text Database
EBSCO Online allows three methods of searching: ARTICLESearch, JOURNALSearch and Browse. ARTICLESearch allows simple:
or advanced keyword searching for articles:
The database allows Boolean (“and”, “or,” and “and not”), proximity (“near”) and exact phrase (” “) searching. It also includes two forms of truncation:
*(prefix) key* searches key, keying, keyhole, etc.
**(forms of word) sink** searches sink, sinking, sank, sunk, etc.
While on the surface the search interface looks fairly basic, it does actually allow for fairly sophisticated searching using Microsoft’s Index Server Query Language. The “Search Tips” link takes you to some of the more detailed search capabilities.
ArticleSearch only searches for those journal titles that are listed as “Viewable on EBSCO Online.” That would mean that only 148 of the 241 journals listed as “On Subscription” would actually be searched.
JOURNALSearch allows searching by journal title, ISSN, publisher and content description:
Browse allows you to browse by title or subject categories for the tables of contents and abstracts (with links to full-text for those journals for which we have subscriptions):
Because only 148 of the 241 publications listed as “On Subscription” are currently searchable/available, most still with only 1-2 years of content, I think one might want to wait to consider EBSCO Online as a full-fledged A&I/full-text database for public use. As more of UCD�s journals are converted to EBSCO, the usefulness of this source as a database will probably grow.
Conclusion
There are currently enough oddities in the system to keep me from endorsing the EBSCO Online database as a publicly available A&I database for UCD. Instead, I would recommend a title-by-title approach for linking to electronic journals via EBSCO Online. The percentage of late receipts for online issues (especially those locally mounted on EBSCO’s servers) is currently unacceptable. The fault may not lie with EBSCO, but the reality is that the paper copies of current issues are making it through the U.S. mail and manual check-in process faster than the electronic versions for too many journals. The number of journals that link to publishers�/aggregators’ sites for which we have no agreements/contracts in place is also too high. There are 27 journals listed as “On Subscription” for which we have no online agreement (titles from American Historical Association, McGraw-Hill, Ingenta, Cambridge Scientific, University of California Press, American Ceramic Society, Nature, and the Industrial Research Institute to name a few). This would undoubtedly lead to patron annoyance and frustration. To EBSCO’s credit there is a disclaimer that addresses this, but realistically I doubt that many people would read this disclaimer. The number of missing issues (especially from Blackwell) should also be addressed. EBSCO as the aggregator (whether or not the material is locally mounted or accessed via a publisher or via another aggregator) should ensure the completeness of the materials available through EBSCO Online. EBSCO Online does link to 130 titles for which we have no other online access. That alone makes working with EBSCO Online in some way or form a must. The fact that the CDL has entered into an agreement with EBSCO, whereby EBSCO will be securing e-journal licenses for the University of California system, also clearly makes working with EBSCO Online a priority. Because it is clear we are going to be using the services of EBSCO and that EBSCO Online is an integral part of this, we must ensure that EBSCO provides the University of California system and its patrons a better product than that which is currently available.
