Keeping Count

Licensed electronic resources are fast becoming a major means of access to content in all libraries. The members of the Association of Research Libraries (ARL) alone spent over $132 million on electronic resources in 2001. LJ's 2003 Budget Report (LJ 1/03, p. 55) recorded an eight percent increase in the amount spent on Internet-related expenditures, including electronic products and services. Online resources—especially large aggregated databases—have proven so popular among users that librarians could never imagine not offering them.

However, as library budgets continue to shift toward e-content, both public and academic librarians are under increasing pressure to prove the value of these costly resources. The trouble is, they can't. Now, Project COUNTER (Counting Online Usage of NeTworked Electronic Resources) may give librarians the tool they need to measure e-usage accurately.

Librarians, and publishers, need to understand how much these databases are used and when. Within an aggregated journal collection, which titles are used the heaviest, and which the least? There are questions about how the content itself is used. How much is printed out, e-mailed, or downloaded? A decade ago, librarians believed that the electronic world promised a vastly improved means to measure usage, giving them a powerful way to respond to patron need. Publishers believed it would help them understand the value of the publications they sold. What went wrong?

Lack of definitions, for one thing. Publishers never agreed on what constituted a "search" or a "hit." Without a common language, there couldn't be much discussion or standard calculation. Also, the data that libraries receive from publishers are often inconsistent. Sometimes specific counts are provided, sometimes they aren't, and often librarians need to go to a site and retrieve data. The information can be available in a variety of formats, from HTML to ASCII to hard copy. Even if publishers report on usage, what they report on will differ. Some may track and release number and length of sessions, or downloads, or views; others may provide number of hits, requests, or dynamic pages.

Yale's Ann Okerson echoed many librarians' frustration when, at the 2002 Charleston Library Conference, she asked, "How can we compare similar services with dissimilar data?" The lack of standard measurements has meant that some libraries don't try to compute use. Others try to draw some meaning from the statistics they do receive. Still others are forced to hire additional staff to sort through this techno-Babel.

In the past few years, librarians, vendors, and publishers have recognized that the development of acceptable, global standards for measuring online usage required an international response. This was the impetus behind the creation of Project COUNTER, an effort to form such a protocol. The project, which had its genesis in the UK, was formally launched in March 2002, with an expanded, international steering group comprised of over 30 representatives of individual libraries, publishers, and intermediaries, as well as their professional and industry associations. To date, COUNTER has gained early and broad international acceptance, including the support of the Association of American Publishers, ARL, National Committee on Libraries and Information Science, and National Information Standards Organization (NISO).

The objective of Project COUNTER is to address the measurement problem by developing an international Code of Practice governing the recording and exchange of online usage data. The steering group agreed early on to focus first on journals and databases. These major items in libraries have been available online for several years and have a core of well-accepted definitions and content structures.

The first part (Release 1) of the Code of Practice contains an extensive list of data elements and other terms used in the usage reports and throughout the Code of Practice. Where possible, existing definitions from NISO, ISO (International Standards Organization), ARL, and other organizations have been used. Among the terms defined are vendor, aggregator, article, full-text article, search, item request, consortium, and consortium member. The protocols to be followed when an aggregator or gateway is involved in the delivery of vendor content to the customer have also been defined to avoid duplicate counts.

To date, usage reports have been problematic since they are generated differently from one platform to another. The Code of Practice specifies the requirements to be met by the data to be used for building the reports. In order to count only intended use, all unintended requests are removed. For example, when a page is slow to load, users tend to click repeatedly on a link. With the Code of Practice, all double clicks on an http link within ten seconds of each other will count as only one request. Where a PDF link is involved, the filter is set at 30 seconds, owing to the longer time it takes to bring up a PDF.

The full text of Release 1 (available at www.projectCounter.org) specifies the requirements vendors must meet for their usage reports to be designated "COUNTER-compliant."

By providing statistics that are consistent, credible, and compatible, publishers give librarians tools to demonstrate just how popular and useful online products and services are, thus helping ensure that adequate funding is provided for their purchase. In turn, this practice will supply publishers with the courage of their convictions: they'll be able to show purchasers just how effective their resources are. More importantly, many librarians will make COUNTER-compliant usage statistics a condition of signing agreements with publishers for online services.

To comply with Release 1, vendors will have to provide to customers (at no extra charge) the set of basic usage reports identified as Level 1. So far, these include two reports concerning journals and three for databases. The first journal report details the number of full-text articles requested each month, identified by journal. The second calculates turnaways for each month, identified by journal; this applies only where the user access model is based on a maximum number of concurrent users. The first database report breaks down total searches, sessions, and full-text requests by month and database. The second includes turnaways by month and database. The third reports referrals by aggregator or gateway.

Whenever possible, vendors that can provide the more detailed Level 2 reports will be prompted to do so. And vendors will be encouraged to make every effort to use COUNTER definitions in other usage statistics they provide to customers. The Code of Practice specifies that usage reports must be delivered at least monthly. There will be no format problems: reports must be delivered as a CSV file, a Microsoft Excel file, or in a format that can be easily imported into Microsoft Excel.

Although there will be only one valid version of the Code of Practice at any given time, different levels of compliance will be possible. Starting in 2004, publishers will require that vendor usage reports and processes be audited by an approved third party as a requirement for COUNTER-compliance. Detailed auditing specifications will be made available, along with a list of approved auditors. In 2003, as an interim measure, vendors may obtain COUNTER-compliant status by signing a declaration and demonstrating to COUNTER that they can provide at least Level 1 usage reports.

Support for COUNTER has been widespread in the library community. COUNTER provides libraries, for the first time, with data and protocols that will allow them to compare e-usage.

Librarians can encourage the adoption of COUNTER by including a clause in license agreements that specifies that vendors be COUNTER-compliant. A standard form of words for this clause is provided in the Code of Practice. A register of COUNTER-compliant vendors will be maintained on the project's web site.

Librarians are well represented in the management of Project COUNTER. Given the size and heterogeneity of the international library community, and the diversity of its requirements, a wider consultative process is necessary. The library community will be included on a regular basis to ensure that its needs are being met by the project. In fact, the first draft of the usage reports to be included in the Code of Practice, available in July 2002, was tested in an online survey conducted through the COUNTER web site. Over 650 librarians worldwide responded, and the draft was modified as a result of the feedback.

A major objective for 2004 and beyond is to enhance COUNTER's value to other library collections by extending it to cover e-books and other content types and by deepening it to provide more granular reports, such as usage of individual articles. This will be of great value to publishers and authors, as well as librarians, who will be able to identify the most heavily used articles as well as the types of institution or department where usage is heaviest.