This is the 13th year of the LJ Index of Public Library Service and Star Library ratings. The 2020 scores and ratings are based on FY18 data from the Institute of Museum and Library Services (IMLS) Public Library Survey (PLS). Because of that delay, they don’t reflect the impact of the coronavirus; that won’t be reflected in the data until 2022.

The big news in this year’s edition is that successful retrievals of electronic information (e-retrievals)—measuring usage of online content, such as databases, other than by title checkout—joins the six other measures that determine the LJ Index: physical circulation, circulation of electronic materials, library visits, program attendance, public internet computer use, and Wi-Fi sessions. The addition builds on last year’s new measure, Wi-Fi sessions, and is likely to be followed by library website visits next year, as PLS began collecting that data in 2018. Together with 2016’s addition of electronic circulation, they will build a much more complete picture of digital library use than was previously possible. Now that electronic circulation has been well established as a separate statistic, as of this year we have also replaced total circulation with physical circulation, so that e-circulation is not counted twice.

SCREENING FOR EXTREME OUTLIERS

This year’s edition of the LJ Index also incorporates, for the first time, a new criteria for libraries to achieve Star Library status. To address the longstanding, growing problem of extremely high statistical outliers, we have introduced a bar on the extent to which a highly extraordinary outlier on a single per-capita statistic will be allowed to earn a library star status. A library’s LJ Index score will not earn it star status if any of its seven per-capita figures is three or more standard deviations above its expenditure category average (the widely accepted statistical standard for an outlier) and if the standard score (a translation of the reported figure to its number of standard deviations from its group mean) for any such statistic accounts for half or more of the library’s total raw LJ Index score (which, currently, is the sum of the standard scores for the seven per-capita statistics).

While a figure that is more than three standard deviations from a spending peer group’s average is highly likely to be erroneous—or at least non-comparable to other figures for its group—that is not assumed by the application of this new criterion. Unusual circumstances often generate exceptional statistics. The greater concern in determining Star Library status is to avoid circumstances in which a single highly extreme outlier for a library, even if it is accurate, is allowed to overcompensate for six other relatively lackluster figures.

Given the combined impact of these changes, it is little surprise that there are a record number of new Star Libraries this year.

MISSING MEASURES

One-hundred-and-fifteen—more than two out of five—of 2020’s 262 Star Libraries were not Star Libraries in 2019. There was a dramatic drop in the number of libraries scored on the Index this year, as there was in 2019. The new new per-capita output measure—e-retrievals—was not reported by almost 1,400 libraries. Almost 1,200 additional libraries that reported e-retrievals did not report last year’s new per-capita output measure, Wi-Fi sessions. Combined, those more than 2,500 non-reports were the biggest factor lowering the number of libraries scored on the LJ Index to 5,608. Other disqualifying factors included, but were not limited to, four: not meeting the IMLS definition of a public library, having a legal service area population of less than 1,000, spending less than $10,000 in total operating expenditures, or declining to report any of the other data required to calculate the LJ Index score.

REPORTING OF E-RETRIEVALS BY STATE

Nationwide, 85.2 percent of U.S. public libraries reported electronic information retrievals for FY18. In individual states, however, levels of reporting for this newest LJ Index statistic varied dramatically.

For 17 states, e-retrievals were reported for FY18 by all public libraries: Arizona, Delaware, Georgia, Idaho, Kentucky, Louisiana, Montana, North Carolina, New Hampshire, Nevada, New York, Ohio, Oklahoma, Oregon, Rhode Island, South Carolina, and Wyoming. E-retrievals were also reported by the District of Columbia and Hawaii.

Conversely, 13 states had double-digit percentages of libraries that did not report e-retrievals for FY18, despite it being the second year of collecting such data for all states and, for most, the third year. These are Maine (91.6 percent), Utah (77.0 percent), New Jersey (63.4 percent), Connecticut (60.9 percent), Minnesota (48.5 percent), Wisconsin (38.1 percent), Florida (37.0 percent), Tennessee (30.6 percent), California (29.8 percent), Illinois (24.7 percent), Washington (23.0 percent), Virginia (21.5 percent), and Vermont (17.9 percent). There were no states in which no libraries reported e-retrievals.

Several factors probably account for most of these differences in reporting. States in which databases are licensed at the state level, rather than by individual libraries, are more likely to report data for all or most libraries. Similarly, states in which the state library agency or one or more library consortia organize cooperative purchasing of e-resources or negotiate discounted licensing costs for individual libraries are likely to have more comprehensive reporting on e-retrievals. Where there is any higher-level organization of e-resource purchasing, it is more likely that local libraries have ready access to statistics on e-retrievals by their users.

An excellent example is the Ohio Public Library Information Network (OPLIN), which provides a database statistics portal for its public libraries (oplin.ohio.gov/statistics). Other states, like North Dakota and Pennsylvania, provide usage data to individual public libraries for databases provided by the state library. Local libraries must supplement these data with comparable data for e-resources to which they subscribe individually. (See library-nd.libguides.com/PLS and bit.ly/PAcounter5.) In still other states, like Texas, individual public libraries are not expected to report e-retrievals for databases licensed by the state; the state library agency parses those statistics for them. (See the 2019 Texas Public Libraries Annual Report: Issues and Explanations.)

Notably, a study conducted in North Carolina indicates that database usage tends to be higher for public libraries for which database usage statistics (available monthly from NC Live) are downloaded more frequently. The authors offer two possible explanations for this relationship: “Libraries are downloading their usage reports and then intentionally promoting resources to patrons,” and “Libraries that download statistics from the NC LIVE website are simply more familiar with the databases that are available and are more likely to use them and suggest them to patrons.” Other predictors of database use include two capacity indicators—the number of databases to which the library provides access and the number of public internet computers in the library—and one community indicator, the percentage of the legal service area population with a bachelor’s degree. (See Jill Morris & Emily Guhde, “Making Usage Data Meaningful.”)

COLLECTING & REPORTING E-RETRIEVALS DATA

If the PLS is to represent fully the important role that e-resources play in today’s public libraries, there is a pressing need for IMLS to address the high levels of non-reporting on e-retrievals with the Chief Officers of State Library Agencies and State Data Coordinators in underreporting states. Those State Data Coordinators and continuing education providers should redouble their efforts to train and support staff of local libraries in collecting and reporting this statistic.

State Data Coordinators offer a uniquely well-informed perspective about the challenges public libraries face in collecting and reporting data on e-retrievals. Among those who responded to an inquiry about the biggest obstacles their libraries face in reporting e-retrievals, there was unanimity on two points, confusion and burden. According to Evelyn Lindberg and Kathleen Sullivan of the Washington State Library, “Public libraries in Washington State subscribe to dozens of electronic collections and must determine for each one what constitutes a user having ‘examined, downloaded, or otherwise [been] supplied…full content units or descriptive records.’ This is especially complicated for collections that use streaming models, or include content other than traditional documents, music or video (e.g., curricula). All but one of our public libraries host at least 22 collections (the average is 39), so generating a measure of e-retrievals means determining the correct metric and then extracting that figure from many sources. Non-response occurs disproportionately among our smallest libraries (serving fewer than 7,000 people), where the task likely overwhelms capacity.”

PROJECT COUNTER

That collecting and reporting e-retrieval data is not even more challenging can be credited largely to the efforts of Project COUNTER, which originally stood for Counting Online Usage of NeTworked Electronic Resources. Building on previous efforts of the International Coalition of Library Consortia and the Association of Research Libraries, Project COUNTER created, and periodically updates, a Code of Practice that dozens of online content vendors subscribe to and comply with. As with many such large-scale, collaborative efforts, however, the “devils in the details” are the extent to which vendors actually comply with COUNTER’s Code of Practice and the extent to which library managers are sufficiently well-trained to understand and exploit it. Most state library agencies and library consortia do not offer training to library managers to take advantage of COUNTER’s data standards and to navigate the remaining challenges associated with noncompliance by some vendors. In the absence of such training, it is little wonder that many library managers—especially those from very small public libraries—feel confused and burdened by the PLS request for a count of e-retrievals.

HELP IS HERE

At least seven state library agencies provide local data reporters with specific guidance about how to report e-retrievals. They crosswalk from each electronic resource to its particular unit of measurement (e.g., full-text retrievals, abstract requests, sessions, checkouts) to the PLS item, e-retrievals, or to circulation of electronic materials—the other PLS item with which it is most likely to be confused.

If you know of another state library that provides similar guidance, please contact the author so this list can be updated.

LJ INDEX REMINDERS

FREQUENTLY ASKED QUESTIONS

Eligible libraries are grouped by total operating expenditures and rated based on their differences from the average for the seven per-capita statistics. For answers to basic questions about the LJ Index and its Star Library ratings, visit the Star Library/LJ Index FAQ.

BE YOUR OWN STAR

If your library does not appear on the 2020 list of Star Libraries, don’t assume that there is nothing in this project for your library. See the 2015 Star Libraries/LJ Index article (bit.ly/indexscore2015) for several examples of ways you can reanalyze LJ Index scores for your library and a subset of public libraries and come up with your library’s own claim to distinction.

FACTORS AFFECTING WI-FI SESSIONS STATISTICS

If your library is still struggling with collecting and using statistics on Wi-Fi sessions, it may be worthwhile to consider the variety of factors that can affect how high or low your library’s Wi-Fi session count is. These factors are discussed in the 2019 Star Libraries article. 

Keith Curry Lance (keithlance@comcast.net) is an independent consultant based in Boulder, CO. He also consults with the Louisville, CO–based RSL Research Group. In both capacities, he conducts research on libraries of all types for federal and state library agencies, state library associations, and other library-related organizations. For more information, visit www.keithcurrylance.com.