Online Databases: The Past Catches Up

If you were in library school anytime during the last 40 years, you are probably familiar with F.W. Lancaster and his books on indexing, online searching, or paperless information systems. I recently reread the first online searching textbook ever written, while preparing a Library Trends article (Spring/Summer 2008) in tribute to Lancaster. The book, Information Retrieval On-Line (Melville), by Lancaster and Emily Fayen, appeared in 1973 when only a few librarians and subject specialists were tackling the new world of online databases.

Lancaster and Fayen’s forecasts for the future of online systems were like those of the sf writers of the 1950s and 1960s, portending the discoveries of the new millennium. The authors were remarkably prescient despite their own disclaimer: “It is dangerous to try to predict what is likely to happen. [Our predictions] should be regarded as representing our feelings about what future trends might be or, perhaps, what we think these trends should be.”

They foresaw a great increase in the number of information services accessible remotely, including both large “general purpose systems” that would be used mainly by information specialists and “specialized systems” used directly by the practitioners in those areas. Long before the World Wide Web, they envisioned the duality of the online world we live with, though the reality inverts their prediction—large general purpose systems (think Google) are widely used by end users, and librarians now spend many hours educating users about topic-specific proprietary systems.

While their prophecies that the specialized systems would become more “user oriented” and “easily accessible” have become reality, their wager that these systems would require “comparatively little effort” to work with has not been borne out. And while today’s online specialty systems are clearly easier to use than their predecessors, user instruction remains more important than ever. The authors’ admonition that “computer aided instruction” should be “incorporated into systems” hasn’t proven nearly as effective as in-person training.

Today’s systems are beginning, as they wrote, to “exploit the interactive, heuristic, and browsing powers of the online computer more fully for practitioners in a field, rather than information professionals.” Specialized systems such as Chemical Abstracts Service’s SciFinder and Thomson Scientific’s Web of Knowledge go beyond the basics to be meaningful to their users. New versions of generalist systems such as EBSCOhost 2.0 have built-in user-centered features, and ProQuest’s search widgets allow librarians to create preplanned searches.

Lancaster and Fayen also thought future systems should be oriented to natural language rather than controlled vocabularies. Although mainstream systems no longer require commands at input, most haven’t yet fully achieved the natural language goal. Subject searching is still improved when correct indexing terms are employed, and the best indexing companies (such as ProQuest and H.W. Wilson) still add controlled terms to describe documents.

The expectation that vocabulary search aids at the time of searching would be “incorporated, bringing together synonyms and semantically related terms,” has been realized by some of these systems. WilsonWeb, for example, allows users to pick controlled vocabulary terms or use the vocabulary’s cross references.

The web has driven many changes prophesied in 1973, such as relevance ranking of output, now an important part of systems like Factiva and CSA Illumina. Search engines are ahead of proprietary resources in realizing other predictions, like how Boolean expressions remain the norm for most systems, despite the authors’ desires. Developments such as CrossRef, digital object identifiers (DOI), and link resolvers confirm their forecast of interfaces between systems and full text.

Interfaces with bibliographic packages such as ProQuest’s RefWorks and Thomson’s EndNotes recall the projection of having systems provide “online support to personal files.” Products like CSA Illustrata and Knovel Books support indexing and identify charts and tables as separate objects; they bring us that much closer to predictions of system interfaces with statistical packages.

None of these portents remains controversial; indeed, many researchers wonder why some developments have been only partially achieved. The Internet, developments in technology, and great leaps forward in software, standards, and digitization have made Lancaster look clairvoyant. Keep looking in the coming years for those prophecies that have not yet been fully realized.