The Database Universe

Few would characterize the database industry of the last 12 months as heavenly, but it does share characteristics with the constellations. Some old-time companies are like comets that shone brightly, disappeared from view, only to reappear brighter than ever. Many dot-coms illuminated the sky, only to burn out as quickly as a meteorite. And several industry stars are growing larger, by attracting nearby matter. Meanwhile, companies continue to launch new products as customers travel via the web throughout this information universe at ever-increasing speeds.

Dialog lit up the information skies for several decades. Because of poor stewardship in the 1990s, it went off the radar screen of many libraries. Under Gale Group, Dialog is coming back into view with new leadership, new products, and a return to a customer-oriented philosophy.

On the product front, the NewsRoom database will combine 6500 news and business sources into one giant database available on Dialog, Profound, and DataStar. NewsEdge and Dialog content will be further integrated in the future. Even better, Dialog at last has added linking to full text. Initial linking includes over 8000 journals with links from 20 different files. Users must have subscriptions to the journals for the links to work. Thomson has also moved Intelligence Data from the Thomson Financial Group to Dialog, building on Dialog's business information strength.

On the customer service front, Dialog has revitalized its instruction program and restarted the Quantum2 web site for information professionals. Most importantly, it has listened to complaints about pricing structures and has eliminated DialUnits.

Other longtime companies that never disappeared from view are now more visible. For schools of library and information science, The West Group (Westlaw) is back with an aggressive educational program. ISI's Web of Science is so successful in academic libraries that its citation databases are now among the most heavily used by faculty. Both H.W. Wilson and Factiva are focusing on a complete redesign of their search software, and the latter is moving into the academic market. OCLC's acquisition of the bankrupt netLibrary and its work in integrating new standards and multimedia into the WorldCat database remind us we shouldn't take OCLC for granted.

The press is full of stories that tout the death of the dot-com world, and the information industry has also had its share of flame-outs. netLibrary was once one of the brightest stars. Marketed heavily as the only e-book service geared to the needs of libraries, it was building a notable customer base in libraries-but not quickly enough. After filing for bankruptcy, netLibrary was acquired by OCLC. Most librarians consider this a positive step, and almost immediately a new pricing structure and new software enhancements were announced, including the ability to create bookmarks and open multiple e-books simultaneously.

More information companies burned out this year, and others are likely to follow. iPublish, the e-book subsidiary of AOL/Time Warner, shut down, as did NBC Internet, the Internet arm of NBC. Questia, the research product aimed at college students, is in deep trouble and will likely fold. Aurigin, which provides software for intellectual property searching and analysis, saw six executives (including its CEO) resign and is barely holding on.

Over a third of the firms described here provide gateway access to other systems and services. The most interesting include Scirus from Elsevier Science, which now offers access to the arXiv.org free e-print service, and OCLC's WorldCat, which links to Alibris, abebooks, and other rare book services.

Bibliographic databases are another type of gateway into the universe of full text. Bibliographic databases remain popular-with over 71 percent of the companies listed here offering them. Increasingly these serve as entryways, offering copious links to full texts of journals, magazine articles, biographies, among others.

EBSCOhost users can now link to the MARC records in OCLC's WorldCat database in addition to many journals, document delivery services, book vendors, and online catalogs. ABC-CLIO history databases offer 'CLIO Links' to full articles from JSTOR, History Cooperative, and Project MUSE. Wilson is linked to Infotrieve for document delivery and to D & B for corporate profiles. Even Dialog has added linking to full text in Dialog, DataStar, and Profound.

Corporate libraries in particular want products that serve as gateways to a variety of external information but that also provide consistent integration and searching of internal information. Information content companies are now offering intranet development products that allow special librarians to create a single information system. Factiva and LexisNexis both provide software, controlled vocabulary taxonomies, and services to help with intranet development.

The information universe still is dominated by a few huge corporations, each a galaxy of names of once-independent smaller companies. The acquisition of Academic Press by Reed Elsevier was approved in both the UK and the United States this year. Wolters Kluwer acquired SilverPlatter and merged it with Ovid, and Thomson purchased NewsEdge.

These companies began to integrate better the rich information resources from their current and former acquisitions. Thomson moved NewsEdge and Intelligence Data into Dialog; LexisNexis began adding some Elsevier science journals; and, although still maintaining separate systems, Ovid is moving toward better integrating SilverPlatter with Ovid.

Other major information companies continue to thrive as well. This year ProQuest acquired Micromedia, a stable company that is well known to Canadian libraries for its reference databases such as Canadian Almanac & Directory and Canadian Business and Current Affairs Database and for aggregating and distributing other databases. This acquisition gives ProQuest an immediate strength in the Canadian market. ProQuest also purchased SoftLine, which provides full text from alternative presses.

H.W. Wilson Co. continues to build on its strengths. In 2001, it added retrospective indexing to its major databases like Readers' Guide, continued to integrate its bibliographic and reference products, added more full text, and made its databases available through additional online services.

Several smaller companies had a great year as they made tactical acquisitions and solidified their positions in the information world. CSA moved beyond scientific materials with the addition of databases in the humanities and social sciences. Last year it acquired a number of databases, including Library and Information Science Abstracts and British Humanities Index, when its parent company purchased R.R. Bowker.

Ingenta, Inc. has made several high-profile acquisitions, including UnCover in 2000 and CatchWord in 2001. Early in 2002 it announced a joint venture with Gale's InfoTrac. In just a few years it has become a dominant player in the provision of full-text scholarly articles.

And then there is divine, inc., a company founded in 1999 but that purchased this year, among other information companies, RoweCom, Sagemaker, Open Market, and Northern Light. These ambitious acquisitions immediately make divine a force in the marketplace, by giving it content, search software, and web services. Divine made Northern Light's pay-per-article 'Special Collection' available on Yahoo! and will likely focus on products for the corporate marketplace. Keep your eyes on divine.

Joint ventures continue to bring together sometimes competing companies for one-time deals or to create new products. Ingenta's scholarly journals will be accessible through Gale's popular InfoTrac service, while Congressional Quarterly (CQ) and LexisNexis began cooperative marketing in anticipation of joint product development. EBSCOhost, which has strengths in full text, and ABC-CLIO, which has strengths in indexing and abstracting, worked together to create World History Full Text. Chemical Abstracts Service's TOXCENTER database combines content from Chemical Abstracts, Medline, BIOSIS, and International Pharmaceutical Abstracts.

Existing products are growing with new content, back files, and fuller coverage. Some are also adding more images. OCLC and VTLS will work together to incorporate sound, graphics, and motion into OCLC's WorldCat megadatabase. RLG Cultural Materials is a multimedia collection of digitized sources, including manuscripts, art, documents, and memorabilia. Chemical Abstracts online now goes all the way back to its 1907 beginnings, and H.W. Wilson will eventually offer Readers' Guide Retrospective, starting with its first issue from 1890.

Clusters of topically related materials are becoming commonplace. Gale Group's Opposing Viewpoints Resource Center, for high schools, and Consulta, a Spanish-language Resource Center, join its other topical Resource Centers that include Biography, Business, Health, and Literature. Gale's isn't the only current topics product to join longtime players in this market. NewsBank, SIRS, and Facts On File have long-established products, while other news products include LexisNexis Current Issues Universe, CQ Researcher, and Controversies@facts.com.

Oxford University Press offers 100 dictionaries and reference books with Oxford Reference Online; ParaText's Reference Universe includes indexes of major reference works; and CQ Electronic Library combines access to several CQ products, including CQ Researcher, CQ Public Affairs Collection, and CQ Weekly. There are many other topical combination products available for school, academic, and specialized communities, including CABI's Animal Health and Production Compendium, Roth's Lit Finder, and NISC's Family and Society Studies Worldwide. Facts On File's Curriculum Resource Center provides handouts of maps, flags, time lines, science projects, science experiments, and other curriculum-related materials for the elementary school audience as well as middle school and high school markets.

Users are no longer satisfied searching separate databases. They want to search substantial collections of databases through one interface and with one search, following the Factiva, LexisNexis, and ProQuest models. Integration of all products offered by a company is important, as cross-searching among products becomes the norm. Many companies have enhanced integration within their products, including ISI with its Web of Knowledge platform. Web of Knowledge is also integrated with bibliographic management software, making it easier for users to build their own databases and bibliographies.

Nearly 90 percent of the firms in our survey plan to launch new products this year. LexisNexis has new products and subsets of databases targeted to specific groups and also has software aimed at those groups. For example, PRAnywhere includes tools needed by PR professionals to integrate media contacts, calendar functions, and Nexis articles. Government investigators will benefit from a product that combines LexisNexis public records with i2 Group's visualization and analysis software, which will reveal patterns and connections in the records. A special version of Time Matters software assists lawyers in tracking online searching by client. For the college and school library market LexisNexis announced Current Issues Universe and Environmental Universe.

Despite netLibrary's troubles, e-books are still with us. ProQuest, the Pearson Technology Group, and O'Reilly teamed up to launch a joint project called Safari, which includes texts and technical e-books. Several products-including LINK, Wiley Interscience, and ScienceDirect-now provide linked access to prepublication articles.

To differentiate themselves from free information available on the web, companies are emphasizing access to quality content from peer-reviewed journals. Ingenta hopes its scholarly journals will bring new customers through InfoTrac; EBSCOhost has subsets of scholarly journals for its academic customers. Primary publishers like Nature and IEEE sell their products based on the high quality of peer-reviewed content. The human element in selection is recognized by many as important. SIRS, for example, prides itself on staff selection of articles, web sites, and government documents that are of high quality and appropriate for the customers of SIRS Knowledge Base.

Indexing as a method to improve searching is alive and well. Alexander Street Press is emphasizing comprehensive 'semantic indexing,' where subject specialists index on as many as 300 fields, which include terms that capture concepts, meaning, topics, authors, and more. Factiva is reconciling the indexing from its former Dow Jones and Reuters products to create consistent (and improved) indexing of company names and topics. Westlaw continues to emphasize the importance of its editorially enhanced legal materials and KeyCite indexing.

These products are marketed mostly to librarians, but they are aimed at end users. Librarians want products that combine good quality with search features and design that are easy for their users. Several companies are making PDA-compatible products, so users can check information wherever they are. Companies that cater to the medical market, like Ovid, or to lawyers, like Westlaw, are finding a demand for wireless access on handheld devices.

Pricing continues to be tricky. Companies still offer options that carry the same price for all customers-flat-fee subscription prices, for example. But most pricing is now negotiated between a library or a library consortium and the information company.

Connect-time pricing is going the way of the horse and buggy, with only 11 percent of these firms offering this pricing for one or more of their products. Flat fees per search are also uncommon. Concurrent user subscriptions and site licenses are much more popular in the library market, so libraries can predict their expenditures up front and encourage patrons to search as much as they like. Consortia are still the method of choice both for large deals and to make expensive databases accessible at smaller libraries.

Sometimes the evolving price structures can mean better deals for those that waited. ScienceDirect moved away from charging more for e-journals or demanding that libraries purchase all of the Elsevier journal titles (the so-called 'big deal'). With eChoice, libraries can select the titles they want online and pay the same price for print or electronic subscriptions-or 25 percent more for both. LexisNexis has reduced its prices for public and high school libraries.

Special deals pop up every so often. EBSCOhost is offering subscribers of Academic Search Elite or Academic Search Premier free access to three of its databases that include nonscholarly literature-Primary Search (designed for elementary school libraries); Military FullTEXT; and MAS Ultra: School Edition (for high school libraries). CSA offers unlimited sitewide usage for all of its products.

Library budgets are beginning to weaken, and in state-supported public institutions they are expected to suffer more in academic year 2002-03. Perpetual access once a subscription is terminated is expected to be an issue. Back files are being built, but some libraries can barely afford to purchase access to current products. Even some free information on the web is disappearing. The Financial Times will join the Wall Street Journal and other premium content providers in charging a subscription fee for its end user web version. More disturbing are the black holes of information content.

The Tasini Supreme Court decision in the United States is already having an effect as general news and magazine databases are removing articles written by freelance authors. These articles are disappearing into a black hole, likely never to be seen again. A remnant bibliographic record may remain, but studies show that bibliographic records without linked full text will usually be ignored.

Gannett has pulled all of its newspapers from aggregator services and reportedly won't return them until all articles by freelance writers have been removed. ProQuest Newspaper database subscribers were notified that 62 percent of the Minneapolis Star-Tribune articles will be removed from ProQuest-nearly a half-million articles in all. Aside from such examples, the exact extent of this phenomenon is unknown and a matter of some debate. Tasini removals mostly affect pre-1999 articles and sources that rely heavily on freelance writers. Most publications from 1999 on have required freelancers to sign digital rights clauses. Some papers are removing all pre-1999 back files rather than go to the trouble of removing contested materials. Scholarly journals typically have been less affected.

Other content is disappearing into the black hole of safety concerns. U.S. government web sites are removing materials that range from location of reservoirs to full-text reports on security breaches in airports. (See Notable Government Documents, p. 60-64.) More removals are likely. In March 2002 the White House Chief of Staff ordered all U.S. federal agencies and executive departments to examine their public documents for 'sensitive' information and report to the Office of Homeland Security. Companies are also examining the information on their web sites to see if any could be used for terrorist activities.

If one trend is clear, travel at hyperspeed on the World Wide Web will continue to grow. All the firms that responded to this year's questionnaire offer web access to at least some of their products. Two-thirds of the firms say that web access is the most popular access method for their products. Old-fashioned travel still coexists, however. Nearly 70 percent of these companies provide access to some print products and about two-thirds provide CD-ROM access to at least some of their works. Migration from local loading to web access is accelerating as more libraries decide to let the vendors do the routine maintenance. Some large universities or consortia still choose to load their most popular databases locally, but this is less common than even a year ago.

Links between bibliographic databases and full text, links from full-text articles back to bibliographic records, links from references to full articles and back again are all working toward an information universe where users can travel at will. SFX and Open URL linking technology are expected to transform library OPACs and information resources.

All of this is possible because telecomputing infrastructures, both nationally and at the local library level, are improving quickly. Faster connections and increasing bandwidth will make streaming audio and video commonplace in the near future. Better infrastructure, linking standards, and better software are also improving information products. Yet despite all the recent advances, it sometimes feels as if we are still looking at the heavens through the limited range of a telescope-we can begin to see where we want to go, but we can't quite travel there yet.