Academic discovery solutions from EBSCO, Clarivate, and OCLC are integrating artificial intelligence (AI)–powered tools, including research assistants and natural language search features, to simplify the early stages of new projects and identify relevant resources more efficiently. In public libraries, the discovery layers of integrated product suites from BiblioCommons and Communico are helping users find out more about what their library offers every time they search for content. Advancements in discovery continue to be crucial, as academic libraries compete for screen time with large language model (LLM) chatbots and public libraries deal with patron expectations raised by streaming services and online retailers.

AI ASSIST

EBSCO’s AI Insights was recently launched as an optional, no-additional-cost feature for EBSCO Discovery Service (EDS) and EBSCOhost that, if enabled, can accelerate the research processes of students, faculty, and librarians by generating summaries of full-text articles. Highlighting two to five key points of a selected article, AI Insights is designed to help researchers quickly determine whether that article is relevant to their research.

Ashleigh Faith, director of AI and semantic innovation for EBSCO, says that the new tool is already proving popular with undergraduates and others who are researching a new or unfamiliar topic. In the research process, there are a lot of steps “to getting to the right information that you then need to assess,” notes Faith, describing AI as a tool that can help users reach the assessment stage more quickly.

Separately, natural language search also recently debuted for EDS, enabling users to create queries using everyday language, similar to what they might use with Google or other internet search engines. “A lot of librarians and students have told me that they feel it’s freeing,” Faith says. “They can [make] queries without being worried about using exactly the right words.”

MORE TO DISCOVER WorldCat Discovery features integration with WorldCat.org and a central index of almost 3,200 collections (top). Communico’s new Explore discovery layer surfaces books and other content, events, library services, and more with a single search.

Clarivate has launched the generative AI–powered Primo and Summon Research Assistants for its discovery layers, which are also optional features, currently deactivated by default, for Primo VE and Summon customers. Both enable researchers to use natural language queries to retrieve five articles from academic databases in response to a search. The tools use retrieval augmented generation (RAG) architecture, which involves a LLM AI built on material from Clarivate’s Central Discovery Index (CDI). It generates responses based on query conversion: First, the user’s natural language query is sent to the LLM, where it is converted into a series of Boolean queries, with variations connected by “OR.” The Boolean queries are sent to CDI, and up to 30 top results are re-ranked using embeddings (numerical representations of words or phrases that help clarify their relationship for the purposes of the LLM) resulting in five sources identified as best addressing the user’s question. Those top five sources are then sent back to the LLM with instructions to generate overviews of the articles with inline references to send as a response to the researcher.

A few issues that surfaced during a beta test have been resolved, notes Yisrael Kuchar, senior director of product management, discovery solutions for Clarivate. For example, when users were taking advantage of the natural language search capability, they might query something like “global warming articles published since 2022.” “The word ‘articles’ and the word ‘2022’ were part of the query itself,” explains Kuchar. This was generating unexpected results. “As we evolved, we were able to teach it, ‘OK, this is not part of the query, it’s a filter to apply,’” and natural language queries began narrowing searches by time frame, resource type, and other filters. The new research assistants now have been activated by more than 300 institutions globally, with an additional 500 trying them out to some extent, says Kuchar.

“There are some libraries that are extremely hesitant to enable these kinds of [AI] features and others that are more enthusiastic—and, of course, many in in the middle,” notes Marshall Breeding, independent consultant, author, and creator and editor of librarytechnology.org and the libraries.org directory. “Your main search box on the library web page, that’s kind of representing the library. Is the library willing to stand behind something it doesn’t understand or that provides inconsistent results?”

Breeding notes some parallels to the early days of index-based discovery layers 15 to 20 years ago, saying that many librarians then were also “uneasy about the differences—often small, sometimes bigger—from one search to another.” However, as Kuchar and Faith both emphasized, the new features are optional for end users. Advanced researchers still have access to standard filters, facets, and other tools for refining searches.

UNDER THE HOOD

One of the primary benefits of OCLC’s WorldCat Discovery service is its deep integration with WorldCat.org, the organization’s library-collaborative database of 559 million bibliographic records. Combined with a central index of 3,156 collections from leading publishers—including 122 million open-access resources—OCLC claims that WorldCat Discovery generates unbiased search results from more than 4 billion items from a variety of resources. Although OCLC recently debuted AI-generated book recommendations on WorldCat.org, the organization has primarily used AI behind the scenes, developing a tool to speed the de-duplication of records in WorldCat, for example, and using machine learning to analyze transaction data in its WorldShare Inter-library Loan network and speed up turnaround times.

During the past few years, OCLC has also used machine learning to optimize the relevance algorithms of WorldCat Discovery. As users click on different parts of their search results, those interactions are anonymized and used to run automated tests.

“We don’t send any sort of user data into that system—it’s all anonymous; there’s no reference to who the person was that made the click,” says Jay Holloway, director of end user services, global product management, for OCLC. Assisted by machine learning algorithms, the collected data eventually enables WorldCat Discovery to improve the ranking of the results researchers appear to be finding most useful. This type of use “fits perfectly with how OCLC wants to use AI—we want to use it with human oversight, and we want to use it in a way that aligns with library ethics,” Holloway says.

He also notes that WorldCat Discovery has the functionality to replace subject vocabularies with locally preferred terms, and thinks AI could potentially play a future role in facilitating that work. “We built this feature, but one of the challenges I’ve seen with it is the adoption of the functionality requires that a discovery librarian understands quite a bit about metadata,” Holloway says. “I’m curious if this might be something where AI could help.”

THE FULL TOUR

Protecting patron privacy is a core library value. But for years now, patrons have been using services such as Netflix and Amazon, which collect oceans of data on user habits—drilling down to the moment when their customers turn off a TV show or the speed at which they read any specific page of an ebook—in order to offer personalized recommendations. These commercial services have raised library patron expectations across the board. BiblioCommons and Communico both offer commercial discovery solutions for public libraries aiming to meet those expectations while continuing to maintain high privacy standards.

For public libraries, discovery layers traditionally have been good at record grouping for end users, notes Paul Quelch, founder and CEO of Communico. They enable users conducting a search to see “all the formats that a library has for that particular title or item,” displaying the library’s holdings of hardcover, paperback, ebook, audiobook, and other versions of a title using a minimal amount of screen space.

But discovery layers should do more than that, Quelch says. “Books now account for, I think, 30 to 35 percent of what the [public] library does,” he says. “So when [patrons] do a search for ‘Harry Potter,’ they should also be discovering all of the things that the library offers in relation to that search term. It could be that the library has a program or an event around Harry Potter.... It could be that its library of things has a magic kit,” for example. 

UNCOMMON SERVICE BiblioCommons’s new Browse and Discover interface for BiblioCore showcases reading lists as well as pathways to browse by age group, genre, best sellers, and much more.

Communico has added a new discovery layer called “Explore” to its suite of products. The Mid York Library System consortium in upstate New York will be the first to debut it this summer. Explore will be integrated with the company’s existing Create content management system (CMS), Broadcast digital signage solution, Connect customer app, Attend event promotion tool, Reserve meeting room and equipment reservation interface, Roam library-staff application for mobile devices and tablets, Interact self-checkout, and Schedule appointment-based services module. Working in concert with these other modules, the goal of the new Explore discovery layer is to help surface “everything that the library has that might be relevant” when a patron conducts a search, Quelch says.

In addition, the modules all work together when library staff make updates such as new book lists. And some content—such as ebooks and online magazines—can be set to automatically refresh when updated by vendors. “Record sets that you’re creating can be reused across the whole platform,” Quelch says, with updates potentially refreshing a library’s digital signs, website, email marketing campaigns, patron-facing app, and more, depending on the library’s configuration of the system. “So it saves a huge amount of staff time.” Regularly updating what patrons see when they come to look for library resources encourages more frequent visits, and the Communico modules working together, Quelch says, prompts continued exploration, and encourages patrons to discover the many resources their library offers other than books.

BiblioCommons has long followed a similar strategy with its BiblioCore discovery layer, which works with its BiblioWeb website builder/CMS, BiblioEvents events manager and patron-facing calendar, BiblioEmail integrated marketing solution, and BiblioApps for patrons’ mobile devices.

Breeding describes BiblioCommons as “an example of what a public library discovery interface ought to look like. It ought to be easy to use and have more patron engagement features” than a simple catalog.

As many as three-quarters of patrons don’t search a library’s website when looking for content, says Raena Morrison, VP of sales and marketing for BiblioCommons. “That’s what makes it really important to be able to discover everything through the catalog.” BiblioCore also features an analytics platform “that will tell you how your promotions are performing and if you should change them, or if you should put them somewhere else” within the BiblioCommons suite, she says.

The platform offers users a “unified single experience across all of the products,” she adds. The suite “will do things like show events in your search results, or you could show a reading list in your blog. And it’s automatic. It can be dynamically updated, so [staff] set it and forget it.”

WHAT’S NEXT?

BiblioCommons is also currently working on an AI initiative, although Morrison couldn’t disclose many details and notes that a product may not even come to market. “It’s critical that these tools maintain the [public’s] trust and uphold the core values of libraries, and that includes providing accurate information, protecting privacy, and eliminating bias,” he says. “We’re working with our libraries to see if we can achieve these things now. It will be great if we do, and we will roll it out and probably make it optional, but…we’re not just checking a box. We’re not just going to throw out an AI assistant to say we have one.”

And like many other library vendors, BiblioCommons continues to work toward striking a balance between patron privacy and suggesting resources, programs, and content that individual library users will enjoy.

“In general, we’re seeing a shift toward that more personalized, context-aware discovery that mirrors commercial discovery like Amazon and Goodreads,” Morrison says. There are a few context-specific recommendation enhancements that can be made for anonymous users. For example, with BiblioCommons, if a user switches the language selection from English to Spanish, it won’t just change the language on the library’s site—it will change the displayed book lists and collections to prioritize the library’s Spanish-language holdings, she notes.

But there is only so much personalization any system can do without collecting and retaining personal information. Vendors who discussed patron privacy with LJ in the context of this feature agree that clarity and full disclosure are table stakes in the library field whenever usage data is being collected and retained for personalization of search results.

“What we’ve learned is that if you let patrons know that you’re going to be using their data and give them the option to participate, that’s a much more effective way of engaging your patrons and doing personalization than just taking the data, even if you’re going to anonymize it,” Morrison says. Even actions as simple as clicking on a star rating for a book in BiblioCommons will alert patrons that the information about their rating will be retained. “We have things like that throughout our catalog that are really explicit about how we’re going to use that information,” she says.

For more general recommendations based on broader usage trends, streamers such as Netflix and online retailers like Amazon “need a huge amount of data” for their recommendation engines to work, notes Quelch. “If what you’re trying to do is spot trends—people that check out this book also are interested in this type of item and attend these types of programs and events—that’s just one trend. You need millions of these [trends] to make it really powerful.”

Even large systems such as New York Public Library and Toronto Public Library can’t create similar recommendation engines using only their own anonymized data, Quelch says. At a recent LJ Directors’ Summit event, he gave a presentation in which he asked the audience whether they would be willing to share anonymized usage data with other library systems to help build an industry-wide recommendation engine, and the majority of directors in attendance seemed open to the idea.

“You have to strip out all the personal information,” he adds. “Then you can start to build a model that works for the customer but also works for the library and their  philosophy on patron privacy.”     ■