Building Bearcat

The Newman Library at Baruch College, CUNY, had been looking for new approaches to help increase awareness of the more than 200 databases available to its users. Urged by a desire to help simplify the search experience, we began to look at a number of federated search products on the market that would allow users to search across multiple databases via a single interface.

We observed that many students only use databases they already know. For example, LexisNexis is one of the more recognizable databases to our students, and many of them look there first or even exclusively instead of searching in more appropriate databases that might better meet their information needs. We have also observed that locating databases by subject isn't very easy when one relies solely on the database's name, as many are totally nondescript or don't lend themselves to subject identification. What are students thinking about when they find databases with names like Hoover's Online? Vacuum cleaners?

And then there's the problem of multiple disciplines and access points. One of us encountered a student searching for articles about “the curse of the Bambino,” the nearly incurable supernatural malady acquired by the Boston Red Sox in 1919 when they traded Babe Ruth to the New York Yankees. What subject does this fall under—News? Religion? Statistics? It's likely the average person won't ponder this any longer than it takes to invoke Google.

We decided that a federated search tool could mitigate this kind of behavior and serve as an effective tool for performing interdisciplinary research, as well as aid in the discovery of new databases. We hoped that federated search might help new users who don't know exactly where to begin and those who may feel intimidated when faced with the sometimes overwhelming variety of databases we offer at Baruch.

In spring 2006, the Newman Library selected Serials Solutions' 360 Search to do the job. But we soon discovered that selecting the right product was only half the battle—we still needed to get the product up and running before our investment would begin to benefit our users.

A setup committee of five librarians was created, chaired by our head of collection management, who works with Serials Solutions regularly and has experience setting up electronic resources. Other members include reference and instruction staff with varying experience and departmental duties at the college. This provided a good mix of people with differing expertise and helped ensure that none of our constituencies would be neglected.

Before we actually set to task, we looked first to the literature to get a handle on how federated searching works behind the scenes. We soon learned that much of what happens there involves reconciling the different record formats of the various content databases.

For example, a query to a federated search system undergoes a complex communication with the databases (see Figure 1, p. 7). After a query is submitted, it is translated into a form understood by databases using a connector built by the federated search vendor. This translation is sent to the database via a number of possible gateways (e.g., Z39.50, XML, or HTML) where the search is then conducted. A response is transmitted back to the federated search product to be sorted and organized in accordance with the structure of the federated interface. Of course, each of these connections can also fail on occasion, and other problems can easily happen anywhere along the way. Getting results from a variety of gateways means they may show slight differences in formatting, or may not even be relevant to the search.

Armed with our newfound understanding of the products and the field, we decided to implement our federated search tool in the following series of phases:

Phase 1: Setting it up

Phase 2: Testing and tweaking

Phase 3: Launching publicly

Phase 4: Optimizing and expanding

Our first phase required making a number of decisions that proved to be more time-consuming than we had anticipated. The process began in earnest when Serials Solutions sent us an information package including a list in the form of a cumbersome 2000-line spreadsheet with all known databases the product could connect to, a customization form, a best practices guide, and a user guide.

We began by reviewing the connection list to determine which of the databases we had access to and then chose which databases to federate. We soon found out that we couldn't federate all of them even if we wanted to as Serials Solutions advises against federating more than 150. We decided to leave out certain databases, such as those that permit only one or two simultaneous users, since inclusion would render them continually unavailable. We also chose not to federate those databases that don't support text searching and those that are numeric in form (for example, WRDS or NetAdvantage).

We also had to determine whether we should include reliable surface web resources like Euler and Google Scholar. We envisioned that their inclusion would give our students good reason to use the federated tool in lieu of Google and added them to the list.

Our next task involved organizing the included databases by subject. We started with our library web site's subject list of databases and expanded the number of subject assignments given to each. Our hope was to increase the likelihood that users would obtain the broadest set of results possible. Fortunately, there were no limits to how many subjects we could assign to a particular database. This part of the process was a bit onerous, as we struggled to make sure our subjects were well represented by more than one database and tried to assign them consistently.

Then we had to design the look and feel of the federated system. We selected from a series of options provided by Serials Solutions that closely resembled the rest of our web site. But here, too, we had many decisions to make. Did we want the default to be the basic or the advanced search? The subject tab or the database tab? We also had to devise the results pages, down to the text of the error messages.

The committee had many discussions regarding branding. From our own investigation of other federated search pages at academic libraries, it seemed that those systems that made a lasting impression featured a memorable name. We eventually settled on Bearcat, already familiar to students and faculty because it's our school mascot. With the administration's approval, we set to work on the logo (see Figure 2).

While it was exciting to have so many options, it was difficult to visualize many of these design choices as there was not yet anything concrete to view. Unfortunately, we found that the Serials Solutions demo site didn't provide enough of a realistic feel for how the tool would perform with our specifications. With limited experience using a live federated search tool, we found it frustrating not to see exactly how our ideas would pan out.

After settling on the many options, we sent them along to Serials Solutions and six weeks later received a live URL for our product. We were ready to begin Phase 2: intensive testing.

We saw testing the performance of Bearcat and making changes needed for improvement as critical elements in the implementation process. Our chief concern was to make the product as user-friendly as possible. We worked through a process of trial and error and compared each of our search results against those provided via the native search interfaces of our databases. We were surprised at the number of unexpected outcomes and error messages we found. We were also challenged by the occasionally inordinate amount of time it took for results to be displayed.

Irrelevant results suggested that fields chosen on the federated search page may not have been supported field categories in a few of the searched databases. In some cases, the team discovered that deduplication had failed. While it was equally important to make sure the sorting and date limiter features worked, these features didn't pose as many problems as word searching.

We next introduced Bearcat to other librarians at Baruch and enlisted their help in testing the system. Together, we conducted a variety of different searches employing Boolean, phrase, and field searching (keyword, title, author, or subject).

To help keep track of problem searches, we made use of our Newman Library wiki to share information with one another. As a mode of communication among Baruch librarians, the wiki facilitated collaboration and ensured transparency regarding the work and progress of the committee. Yet, despite our labors, it initially appeared there was little that could be done to influence the way Bearcat performed. Serials Solutions helped explain and resolve some of the inexplicable phenomena, and performance improved.

Once things started running more smoothly, we created an instructional page using the materials provided by the vendor and incorporating some of our own findings. Throughout the process we found Serials Solutions to be knowledgeable and responsive. In particular, changing the default field search from keyword to title provided better results. Not surprisingly, this has to do with a lack of standardization among databases in how search fields are defined.

After testing and sharing our findings with one another and the vendor, we were on the brink of Phase 3, our public launch. The actual Bearcat rollout was divided into prelaunch and postlaunch periods. Before the launch, we thought it was essential to prepare the Baruch community for this new tool. During this time, librarians not only tested the system but also began to consider how they would use Bearcat to assist users. Many of the comments and suggestions our colleagues offered, from interface design to system performance, helped improve Bearcat.

Another key initiative during the prelaunch was the involvement of the Newman Library marketing team. Early on, this team started creating awareness for our new search tool via a featured article on Bearcat in the school newspaper, the Ticker. As the launch date approached, the marketing team stirred further interest and anticipation by designing posters and flyers with the help of students in the student-run Direct Marketing Resources Center, located within the library. These students were some of the very first nonlibrarian users of Bearcat and offered valuable insight on how other students were likely to use it. Although there were some initial problems, these students were very positive about the ability to search across many databases via one convenient search interface. To expose users directly to the forthcoming Bearcat, the marketing team also set up a computer kiosk in the main Baruch computing lab, prominently displaying the Bearcat logo. The kiosk allowed students to try out Bearcat and comment on their experience. While the kiosk was not used as heavily as we would have liked, it was one of the many efforts that led up to Bearcat's successful launch.

On February 25, 2008, Bearcat search was unveiled to the Baruch community. To prompt and encourage its use, librarians demonstrated it at the reference desk as well as in their various teaching capacities. Although we only have two full months of statistics to date, usage seems to be quite robust. In March, we had 2,149 sessions with 4,435 searches; in April, 2010 sessions with 4,048 searches. It seems that when people use Bearcat, they tend to keep using it, which is encouraging. Even better, early comments show that 93 percent of respondents were satisfied with their search results and would consider using Bearcat again. These early users were keen to mention some of the benefits of searching Bearcat, such as the time saved and the new research uncovered. Yet other early users noted areas for improvement, mostly related to performance. For example, some users suggested a faster search system, the further subdivision of subject categories, and the placement of Bearcat on the library's homepage.

The Bearcat launch marked the beginning of our ongoing fourth phase of development, in which we continually look for ways to improve the product and expand its use. As part of this phase, the library plans to conduct more formalized usability testing with students and faculty. This will allow us to see how people actually work with Bearcat and discover if our assumptions prove to be correct. Will users notice that default search is by title instead of keyword? Will they make the connection between results and the originating databases? Will users forget that they can access materials by using individual databases? With further study, we also want to determine the long-term effects of federated search on our subscription databases. We hope to analyze our statistics in greater depth and find out if the use of databases included in Bearcat grows, and if the use of those not included declines. This may affect our future collection development plans.

We would also like to know if increased visibility translates into increased use. We have the ability to place specialty search boxes on our subject guide pages and homepage and in various other locations. For example, we can put a search box on the accounting subject page that would search across all the databases we selected for accounting. This would mean a more seamless search experience.

We expect that as federated search becomes more widely adopted in libraries, database companies will find it in their best interest to cooperate with vendors and perhaps better standardize search and retrieval functionalities. As this happens, we hope federated searching returns better and more relevant results and that we will be able to connect more users to more of the databases to which we already subscribe.

With these improvements and further study into user interaction with Bearcat, we hope to gather insight regarding any relationship that may exist between federated searching and information literacy skills. Will federated searching lead to the erosion of critical thinking? Will users assume the information they seek doesn't exist if they can't find it in Bearcat?

Collecting this information will allow us to provide better and more informed instruction in both face-to-face and digital interactions. Indeed, everything we currently teach in terms of search strategy, resource selection, and evaluation will continue to be directly related to how well Bearcat works in the future.