Real Simple?

Exploring online user satisfaction in Michigan

By Sonya Schryer Norris -- Library Journal, 11/1/2006

“Give us simple!” the focus group demanded. In 2005, we asked patrons what they wanted from a digital library interface. They told us they wanted reliable information. They wanted an uncluttered design. Most important, they wanted a search box on the homepage that would access everything. So we gave it to them.

Founded in 1992, the Michigan eLibrary (MeL) is the nation’s oldest statewide digital library. Michigan librarians support MeL by conducting database evaluations, participating in usability testing, promoting the service to their patrons, adding to a collection of librarian-recommended web sites, and joining MeLCat—the statewide catalog and patron-initiated borrowing service. The Library of Michigan contracts with the Michigan Library Consortium to provide database training and technical and training support for MeLCat. Among other honors, MeL won a Digital Government Achievement Award from the Center for Digital Government this year.

In May 2006, the Library of Michigan partnered with the East Lansing Public Library to conduct a usability study of MeL. We wanted to know whether we were meeting MeL’s goal of providing a usable, understandable, searchable digital library for all Michigan residents.

First, we posted a survey to the MeL site consisting of 20 questions that explored specific aspects of user satisfaction, including effectiveness, efficiency, ease of use, organization of information, labeling, visual appeal, and content (see “Usability Survey,” p. 35). Then we conducted traditional usability testing. Four user groups—students, businesspeople, persons over 65, and librarians—were asked to complete 20 tasks within MeL while the researchers monitored their progress and elicited their opinions about the site.

Getting usability results

What did we discover? Of the 619 responses to the survey, a strong majority of respondents reported that MeL is easy to use (82 percent) and easy to understand (85 percent) and that they usually find what they’re looking for (81 percent). When we broke out the general users from library personnel, we saw that general users were less satisfied with MeL on every measure except how much they liked MeLCat.

While MeL was designed with “every resident” of Michigan in mind, the survey clarified who is actually using it. For starters, two-thirds of respondents were librarians or paraprofessionals. While not a surprise, this did force us to reconsider who our “average” users really are. In addition, only 2.5 percent of survey respondents described themselves as beginner Internet users. MeL users span the age continuum fairly evenly, though only four percent of respondents reported being over 65. It was also a well-educated group: 90 percent had at least some college, and 48 percent had at least a master’s degree.

The current design offers users a choice among MeL’s four main components: MeLCat, a statewide catalog and borrowing service running on Innovative Interfaces Millennium Silver 2006 LE release software; MeL Internet, a collection of librarian-recommended web sites; MeL Databases, a collection of more than 40 subscription databases; and what we call MeL Advanced Search—Innovative Interface’s MetaFind federated search software, which searches across all of MeL’s components. A “Google-like” search box, using MetaFind, also appears on the homepage.

In my research with Angela Semifero and Everyl Yankee, we found that only four of 15 nonlibrarian study participants mastered the components. First-time users didn’t know where to click to complete the tasks. For instance, we asked participants to locate an interactive test-taking resource available through MeL. Users searched across all of the components fairly evenly. Users picked a component from the homepage, or used the homepage search box, and if the item wasn’t found, they simply went on to another, without a good idea of what was in each component.

When it came to the clean, simple design that the focus group requested, we got only lukewarm feedback. “Not confusing but plain” was a typical response to MeL’s simple blue-and-green look. Barely half of survey respondents agreed with the statement, “I think MeL is visually appealing,” and a third were neutral on the question.

The most important difference among usability study participants was not which demographic group they came from but whether they were beginner, average, or advanced Internet users. This was the single biggest determinant of both success and satisfaction. More advanced users were happier users.

Borrowing statewide

“Cool. I think that’s great.” This was the response of a businessperson ordering a book via MeLCat for the first time. In the online survey, 69 percent of respondents agreed with the statement, “I like MeLCat for finding books.”

The MeLCat system was launched in January 2005. There are currently 112 participating libraries, with 40 more being added every quarter. MeLCat has 19.4 million items, with an average delivery time across Michigan of three days. In 2006, the average number of items borrowed in a given week was 3,174. MeLCat runs with interlibrary loan capabilities via INN-Reach and is a multitype library system that is open to Michigan libraries regardless of their local integrated library system.

Participants were asked to order three items via MeLCat to determine whether the ordering process was working for average users. These tasks were spaced out so that participants would have a chance to become familiar with the site. Five of 15 nonlibrarians successfully ordered The Diary of Anne Frank on cassette or CD ten minutes into testing. Approximately 15 minutes later, nine of 15 nonlibrarians successfully ordered a small business management book. By the end of the testing session, 12 of 15 nonlibrarians successfully ordered a documentary about Emmett Till on DVD or VHS.

In essence, MeLCat proved learnable and was mastered with a little practice by our participants who had basic Internet skills. The biggest stumbling blocks observed were:

• not knowing to search for available copies if the first title was not requestable

• not being able to locate materials in specific formats

• confusion over similar-looking titles (such as books about Anne Frank rather than The Diary of Anne Frank).

Few nonlibrarians limited or sorted their search results. Most people scrolled the first page of results until they found what they were looking for or gave up and started a new search. Internet-savvy search terms proved crucial for quick success.

The survey and usability testing elicited high demand for some additional features in MeLCat. In particular, respondents wanted to be able to limit a search by format, hide results that are not requestable, and log in or be remembered by MeLCat so they don’t need to retype their names and library card numbers for multiple item orders.

Finding e-sources

MeL Internet, a collection of librarian-selected web sites, was MeL’s original digital collection. In the online survey, 39 percent of librarians and paraprofessionals reported that they liked MeL Internet for finding web sites, while only 12 percent of the general public reported the same (35 percent of the general public was neutral, and 33 percent said the question was not applicable to them). MeL Internet had two barriers to success: a global search function that worked poorly, backed up by a directory system that most users bypassed in favor of the global search.

With this feedback in mind, we will re-create MeL Internet from the ground up using new software—Internet Scout Portal Toolkit—with a new purpose: to catalog the best of Michigan’s online resources.

Through Library Services and Technology Act grants and other federal and state funds, Michigan provides residents access to over 40 subscription databases. When asked to respond to the statement, “I can easily find the databases I’m looking for in MeL,” 66 percent of librarians agreed, while just one-quarter of the general public agreed.

Advanced searching

What did we learn about federated searching? The online survey found that 83 percent of respondents liked having a search box on the homepage. Yet only 57 percent of survey respondents agreed with the statement, “When using the search box on the homepage, I usually find what I’m looking for.” And, in the end, only 37 percent of respondents reported that they used the federated search capability regularly.

One participant, who was vocal in her dislike of the federated search results screen, strongly agreed with the statement, “I liked having a search box” on the homepage and commented, “What would I have done without it!!” She used the federated search to complete most of the information-finding tasks we assigned her and was often successful, but that success was time-consuming (two to four minutes or longer for each task). Ultimately, she disagreed with the statement, “Overall, I was satisfied by the performance of this web site.” She said that rather than using MeL, she would ask a librarian for help or use Google. She did not delve into the MeL product components or learn to differentiate among them during the testing, instead relying on the federated search while she grew more and more frustrated.

Users want the promise of Google-like simplicity, but when a homepage search box doesn’t deliver on content, it can turn people off to the rest of our resources.

What we learned

Most important, usability testing proved that regular users like MeL. A strong majority of them find the site effective, efficient, and easy to use. The planned redesign in 2007 will build on the strengths that keep our regular users coming back and address the areas where MeL is less successful.

The statewide catalog and borrowing service has been in operation for almost two years. It works well, and people like it. We will continue to build on the success of MeLCat by adding libraries and promoting this service to the residents of Michigan. In addition, we found that the federated search box on MeL’s homepage gives users what they asked for but not what they want. We are considering new and creative solutions to satisfy this user demand in the 2007 redesign.

Each year, we put over $3.5 million behind resources identified by the ever-problematic term database. The next interface will consider additional means of presenting MeL’s databases, including a subject-based approach.

For us, usability testing found that a simple design for a complex resource doesn’t necessarily result in patron satisfaction. Our ongoing efforts to improve access to MeL’s resources will mean that the MeL homepage will be redesigned to be robust with content and visually daring, with a modern, updated logo and a dark purple and olive green color scheme. Look for it by spring 2007.