Product Pipeline

Melissa L. Rethlefsen looks at social search engines, where search is heading, and what it means for librarians

By Melissa L. Rethlefsen -- netConnect, 4/15/2007

Today's web depends on search engines. Without them, the web becomes completely unmanageable. As much as search engines rule our search experience, most agree current search engines leave something to be desired. New search engines and search tools frequently appear, and the traditional search engines are continuously updating their look and features to remain competitive. Along with Web 2.0 and Library 2.0 has come Search 2.0, a trend toward social, AJAX-rich search tools. Collarity and Sproose are two versions of social search engines, one relying upon dynamically generated communities of interest and the other upon user voting and tagging. Experimental search engines SearchMash and Ask X point to the future of user interface design for the traditional search engines.

Personalized search is rumored to be the next Google-killer. People want results that are personally relevant and current. To date, personalized search has depended on tracking search histories and followed links. While new search engine Collarity relies on just those methods, its algorithms seek to provide results that are granular enough to know if you are the type of person who, when searching for Java, wants results on the programming language or on coffee.

Collarity was built with three concepts in mind. First, your search results shouldn't be the same as someone else's. Second, you shouldn't need to search to find something someone else has already found. Lastly, search results should provide what you want now, not what you wanted last week. To achieve these lofty goals, the company developed the Collarity Compass, an AJAX-driven search overlay that helps users navigate to meaningful search results using real-time-generated related searches and URLs. A slider tool across the Compass top allows the user to move easily from global results (think Google's general search engine) to community results (based on a specific web site) to personalized results. The further you slide toward personalization, the more specific and pertinent the search terms are to you.

How does Collarity's search interface work? Essentially, Collarity continually builds communities of interest behind the scenes based on user behaviors. User behavior information is collected anonymously and unintrusively from web logs on partner sites, granting both higher levels of privacy to users and of accuracy to search results since results are no longer based on yesterday's searches. Communities are clustered groups of keywords, followed links or navigation, and other behaviors that are developed dynamically as people use Collarity to search.

Users can control relevance but without needing to be active content creators or even knowing their contributions. Better yet, because Collarity is unintrusive, it can't be spammed like other social search engines that rely on tagging or other means of inputting data. Personalized results require a log-in, but community-level results are completely anonymous.

Collarity was first designed as a site-search tool, not a global search destination. You can test out Collarity at its web site, but, better yet, test it at one of the sites that has adopted Collarity as its search tool. Currently, Collarity's main installation is its Compass on the Fox TV MyFox web sites; the Bruce Clay blog is another enthusiastic customer. Performance on the Fox network channels could stand improvement (the Compass is often slow to come up and occasionally conflicts with some of Fox's advertising), but it works perfectly with the less ad-heavy Bruce Clay blog.

Using Collarity: When searching at Collarity or a web site using Collarity, the Collarity Compass pops up below the search box and automatically suggests related terms and URLs as you type. Clicking on one of the suggested terms adds it to your search; as you add terms from its suggestions, the related URLs change and more terms appear. Mousing over the suggested URLs shows a preview of the content you can expect—often, it is not necessary to go to the full results. The full results are set up differently on each Collarity roll-out; MyFox results display a tag cloud along with the results. Clicking on a term in the tag cloud adds the chosen term to the search.

The Compass is also set up for each site's specification. For instance, Collarity's own Compass and the Bruce Clay Compass allow users to slide easily between three different sets of suggested terms and URLs: Personal, Community, and Global. MyFox's Compass is set to search at the Community level only. Some Compasses are likewise set up to allow site searching or full web searching, which can be selected via a tab.

Collarity provides a tool for web site publishers, too, Collarity Discovery. It tracks user behavior on a specific web site by displaying the communities of interest Collarity identifies, along with more typical metrics like keyword and URL data.

For librarians: Collarity wants to be your web site's search engine. To get the word out about Collarity and its search mechanisms, Collarity provides library web sites with free (and advertising-free) Collarity Compasses, as long as content is available easily through public indexes or other means. Collarity wants to work with libraries for better access to libraries' licensed databases, which could perhaps lead to a future method of federated search. Collarity's emphasis on changing relevance, personalization, and behavior-based results makes it a technology to watch.

Where Collarity prides itself on its unintrusive, implicit way of incorporating the wisdom of crowds into search, Sproose's search results are built upon explicit user behaviors: tagging and ranking. User accounts are required to tag and rank search results, though temporary ranking is possible without an account. Sproose has 250,000 registered users and over one million ranked web sites. Tagging doesn't seem as popular; one of the “popular” tags I tested was only used four times.

At the moment, search results are fairly bad. A search for “sproose” doesn't even pull up Sproose as the first entry; the first result is a dead link, and the second is completely unrelated. The third result, which was user-ranked with four stars, appears to be for the Sproose contact page but redirects to a spam site. Obviously, some kinks need to be worked out. Over time, as more people actively rank and tag results in Sproose, results should improve, but Sproose will remain vulnerable to spammers, as every successful social software site is.

The MySproose section, which lets users bookmark results, create a blog, and keep track of their previous votes and tags, and the discussion functionality, where users can discuss any web page, make Sproose unique and interesting to watch.

For librarians: Social search is one of the hot trends in search. From Sproose to Prefound to Wikiasari to Eurekster to Trexy to Wink, social search is emerging as a possible competitor to traditional search. Search tools like Sproose, Prefound, and the future Wikiasari hope to harness the huge swell of user power and content creation to develop a better search engine. Prefound wisely gives searchers a backup Google search if results don't pan out. Social search tools like Eurekster's Swicki, Rollyo, and Google Co-op's Custom Search are combining social search together with vertical search—peer content creation and curation on specific topics. Social search, in all its flavors, is guaranteed to be a continuing trend, though Sproose shows the growing pains a user-contribution-based search engine experiences.

Experimental search site SearchMash allows Google to experiment with search results display and functionality. Google makes no promises about the site's availability or continued operation, nor does Google suggest that SearchMash features will necessarily be added to Google. Rather, it is one of the ways Google is testing how best to improve the search experience for users; functionality is continually changed and updated. It also happens to be a slick and stylish way to search Google.

Searching for “peanut gallery” in standard Google and SearchMash uses the same search engine index, but SearchMash throws in a number of added features without cluttering up the interface. In SearchMash, the search “peanut gallery” displays a set of thumbnail images from Google Images on the right-hand side, much like Google OneBox results in the traditional Google interface. In addition, expandable categories of blog, video, and Wikipedia results are available. The right-hand column displays different types of information based on each search; a search for “California” shows a map of Sacramento along with expandable listings for images, blogs, videos, and Wikipedia, whereas a search for “library 2.0” shows only the expandable listings.

Certain searches trigger options to refine results using tabs. A search for “diet coke” spawns results with the tabs Recipes, Production, Nutrition, and Restaurants; a search for “garbage” includes tabs for Lyrics, Songs, Influences, Albums, and History. Clicking on a tab refinement adds that term to the search. The SearchMash tabs appear to have little or nothing to do with either Google Co-op or the other current refinements on Google's results. A search for “goldeneye rogue agent” on Google produces a whole bevy of refinements at the top of the results (cheats, guides, screenshots, etc.), but SearchMash doesn't offer refinement tabs for the same search. Instead, SearchMash poses the question, “Are you looking for goldeneye rogue agent cheats? Yes / No.” Searches that would normally bring up Co-op refinements and results in Google don't in SearchMash.

The “infinite scroll” is one of SearchMash's nicest features. At the bottom of the results is a “More web results” link—clicking on this link pulls up ten more results immediately below the previous set. Even better, hitting the space bar in SearchMash twice automatically opens the next set of results. The infinite scroll feature seems to work by preloading the results in the background, making it a lot faster (near instantaneous, even) than advancing through pages of results on Google.

Video search results are particularly slick. The search results show thumbnails of each video. Clicking on a video launches it above the results, without having to go to YouTube or another location.

Using SearchMash: If you are familiar with searching Google, you are familiar with searching SearchMash. The major differences are the lack of an advanced search page, no OneBox results, and the navigation. Switching back and forth between the popular Google search engines is possible to some degree by expanding the right-hand boxes and choosing to see more results, but some advanced operators get dropped. For instance, searching for “site:cdc.gov intitle:monkeypox” in the web search gives predictable results, but switching to the image search engine drops the site limitation.

Links to a SearchMash image search or blog search aren't available from the main page, unlike the main Google search. You can force an image search from the main page by starting your search with “images:” (i.e., images:intitle:monkeypox), or likewise a blog search using “blogs:” or “video:” for a video search.

For librarians: Keeping tabs on Google is critical to our profession, so getting a sneak peek at possible improvements to the Google engine through SearchMash is valuable. More than that, though SearchMash can be unreliable, its stripped-down (and ad-free) interface to Google makes searching easier. Its infinite scrolling alone is a major timesaver. Using SearchMash is also a way to help improve Google, since every SearchMash result asks for feedback on whether the web, blog, images, Wikipedia, and video results were helpful, and many more searches spawn other questions in need of feedback.

Ask X is Ask's test bed for interface design, quite similar to Google's SearchMash but with Ask's functionality. Ask X results display in three panes. The left pane has the search box, dynamic suggested searches, and ways to narrow or broaden searches; the right pane is similar to SearchMash's right column, displaying relevant images, encyclopedia results, blog posts, or other categories of information depending on the search terms; and the middle pane displays the search results. Like Ask, Ask X has Smart Answers, binoculars for seeing web page previews, and the option to save results to MyStuff. Unfortunately, the Ask X interface is slower than Ask and leaves out critical navigation features, such as a link to access the links saved in MyStuff.

The three-pane interface lays out information in an easily digestible format, with the right-hand pane being particularly useful. A search for “china,” for example, displays images, a dictionary result, and the current weather and time in the right-hand pane. Unlike SearchMash, this information doesn't have to be expanded by the user—it's automatically visible. The middle pane starts with the same Smart Answer as in the traditional Ask engine (population information, capital, etc.), but its top-of-the-screen placement requires less scrolling to view search results.

For librarians: Ask has recently received a lot of praise from librarians and search engine experts alike for its Smart Answers and other helpful options—Ask X's experimental interface is a way to get even more out of Ask. Small navigation problems and slow speed are minimal barriers to successfully using Ask to answer questions.

The options for the future of search are seemingly endless: social search, personalized search, vertical search, and specialized search in all varieties. One thing is clear: innovation in search hasn't died. It is only beginning. Just as Google started in a garage as a Stanford digital library project, the future will likely come from humble beginnings.