Search Engine Wars Redux | Stacking the Tech

Ellyssa Kroski -- Library Journal, 7/9/2009

Web 2.0 let everyone in the game: it was the age of participation, and participate we did! So much so the  explosion of information available online has left many of us with an uncomfortable sense of information overload.

As the content of the Web continues to grow, we are seeing search engines and other similar services competing to serve our retrieval needs. To access the vast content stores of the read/write Web, these search tools make use of structured and linked data, real-time search, personalization, and more focused filtering techniques. If you’re a fan of buzzwords, you might say we’ve entered Web 3.0, a new era that is motivated by the need to more effectively organize, filter, and access information online.

At least in search engine land, those buzzwords have been flying of late—including talk of semantic search engines (Hakia, Wolfram Alpha), real-time search engines (Twitter Search, Topsy), decision engines (Bing), computational knowledge engines (Wolfram Alpha), and the not-so-recent vertical search engines (Indeed, Scirus).

There is no doubt a lot of innovation is happening in the search space right now, and while Google remains the behemoth with over 78% marketshare that no longer means that there isn’t competition. One thing that we learned from Web 2.0 was that upstart applications which tackled singular tasks (i.e. YouTube for videos, Twitter for status updates, Flickr for photos, etc.) and did them well, could stand with—and be acquired by—giants.

Within a matter of weeks three very different and highly publicized search engines have recently emerged including Wolfram Alpha (May 18th), Microsoft Bing (May 28th), and Google Squared (June 3rd), each with their own unique offerings. There have also been rumblings of a Facebook vs. Twitter Search smackdown in the works, and speculation that Google is quietly planning a real-time microblogging search engine. Are we witnessing a resurgence of the old search engine wars?

Microsoft Bing, a decision engine
Microsoft Bing replaces the company’s previous and not-so-popular Live Search and provides deep knowledge in several verticals including shopping, travel, local businesses, and health. It bills itself as a decision engine, offering searchers insight into “decisions that matter,” even offering a “Sentiment Extraction” feature that scans the Web for user reviews and ratings.

Although these domains may not be of interest to everyone, what is worth noting is the combination of features and interface design that has engaged searchers in the last month. Bing offers options for faceted search refinement based upon search criteria, providing categorized filters for quickly retrieving pertinent information. For example, a search for Bollywood actress Aishwarya Rai presents news, posters, wallpaper, biography, interview, quotes, images, videos, and xRank (popularity ranking), all as possible search refinements, as well as several suggested related searches.

Bing strives to provide “Instant Answers”—offering relevant information in the search result description to save the searcher the time of clicking through to a website. Other time-savers include the ability to mouse-over video search results to watch clips from within their thumbnail previews, and the option to see website previews and summaries by hovering over an indicator to the right of each search result.

Last year, Microsoft acquired the travel search engine Farecast and has incorporated much of their functionality into Bing’s hotel and airfare search. Another recent acquisition put to work is the Powerset semantic search engine, which Microsoft is using to index Wikipedia pages. This new offering may not have the legacy of Google, but Microsoft’s media blitz has Bing standing like 300 determined Spartans in the face of the juggernaut. And Bing has been rewarded by gaining some of Google’s marketshare—setting it up to vie with Yahoo! for second place in the search arena.

Semantic search and Wolfram Alpha
Wolfram Alpha (WA) is called a computational knowledge engine because rather than return a list of results based on relevancy as a search engine would, it computes the answers to questions based on its collection of facts—more than 10 trillion pieces of data so far. And it has the ability to do this while offering the searcher the option of using natural language queries.

In a time of information overload we don’t want to pick through millions of search results—we want our computers to do more of the heavy lifting for us. One way for them to do that is to first organize our information in a structured way by adding metadata to it so that computers can understand it, add context, and answer questions. This is one of the fundamental concepts of the Semantic Web, and it is how Wolfram Alpha is able to achieve such a service. WA is a completely human-curated initiative, with each fact created and entered by the WA team. Searchers can access source information for their queries at the bottom of results pages.

Wolfram Alpha is an excellent tool for ready-reference and almanac style questions such as “What is the GNP of Swaziland,” “How many calories are in a McDonald's hamburger and fries?” and a slew of other relevant examples that are provided on the website. It is also highly recommended for scientific or mathematics-related questions, as WA is built on Stephen Wolfram’s previous initiative—Mathematica, a computational software program. Mathematics professors are already discussing how the engine’s ability to compute sophisticated math problems as well as return step-by-step instructions leading to the solution will affect their courses in the fall.

Google Squared and Rich Snippets
Still listed in Google Labs—an area reserved for experimental initiatives—Google Squared is so named because of its ability to present search results in an easily scannable grid or square. This semantic search tool which does well with queries for categories of items such as dog breeds, U.S. presidents, and Hawaiian islands, scours the Web for structured data in order to assemble its information.

Hovering over any square in the spreadsheet-like matrix will display the source of the information. Searchers can add and remove both columns and rows to further refine or expand their queries. While the interface is intuitive, it is limiting by its very nature and I found many of the results to be either missing or way off-the-mark. I expect it will be a while before we see this one emerge from Labs.

But Google has another semantic search trick up its sleeve unveiled on May 12th called Rich Snippets. With this new feature, Google users will see additional, pertinent information about websites in their results descriptions (similar to Bing’s Instant Answers), that come from structured data embedded in web pages by their creators. This gives website authors the ability to provide custom details about their site that will add value to their results listing, and make determining relevancy easier on the searcher.

Real-time: Facebook vs. Twitter Search
Twitter Search, formerly Summize, is a real-time search engine which captures and returns tweets as they are posted to the microblogging website. This type of up-to-the-minute search technology is particularly valuable for a real-time communication tool such as Twitter where users continuously post status updates. An excellent tool for following online conversations about important news events such as the death of Michael Jackson, or the protested Iranian election, it can also be used to track customer opinions and gather organizational feedback.

There has been a recent surge of real-time search engines developing including Topsy, Scoopler, Friendfeed, and OneRiot (now bundled with IE8). But the most discussed is Facebook’s new real-time search feature which is now being tested by a limited number of community members. This new search capability is particularly notable because Facebook users will soon have the option to post their status updates and share content such as photos and videos publicly, rather than just with their network. Moreover, Facebook’s imminent real-time search will return results from your friends, Fan pages, and those who are sharing their posts publicly.

There is much speculation right now that Facebook is gearing up to compete with Twitter’s Search capabilities, however, for the time being most users’ information is limited by privacy settings. But Facebook’s real-time search will be useful for tracking discussions among friends versus among everyone in the world as with Twitter Search.

One other wildcard in this area is Google. There are currently unconfirmed rumors that the search giant has a microblogging search engine in the works which will operate in a similar way to their existing Blog Search.

Let the games begin
As we watch the competition unfold in the search arena, librarians and information pros will want to make note of trends that could help inform and improve our own search practices and interfaces.

Many of these new search engines are using structured data to provide one-stop solution pages as well as include relevant information within results descriptions to reduce additional clicks. Others are providing much-sought after real-time results which enable searchers to follow and participate in online conversations. And still others are accommodating users’ natural language queries, offering time-saving media and website previews, and presenting useful filters through faceted search results and recommendations.

Whether the search engine wars have re-ignited or not, there is plenty to observe right now as each new initiative strives to provide searchers with the best access to information in their area of specialization.

Ellyssa Kroski is an information consultant, reference librarian, writer, and conference speaker, as well as an adjunct faculty member at Long Island University, Pratt Institute, and San Jose State University.  She blogs at iLibrarian.

Read more Newswire stories:

New Developments in Georgia State U. Copyright Suit

ProQuest To Launch Unified Content Platform

Newswire Feature: Veteran Librarians

E. J. Josey, Legendary Activist Librarian and Leader, Dies at 85

The Political Winners (and Sinners) of the Year

Columns:
Mistakes Were Made: What We Can Learn from the Political Theory Affair | Peer to Peer Review

Shift Your Paradigm and Drink the Kool-Aid | From the Bell Tower

EBSCO Publishing releases Academic Search R&D; Serials Solutions marketing Ulrich’s XML Data Service; Duke Libraries launch DukeMobile 1.1

People

Best Sellers in African History