Search And Destroy

By Lev Grossman; Maryanne Murray Buechner

The first director of the great library of Alexandria was a very smart man, and we all owe him a big debt of gratitude. When he showed up for work on his first day--that was around 300 B.C.--he realized he had 500,000 papyrus scrolls and no way to organize them. He had stumbled on an important truth: You can have more information at your fingertips than any other human being in history, but it won't do you much good if you can't find the piece you want. He solved the problem by ordering the scrolls alphabetically, a principle that survives to this day. Too bad his name was Zenodotus.

Today we're all Zenodotuses. Each of us has access to more data than anybody has ever had before, but still the trick is to find what we want when we want it. Given the size of the Internet--estimates run around 500 billion documents and counting--that's no easy task. Internet surfers do about 550 million Web searches a day. Amass that many people anywhere doing anything and somebody is going to try to sell them something. The market for advertising to those Web searchers is worth about $2 billion, and it's growing at a rate of 35% a year--far outpacing any other advertising medium. What's more, Google, the reigning sultan of search, is looking vulnerable. The combination of big money and big opportunity has attracted some mighty big players, including Microsoft, Yahoo and Amazon. There's a street fight brewing over Internet search that will make the browser wars look like thumb wrestling.

But for a minute forget about the big numbers, the millions of customers and the billions of dollars. Think about what's at stake culturally and socially in the search wars, and all those zeros start looking pretty paltry by comparison. The Internet is swiftly becoming the primary repository of the bulk of human information. Search is the way we get at that information, and companies like Google wield enormous power. They reflect our common interests and shape how we learn about the world with their rapid-fire search results. This isn't just about dotcom juggernauts duking it out for stock options and bragging rights. Whoever wins the search wars owns the keys to the kingdom of knowledge. That's a big responsibility. Are search engines up to it?

It's kind of surprising to think that for most of the 1990s, very large corporations fought for the privilege of helping us search the Web. After all, there was no money in it, at least not directly--searching is free. Everybody assumed that one day somebody would figure out a way to reap dollars from it. But what's even more surprising is that the first round of the search wars was won by two twentysomething Stanford graduate students named Sergey Brin and Larry Page. In 1998 Brin and Page invented a new kind of search engine, one that assessed the importance of a Web page based not on a simple keyword search but on how many and what kinds of websites link to that Web page. Their approach delivered search results that creamed the competition's, and it served them up in a simple, quick-loading, no-frills format. It was a stone-cold category killer. Brin and Page called their search engine Google. "It was fun, it was short, it was reasonably easy to spell," Page remembers. "We like doing big things, and googol [the mathematical term for a 1 followed by 100 zeros] is a huge number."

Today Google is a highly unusual company, one run by true technologists with a genuine love of banging on things, shaking and breaking them, and making them better. Behind the simple, unassuming Google home page is a wizard's workshop of experimentation, much of it useless, some of it brilliant. "Invariably we try 10 things that don't quite work out in order to do one thing that's successful," says Page, who speaks in a slow, deeply nerdy, singsong voice. "And we learn a lot in doing the 10 things that didn't quite work." For example, you can call a phone number in California--it's 650-318-0165--and do a Google search over the phone. Why would you ever want to? Who knows? Or go to catalogs.google.com and you'll discover a service that searches 6,000 mail-order catalogs for you. Some lost soul at Google literally sits there and scans catalogs--J. Crew, Pottery Barn, whatever--into a computer, glossy page by glossy page, so you can have the convenience of searching them online. It's incredibly useful--fortunes have been made on ideas not one-tenth as useful. But Google does not even bother to promote its catalog service on its front page. To the Google gang, it's just a clever toy.

Do they even make money from it? "No," Page admits affably. "Google's mission is to organize the world's information and to make it universally accessible and useful. And catalogs are part of the world's information." But let's get this straight, you aren't doing this stuff altruistically, for the general good of humanity, are you? "Well, we kind of are. We always kind of figured that if we did a good job of providing the right information for everybody in the world, all the time, that would be an important thing to do."

That kind of attitude should have sent Page and Brin directly to the Home for Penniless Geniuses, but instead they have managed to build one of the strongest brands on the Internet--this despite the fact that when they started, they knew nothing about marketing. "That was true," says Page, laughing delightedly. "I guess we were really lucky, you know?" Whatever they did, it worked better than their rivals' approach. Remember how hard Yahoo tried to turn its brand name into an everyday word--those TV ads with the guy with the big Afro asking "Do you Yahoo?" If you do Yahoo, you probably don't call it that. But chances are you Google.

This year Google will make an estimated 30% profit on revenue approaching $1 billion, according to Safa Rashtchy, an analyst at U.S. Bancorp Piper Jaffray. Right now 32% of all Web searches go through google.com That number shoots to around 70% when you count searches on sites like AOL.com which licenses Google's technology. Meanwhile, in the real world, Google has just finished moving its 1,300 employees into new headquarters in Mountain View, Calif., previously the offices of former Silicon Valley golden boy (and current Silicon Valley crater) Silicon Graphics. Early next year Google is expected to offer stock to the public, an IPO that should raise something like $2 billion in cash. That would make it the largest high-tech IPO ever. The company as a whole could be valued as high as $20 billion after the sale.

It's an amazing feel-good fairy tale, but it may not have a happy ending. Conventional wisdom has it that nobody can stop Google, but conventional wisdom is often wrong. When Google debuted, its technology was radically superior to the competition's. Since then, however, that technical advantage has narrowed to a sliver of its former self. Some analysts--Danny Sullivan of SearchEngineWatch.com for example--believe there are plenty of other companies capable of matching the quality of Google's search results. Meanwhile, Google's lead in indexing the Web is also slipping. As of September, the company had indexed a total of 3.3 billion Web pages, but AlltheWeb is right behind with 3.2 billion, and Inktomi covers a good 3 billion. (According to Page, Google has passed the 4 billion--page mark since then.) And as fast as Google rolls out clever new features, like Google News news.google.com and Froogle, Google's shopping search engine, its competitors roll out clones--like Microsoft's MSN Newsbot uk.newsbot.msn.com) which launched in November, or Yahoo's Product Search.

In fact, Google's greatest asset right now may be not its technology but its brand--and Internet brands are notoriously volatile. (AOL's trajectory suffices as the cautionary tale to end all cautionary tales.) And just when Google is looking increasingly beatable, search is becoming increasingly profitable. You can't leave a tasty pie like that on the windowsill without attracting a few bears. The fairy tale is about to turn into a war story.

Microsoft may be Google's biggest problem. Plan A for Microsoft was probably to buy Google outright, using Bill Gates' massive war chest of $49 billion. According to many news reports, Microsoft held talks with Google this fall, but no deal was struck. Plan B, already well under way, is for Microsoft to build a better search engine. A prototype Microsoft webcrawler (the software that search engines use to index the Web) was spotted online as early as last April. Kirk Konigsbauer, general manager of the project, says MSN Search isn't ready for prime time quite yet. "We decided to take a deep dive into the search business about a year ago," he says. "We're learning a lot. This is a really hard computer-science problem. In fact, our engineers say this is the hardest computer-science problem they've ever had to face." But if there's one thing Microsoft is good at, it's entering profitable markets late and strong--just ask Netscape, Apple or IBM. If Microsoft integrates its new search engine into MSN, its Internet service provider, and Longhorn, its next-generation operating system (E.T.A. 2006), the game may be over.

Google's most enigmatic foe isn't a search engine at all. It's a bookstore. In October Amazon debuted a service it calls Search Inside the Book that's both simple in conception and staggering in its implications. Amazon scanned every page of 120,000 books into a database, and it now lets customers search the books' complete contents online. In one stroke, Amazon made a new and immensely valuable kind of information available on the Web. Zenodotus would be livid with envy. "I would compare it to the invention of the encyclopedia," says Joel Mokyr, professor of economics and history at Northwestern University. "If this is further developed, as I hope it will be, you may one day be able to sit in a village in Nepal and look up every book in the Library of Congress! Think about the implications of that: the knowledge of all times may one day be accessible from every spot on this planet. That's one of the most democratic things I can think of."

And Amazon founder Jeff Bezos isn't done yet. Behind a smothering veil of secrecy, he's setting up a new search company in Palo Alto, Calif., called A9.com "All I can tell you is that we're working on some interesting things that we simply cannot talk about at this point," says Bezos. The scuttlebutt is that A9 will be focused on product search, so it will compete less with Google than with Froogle--a relatively small slice of the search market but potentially the richest. Amazon--which is still glowing from its first profitable nonholiday quarter ever--has been working with a shadowy start-up called Groxis, a company that dabbles in curious, arcane techniques for graphically displaying search results.

Meanwhile, onetime search leader Yahoo is spending a fortune on mounting a comeback. Yahoo bought Inktomi, a top-flight algorithmic search engine, in March for $235 million. In October it swallowed Overture, which specializes in so-called paid search (we'll get to that later), for $1.63 billion--while Overture was itself in the middle of digesting two recent acquisitions, AltaVista and AlltheWeb. The plan, as near as anybody outside Yahoo can make out, is to stitch all those disparate organizations into one huge Frankenstein's monster of a search engine that will strike terror into the hearts of all who behold it. "I think search is clearly among our company's most important strategic initiatives," says Jeff Weiner, Yahoo's vice president of search. Indeed.

Yahoo currently partners with Google, but that deal may not last. "I expect them to dump Google," says Sullivan. "I'm surprised they've stayed with them for so long. Inktomi is perfectly capable of doing what they're using Google to do." (Yahoo, which once sank $10 million into Google, doesn't like to talk about its rival to the press; that's a tangled web.) All Weiner will say is this: "We are certainly going to be leveraging all of our in-house search assets to a greater extent."

But Overture, not Inktomi, is Yahoo's prize asset, because Overture specializes in paid search, and that is precisely what makes search such a hot field right now. When you look for something on the Web--say, "winter jackets"--you generally get two sets of results. One set consists of websites that are relevant to winter jackets, to the best of the search engine's ability to determine that. The other set consists of websites that bid on the top placement in a separate, specially marked group of paid search results. Yahoo calls these sponsor results. Google calls them sponsored links. For both companies, they are a fat and loudly mooing cash cow.

Paid search combines two things that advertisers desperately want: targeted ads (ads that are shown only to consumers who have demonstrated interest in the product) and performance-based ads (for which the advertiser has to pony up only when a user clicks on its links). For retailers weary of the futility of those universally reviled banner ads, paid search is revitalizing the whole idea of online advertising. "Nothing is more valuable than the user at the moment of desire," says Gartner analyst Whit Andrews. "When a user searches, that user has a demand. At that moment in time, they want." Overture now has more than 100,000 clients, and Yahoo's third-quarter earnings statement reports that it has more than doubled its year-over-year revenue in paid search as a result of better pricing and higher volume. Youssef Squali, an Internet and new-media analyst at First Albany, estimates that Yahoo's revenues in 2004 will reach $2.8 billion, and he expects paid search to account for half that figure.

None of this is to say that it's time to count Google out. For starters, Google is the leader in paid search, with about 150,000 paying customers of its own, and it has consistently led the league in innovation. But there are bigger questions afoot than who's going to win the search race--namely, What will they win? And perhaps even more important, What will consumers lose? The Web is rapidly supplanting other media as the primary means by which people get information about the world--not just sports scores but news, car prices, history, famous quotations and potential dates. This is information that matters. Google--or any search engine--isn't just another website; it's the lens through which we see that information, and it affects what we see and don't see. At the risk of waxing Orwellian, how we search affects what we find and by extension how we learn and what we know.

That's a pretty important job. It may be too important to let the market decide who gets it and how they perform it. For example, what if a search engine took money not only to list a certain website but to suppress the listings of its competitors? Some search engines are owned by large, diversified corporations. Should they be allowed to push the websites of their own businesses to the top of the stack? Could a search engine suppress the website of a political party with which its CEO disagreed? Or a particular version of the Bible or of the Battle of Midway? These are important questions, and regulators have barely started asking them.

The Federal Trade Commission (FTC) has taken some baby steps toward keeping search engines honest. In June 2002, after receiving a complaint from the consumer advocate Commercial Alert, the FTC put out a letter urging search-engine companies to make "clear and conspicuous disclosures" about which search results are paid and which aren't. But the agency has no plans to take formal action. Letting the market solve these problems by itself is the American way. We like to assume that the most objective, least biased search engine will naturally win the search wars. (A typical European approach, by contrast, would be to nationalize Google at once, before it's too late.) This means that, for now, we're relying on the integrity of people like Larry Page. But when the competition gets stiff and Page has to answer to his shareholders, integrity may be a luxury Google can no longer afford.

--With reporting by Maryanne Murray Buechner and Jyoti Thottam/New York, Stefanie Friedhoff/Ann Arbor, Eric Roston/Washington and Chris Taylor/San Francisco

With reporting by Maryanne Murray Buechner and Jyoti Thottam/New York, Stefanie Friedhoff/Ann Arbor, Eric Roston/Washington and Chris Taylor/San Francisco