In the lightning-paced internet age, predicting how the world will look five or ten years into the future is a tricky business. The task is even harder if you are an internet company chasing a seemingly invincible competitor, as is the case with Yahoo, which is steadily losing market share to Google in search traffic and search-related advertising.
That鈥檚 not the way Yahoo sees it. The company points out that it is still the US鈥檚 most popular web portal, with some 500 million visitors per month. It is also the US鈥檚 top website for email and online news. This has enabled it to amass unprecedented amounts of data about the habits and preferences of its users, information that if used in the right way could transform its business. With this goal in mind, last year the company launched Yahoo Research, with three labs in California and others in New York, Europe and South America.
Yahoo Research says it is developing 鈥渢he new sciences of the internet鈥�: research into online marketing and interactivity that combines artificial intelligence and economics with traditional computer sciences. Heading the effort is senior vice-president of research Usama Fayyad, a pioneer of data mining. Robert Buderi asked him about his plans to get an edge on the competition 鈥� and what it all means for the web鈥檚 future.
Advertisement
You have accumulated a huge amount of data about your users. How will you use it?
We generate something like 15 terabytes every day from 500 million users around the world. We鈥檙e already using that up to a point. Why do you think the business of sponsoring ads on web searches in total has gone from zero to between $10 billion and $14 billion in almost no time? One reason is that when you type in what you鈥檙e looking for, you are actively telling me what you are interested in. So for a split second I have an insight into your intent and I can match ads to you better.
What the market hasn鈥檛 caught up with is that search is only the simplest kind of interactive web application. You type keywords into a box and you get back links and click. Well, think about something a bit more complex like planning a trip. You pick where you鈥檙e going, the ticket you want, the hotel, the car, the whole thing. In that context, I know more about you than if you typed a thousand keywords into a search box.
Similarly, when you join an online community you are saying a lot about yourself. Interactive applications tell you so much more about people鈥檚 intent. That鈥檚 an example of how data can transform a business. We just haven鈥檛 figured out how to make money from it yet.
鈥淲hen you join an online community you say a lot about yourself鈥�
But why the need for a separate research division?
Yahoo鈥檚 CEO Terry Semel had several discussions where he would say, 鈥淲e have plenty of people worried about the next quarter. Who is worrying about two, three, five years out and what鈥檚 going to happen around the next corner?鈥� So I started asking basic questions about the future, like what makes an online community prosper? What makes for trust online? What economic incentives would encourage people to participate in a community and keep it healthy, in a way that鈥檚 profitable?
The scary thing is that when you scratch the surface, you find you need all these new sciences that don鈥檛 exist yet, combining machine learning, artificial intelligence and microeconomics with traditional computer and data sciences. Nobody is teaching anything like this in academia. So I came back and said that our mission should be to create a research centre that will invent the future science of the internet that underlies all these phenomena.
How are you going about doing that?
The two tenets I stick to are: don鈥檛 be too big and stay extremely focused on a few areas where you will dominate and have the best people in the world. Outside those areas you can admit you are weak. So I want to pick four, maybe five areas, and that鈥檚 it.
We have hired some big names. The first was Prabhakar Raghavan, who now runs Yahoo Research on a daily basis. He used to be chief technology officer of Verity, an enterprise search firm. Among others, we have also hired Andrei Broder, one of the inventors at Alta Vista of search technology as we know it today, and Ron Brachman, one of the founders of AI and knowledge representation. He was at AT&T Bell Labs and is now heading our New York lab. We also hired Ricardo Baeza-Yates, who wrote the bible on information retrieval. He is heading up our labs in Barcelona, Spain, and Santiago, Chile.
With these people we definitely are dominating so far, in terms of numbers of papers, the best papers, interaction with academia, all that good stuff.
Apart from web searching, what other areas are you targeting?
I can tell you about the next two areas we鈥檙e looking at. The second, which we鈥檙e onto now, is microeconomics: the economics of the web that translates to a massive scale. For example, if people in southern California are browsing for a certain type of car in a certain colour with certain optional features, can we predict from that what sales are going to be one month from now, six months from now?
Imagine doing that for every micro market within the US or internationally. That would give you some amazing tools for predicting what is happening to the economy, and what the new trends are. We are going to be investigating this intensely. The third area that we will be targeting is online communities. From a scientific perspective, we want to uncover what online communities are really about.
Where is this all leading?
Interaction is the key 鈥� not just with your PC, but also with things like your cellphone, television and car. Interaction with these kinds of devices will become people鈥檚 preferred medium for finding the information they need, when they need it. As you are driving your car, there are a lot of devices that give you information, like your GPS navigation updating the map. Besides that, with more knowledge about where you are, what you like and what you are doing at that moment, devices can be doing things like looking up restaurants or nearby attractions. A cellphone can start behaving as your personal assistant and not just a device for calling.
The more we learn about user behaviour and interaction online, the better we will be able to deliver relevant information when people need it. Our job is to ensure that Yahoo is the right place for people to go for all the applications that are important to them, be it finding the news, going to movies, organising a party or buying things.
That sounds like a recipe for invading my privacy
The heart of the whole business is having an engaged audience that trusts you and keeps coming back. Once you lose the audience everything else goes away. There鈥檚 a very delicate balance. I have this notion of private and public settings. So if you go to a department store, it鈥檚 not considered intrusive if a salesperson tries to sell you something. If that same salesperson follows you as you walk out of the mall to the parking lot 鈥� that鈥檚 unacceptable. As chief data officer I am responsible for ensuring that Yahoo uses its data responsibly.
Profile
Usama Fayyad was born in Egypt but earned his PhD in engineering at the University of Michigan in the US. From 1989 to 1996 he worked at NASA鈥檚 Jet Propulsion Laboratory in Pasadena, analysing astronomy databases. He then joined Microsoft Research, leading its data-mining and exploration group until 2000. He subsequently created the DMX Group, a data-mining consulting firm that was purchased by Yahoo. In 2004 he was named Yahoo鈥檚 first chief data officer.
