Bill Roth, Ulitzer Editor-at-Large

Bill Roth

Subscribe to Bill Roth: eMailAlertsEmail Alerts
Get Bill Roth via: homepageHomepage mobileMobile rssRSS facebookFacebook twitterTwitter linkedinLinkedIn

Related Topics: SEO Journal, Facebook on Ulitzer, Microsoft Developer, CIO/CTO Update, Telecom Innovation

Blog Feed Post

Is Search as good as it used to be?

In a recent article in the Huffington Post Stefan Weitz, director of Microsoft’s search engine Bing is quoted as criticising ‘conventional search engines’ such as Google, for failing users. He says they haven’t fundamentally changed their model over the years and that the wisdom of crowds philosophy of providing results based on back links is no longer working as effectively as it was back in the day. He questions (legitimately in my opinion) Google’s latest attempt to embrace social tagging through its +1 feature where it allows people to ‘like’ web pages. Now when you search, Google tries to utilise this information by incorporating the ‘likes’ from your ‘social circle’ and using this knowledge to improve the ranking of relevant pages -  very  2011. The premise is that, if people you like, liked it, you should like it also. I’m not so sure – it’s the wisdom of a smaller crowd – why should that be better?

What is interesting though is that despite the criticisms of Google by Weitz, Bing is deploying similar technology utilising information gleaned from Facebook. Here they have access to all our social connections on line & given a query, they want to utilise this knowledge to try & provide more targeted results based on our friends opinions. I don’t know about you but much as I like my friends, I do not have the same taste as them when it comes to eating out, what music to buy, booking my holiday and many other types of things we all search the web for. So how can their opinion be reliably used to help me to find what I want? I believe relying on this information is fundamentally flawed. The only person who is a good judge of the types of things I like is me – and even then I am notoriously fickle and inconsistent over time in my decisions. I don’t expect this is going to be the panacea it has been built up to be.

Weitz also lambasts Google for conditioning us to expect to fail when searching. According to Weitz, conventional search (Google) fails 75% of the time and as such we expect it to fail. He points out that only 25% of the time do we find what we need on page 1 of a search & are constantly refining our query to improve the results – while this is true I am not sure it is entirely fair to blame Google alone for this. After all I am sure they have done their best! I don’t mean to stand up for Google here as they are more than capable to do that for themselves, but It’s easy to criticise from afar. The questions I would throw back at Weitz is what have Microsoft contributed to solving this problem over the years and is Bing any better? The usage stats still show that Google (65% Market Share) is by far the preferred engine. So from this alone you could infer that Bing (14%), still has some catching up to do.

Another criticism Weitz  levees at Google is he claims it has taught us bad habits in that we have learned to communicate in queries that are very short – ‘pidgin English’ as he calls it. Weitz claims Google has ‘conditioned us’ to think this way which is why we get vague results. He argues we should be able to communicate more naturally with the search tool – and this is somewhere where Bing seems to be trying to differentiate itself in an effort to carve out a niche in order to better compete. The theory is that using a more natural language query compared to a ‘pidgin English’ query enables you to extract out a better context and therefore better understand the user’s intent leading to more relevant results. And this is the crux of the problem of why search engines are failing – they don’t understand the context of queries nor the user’s intent – as such they provide a lot of noise & unfocused results back to users. This is why it becomes a lottery to get the right hit on page 1 of the results. While users providing more detailed information in longer more exact natural language queries would be helpful to address this issue, it’s not something that is likely to happen – sorry Microsoft.

Why do I say this?

because we are lazy. We expect maximum return for minimum effort. We love our TV remotes, we expect our car doors to open when we get within a few feet to save us the bother of clicking the button on our keys, we love the convenience of drive through restaurants to save us the time it takes to get out & order, we expect content we want to be pushed to our phones to save us the time it takes to turn on our computers… the list goes on – We want instantaneous results. It’s not Google’s fault we use short queries when searching, they haven’t conditioned us to be like this, we are like this naturally. No one wants to take the time to enter in a 10 word query when they can enter 3 words. The challenge for search engines is to be able to take this limited information and work wonders with it. To do more with less.

There is little doubt solving the context problem, for both Internet and Enterprise search, has been the central piece of the jigsaw in taking search to the next level for some time – yet a solution has remained elusive. We can’t second guess (using previous history or social networks) what the user’s context is exactly (in enterprises we don’t have this social data anyway), as we are fickle and our context changes not just daily but sometimes hourly and social information therefore is a guess at best. We can’t expect users to be more specific about what they are looking for either, they won’t take the time. The solution for me is to understand the meaning of the content being indexed in the first place. Then when a user queries, present them with the most relevant meanings their query can have within the content being searched – to empower users to browse and navigate and understand the information they are searching. As was outlined in a recent posting on KMWorld by Steven Arnold, at Sophia we have developed a tool which utilises the power of semiotics to do this within an enterprise setting. It works really well and is much more reliable than ‘guessing’ using social networks or expecting users to be more informative when submitting their queries.

Read the original blog entry...

More Stories By Bill Roth

Bill Roth is a Silicon Valley veteran with over 20 years in the industry. He has played numerous product marketing, product management and engineering roles at companies like BEA, Sun, Morgan Stanley, and EBay Enterprise. He was recently named one of the World's 30 Most Influential Cloud Bloggers.