In a recent interview with Google’s Peter Norvig, director of research; he admitted that Google employs humans to manually review and report on search results. The human reviewers are contractors, who are trained to spot spam techniques. They are tasked to randomly select specific queries and record their findings.

The argument with this policy is how this data is then used. Giving a penalty to a site, or conversely boosting another sites rankings manually assumes that the site in question doesn’t change its practice or relevancy.

There is an element that needs to be considered here. Humans are more likely to reward fresh news content, because of a perceived "finger on the pulse mentality". Which means that brochure and catalogue sites that typically change very little from one incarnation to the next, are likely to not be rewarded even though they are more likely to fulfil the searchers expectations.

An example of this would be back in the days of the Florida Update, when a search for a "Laptop Battery" would result in many documents explaining the composition, manufacturing process and performance, but no listings on where to purchase one. Not a good user experience for the majority of searchers.

This said if Google’s intention is to use that data to improve its filtration and algorithm then it has to be applauded.

Related Stories:

All about user experience

Comments:
7 Jan 08 Chris Estes
Great! Now I wonder how much weight will be put on the results of the humans? In a typically all utilitarian method, this could have drastic effects on results. I really wonder just how much and how human interaction is going to have with the results.