Spam & Improving Search Quality
SearchEngineWatch Forums moderator Barry Schwartz (who goes by the nickname of rustybrick) has/is providing some coverage of the current Search Engine Strategies conference in San Jose. This particular session included former search engine personnel Doug Cutting (Excite senior engineer), Steve Kirsch (InfoSeek founder), and Louis Monier (AltaVista).
While the session covered some fairly fascinating early search engine history — helpful in getting a viewpoint of where search engines started and including a few clues as to what you might have seen then and might be seeing now — one really telling statement surfaced in the second post at the SEW thread (emphasized in bold below):
Q: Mike Grehan asks … when did you realize spam is an issue?
A: Doug [Cutting from Excite] said spam was thought of as search quality, improve the quality. You can also build spam filters as well.
This little gem highlights the real cause of the "spam" problem. While spam and spammers are complained about by various search engines, what is rarely suggested is precisely who is responsible for spam actually appearing in search results. The truth is that few people actually control search engine results, and those would be the search engines.
In the real (read: "non-public relations") world, people make all kinds of pages; some are interesting/valuable to some, others aren't. It may be that the bulk of what one person or another might consider "spam" is not made by spammers. Or maybe it is; I really don't know.
The bottom line, as highlighted above: want spamless search results? Create an algorithm that weeds out spam.
I know. Easier said than done, as the rest of that second post reveals. It is unfortunate that a company that wishes to provide search results to the world — and becomes successful at it — is sure to be bombarded with pages of such quality as to make its search results worth less, if not altogether worthless. I do remember someone years ago at the SearchEngineForums claiming to have 1,000,000 doorway pages (pages extraneous to a website whose function is simply to dominate search results for various terms). That doesn't seem fair to me either.
Louis said spam in AV was a nightmare. … One guy wrote a script to pound spam into the index and break the system. AV tried everything and they never had a good answer.
I realize that the argument goes that "search engines are using our pages for free" (and thus it's okay to do anything you want). This seems just a bit disingenuous, since it takes little effort to block search engines from websites — which anyone who is clever enough to spam could do easily.
So, search engines are using our pages for free, and we're getting visitor traffic for free. It is a symbiotic relationship, if uneasily so. But, as with any relationship, the most beneficial arrangement would be one in which *both* parties benefit.
Have your say ...
First-time comments will be held for moderation (but comments are appreciated). Otherwise, just be courteous. If your name is a bunch of keywords, your comment will be deleted. Don't post links unless highly pertinent. Posters must be 16 or older.