Yelp has a system. It doesn't seem to be perfect; I've seen what looks like both false-negative and false-positive classification of fake reviews. But it seems to work decently well.
One thing I do like about their system is that you can click a link to read the reviews they consider "not recommended."
Let me just say that Yelp's classification system is complete crap. Until a user starts actively using Yelp, their reviews get filtered out, which excludes most people who only post a review because they loved a certain business. Sometimes, five star reviews will vanish after months of being up, all because of a change in the algorithm. And the Yelp business team is extremely unresponsive when there is a clearly illegitimate review that needs to be removed, or really when there is any concern whatsoever.
This may not be the best place to bash Yelp, but if you're reading this, please don't pay Yelp for their exorbitantly-priced business service. It is absolute garbage.
Also it discourages any new people from participating. I signed up, posted a few reviews of businesses I'd been going to for years, then noticed they got filtered out and stopped participating.
And frankly, I don't have seem to have a lot of tastes in common with the hardcore Yelp users.
Heh, I actually agree with you (and emodendroket's) observation that Yelp buries reviews by new posters. (That's discouraged me from posting, too.)
Maybe I was too generous by saying their system works "decently well"; though I don't think I'd say "complete crap"--unless my business had been burned by fake bad reviews, which it sounds like happened to you.
Perhaps this illustrates how subtle/difficult this problem is.
The good news is that you don't need to read all the books, because many bogus reviews tend to be written by the same "customers". Simply seeing a correlation of bad books getting good reviews, would result in those "customers" losing their ranking power from their reviews.
What's bad (really bad) here, is that not only are the books that are being ranked highly absolute crap, but they are crap in different categories. I.E. A "book" on swift, that was apparently "written", in a few hours, shoved onto Amazon, falsely ranked up - now ranks ahead of a 700 page opus on Perl, in the Perl category.
That's a total fail on Amazon's part, and really, really disappointing.
I don't find it at all surprising, however. Amazon can't even manage a half-decent search across the products they carry. I have this feeling that somewhere in Amazon, it has been decreed that the search and the rankings and everything else must be filtered through the same system that generates recommendations. You search for "4TB hard drive" and the 8th search result is a 3TB drive. Their site skips over much more relevant items apparently because the items are more strongly 'related' in terms of purchase history or browsing history or something. I expect exactly that kind of system is what is being gamed in these circumstances.
When it comes to reviews, the correct answer is that every user should have their tastes profiled and compared against the tastes of other users, with only users whose tastes in other areas counting strongly. I could easily imagine that this sort of thing might be difficult for Amazon to implement, though, depending on their infrastructure.
In this case there were a few things that made it glaringly obvious (to a human at least).
The bogus reviews had one or more of these aspects:
- bad reviews from reviewers with no other reviews
- mentions of the same competing book in my bad reviews
- 5 star ratings of that mentioned competing book (and no other reviews beyond those)
- 1-star reviews of another random book. This one was interesting, there was another book (Service Oriented Architecture for Dummies) that is entirely unrelated. But 2 of the reviewers also gave shitty reviews to that book as well as mine. There's no logical reason that the same people badmouthing my book would have also read, reviewed, and badmouthed that other book (totally different topic) within a few days. So they were clearly hired for a number of books at once.
So yeah, you'd think some of those patterns would be easy to automatically detect. Although I assume it's constantly a game of cat and mouse.
Probably apply ML techniques. Similar to how spam is classified - build a classifier that determines the likelihood of a review being bogus, then flag it for human review if it exceeds a certain threshold.
They probably already have this, to be honest. Some of the fake reviews used keywords from the book title, probably in a learned attempt to game the classification.
Interestingly, Amazon can mine a great deal of the reviewers' histories to determine fake reviews. For example, a longtime customer of Amazon would get a higher "trustworthy" score for reviews than one who signed up 2 weeks ago.
Expensive or not, Amazon has an obligation to make sure their review system has high integrity and is not "gamed" or no one will trust Amazon reviews anymore.
Not to mention the fact that pushing $3 spambooks that probably have high refund rates ahead of $30 expert guides in the rankings could and should lose Amazon money.
Reviews that all mention the same book, or even have little or no content, and have the same rating are suspicious and easily detectable by a trivial scan and compare against existing reviews. A harder problem is when the bogus review content becomes more "clever" (e.g. random rating differences, "entropy" in the content e.g. by misspellings, etc.).
