Note: This blog post is from 2009. Some content, links and indeed comments from others may be outdated--though not necessarily. Corrections are welcome, in the comments. I may revise the content if necessary.Want another tool to help battle blog comment spam? Here's an approach I use that may benefit others. I look for certain bad URLs being referenced in the comment, and if they exist I block the comment. Sure, there are other solutions. I've wondered for a while about sharing this code publicly like this, but I get enough people who've asked for it that I figure I may as well.
Update: Ray has clarified (in a comment) that BlogCFC does already have this functionality, in the "trackback spamlist" feature (on the Settings page of the BlogCFC Admin). I thought that had only to do with track backs, not comments. If you're using BlogCFC, you should use that feature to achieve what I describe here. But some of the thoughts and techniques may still interest some.
What's the problem, for bloggers and commenters, and why Captcha isn't enough
We all know that comment spam is the bane of our existence. How many times have we seen comments referring to wowgold or battery crap or some foreign characters we can't even read. Sure, captchas and other tools are intended to try to stop it. But some still gets by those. These are often real people typing this in, so they get by tools that try to block automated entries. (I appreciate that some tools do still more. Check out the link above to learn more of them.) For those still interested, press on.
These spammers are clever: they'll repeat words from earlier in the blog entry, or from some other commenter, or even from some entirely different blog entry, hoping the blog owner won't notice that a shifty URL has been planted in the text (or the URL field of the comment form), all trying to get a little Google pagerank love for the URL they're pimping.
So I wanted to come up with my own solution that simply detected and blocked any comments with references to those bad urls. What I did works for BlogCFC (admittedly an old edition), but the concept can be of value to you regardless of the blogging software you may use.
And to be clear, this bane of blog comment spam is not just an annoyance for bloggers themselves, but also any who are blog commenters. Most blog software is setup to send us commenters a copy of any other comment someone posts. Even if a blogger is diligent about catching and deleting such comments (so they get no pagerank love from being posted), some of the damage is done in that the fellow commenters on that entry did get the email.
Again, I wanted a solution that let me detect and prevent submissions of spammy URL references. There's no blacklist for keywords in the version of BlogCFC I have.
Even then, I realize some don't like doing blacklists of keywords anyway, since you can get false positives. Then there's the challenge that if you look for some words, the spammers just change them. But for the problem above, their goal is to get their URL listed.
So I was interested in looking only for URLs, not just any "words". Further, I want to check in both the content field and the URL field of the comment. (And if it meant I blocked someone who was merely mentioning one of these spammy URLs, in a helpful way, I'm willing to risk that false positive.)
So the way I do it is that I created a file to track the bad urls. When I get a comment that's got content that's spam, I put any domains it refers to into that file (and then delete the comment, of course).
Then before accepting any new comment, my code reads that file (yes, on each comment submission. I could optimize things, of course, reading the file for a cached period. I could also offer an interface to more easily add URLs to the badurl list file. I just haven't gotten to that. For now, I just edit it, maybe a few times a month after having gotten most of the common crap URLs under control.)
About the blacklisted urls file
Rather than post the badurls.txt file here, you can leave me a comment (which will ask for your email address which is not shared and your URL. Tell me the URL of your CF blog), and I'll send it to you directly. Don't want to give away intel to the spammers, plus by me sending it along you'll get the latest.
Another thing I could do is create a service where the badurls file is kept and accessed/updated centrally. Again, just haven't gone to that yet. Nor even creating a Riaforge project for this. I'll wait to see what people think.
The badurls file is really just one big long list (comma-separated) of bad domains. Here's just a sampel of the first few entries (it's all just on one line):
Note that I don't bother using the full url, and I even leave off the www. part, since some spammers use sobdomains. Of course, I wouldn't add to the list a domain that looked like it could be legit. But if it looks suspicious, it's black listed.
What do the spammers see?
I don't tell the spammers that I'm rejecting them because of the spammy URL. I just report "Invalid request" as an error. I also happen to email myself when people attempt to send comments (in case they have problems with the captcha or for some other reason their comment doesn't make it), so I have fun watching how the spammers flail about trying again and again to get their crap in. :-)
I figure if it was a false positive and someone REALLY sincerely felt that their comment should be let in, despite their referring to one of these urls and getting rejected, they could just contact me directly (as I offer a contact link on my blog, or they may think to enter a plain comment. Again, these are rare instances, I think.) The benefit for cutting down on spam comments has far outweighed the risk.
Update: With regard to the BlogCFC "trackback spamlist" feature, I'll note that it doesn't offer any feedback at all if a comment has a blacklisted keyword/url. It just closes the form as if it took, but the comment is not posted.
What do I do with the badurls file? Show me some code.
I drop the badurls.txt file into the blog root directory (typically blog/client in blogcfc), in the same directory with the addcomment.cfm template. In that file, I make just the following 3 edits to that addcomment.cfm template.
First, I add the following that reads the file in:
<!--- ought to cache this and refresh when file changes --->
<cffile action="READ" file="#expandpath("badurls.txt")#" variable="badurllist">
<!--- the next line is just to test if the data in the file is in fact a valid CF list. if not, email me --->
<cfset listerrcnt = listlen(badurllist)>
<cfmail to="whoever" from="whoever" subject="failure during blog addcomment, badurl list processing"><cfdump var="#cfcatch#"></cfmail>
And in the addcomment.cfm test I place some more code for adding a comment which should go inside this line:
and after the first IF test for:
I added this:
<cfset errorStr = errorStr & "- " & "Invalid request" & "<br>">
Sure, I could have done that in CFSCRIPT. Same with the next chunk coming up. Feel free to change it if that suits you. :-)
Needed (and created) a new UDF, FindList
You'll notice this calls a udf, findlist, which does something that surprisingly no built-in function does: searching one string for any of several items in a list. (For an explanation of how it differs from listfind and listcontains, see the version posted at CFLib. That udf is a little more complicated, as I expanded it based on some feedback from others.)
<!--- FindList, from Charlie Arehart--->
<cfargument name="valuelist" required="Yes" type="string">
<cfargument name="stringtocompare" required="Yes" type="string">
<cfset var found=0>
<cfloop list="#arguments.valuelist#" index="x">
Hope all that may help someone. Feel free to comment.
For more like this: