Thanksgiving - with Spam

[Chipmunk]

And...at least for now the Spamstorm is over. At my request, DougO has temporarily turned off new user registration until he has a chance to upgrade the forums and take a look at Chipmunk's recommendations.

Whew...I was afraid I'd ban a real person or delete a real thread at times.

Shiny!
Yeah, in the wee hours of this morning, I checked "Today's Posts" and what Sinclair said near the beginning of the Battle of the Line popped into my brain.

Ma'am, you are of course correct, a mod normally doesn't have access to the registration settings... I had a problem with my brain being missing.

Though the more that I think about it, the more it seems that there should be a simple "Panic Button" setting that any mod should be able to hit, when the waves of the enemy are overwhelming (sort of like the EMP in The Matrix).


And to join in praise of Jan's Slaying skillz, from the story of the (Whedonverse's) First Slayer:

"First there is the Earth. Then, there came the Demons. After Demons, there came men. Men found a girl. And the men took the girl to slay demons. They chained her to the Earth. Filled her with Dark. You cannot be shown. You cannot just watch, but you must see. See for yourself, but only if you're willing to make the exchange. This is the only way. There is no..."

[phazedout]

'tis the season to be spammy
tra la la la m la la la la
and the references are quite hammy
tra lalalalala lalalala
take some forums add some porn links
tralalalal la la la la
people won't see what they do thinks
tralalalalla la la la laaaaaaaaaaa
Troll the ancient Yule tide carol,
Fa la la la la, la la la la.

[Jan]

Cute! But just to add insult to injury, there aren't even many porn links left. At least, not that I noticed. I don't look at 'em, I just ban 'em!

Jan

[OmahaStar]

I found it both ironic and funny that the spammers replied to this very thread.

[Milkman]

They just don't give up! FYI im having the same problem... i ended up disabling new users for now until things calm down and i can get a new anti-spam thing in place.

[Chipmunk]

It's been a while since I've done any significant forum anti-abuse experimentation, so this inspired me to play with a small members-only support forum for one of my projects. We've had registration turned off for years (we manually reg new Team members), but I'd seen significant spammer prowling in the web access logs.

On Nov-28, I did the fiddling described previously (well, a variation, the forum software we're using is very limited), including a new "tarpit" forum just for spammers (invisible to Team members), and turned on instant self registration...
Wow! Explosion of activity!
It's been a lot of fun, and revealed some interesting spammer tactic changes, from the last time I worked on this.

While I'm still collecting data, I do have enough to share this advice:



1. Put simple "special" instructions in the Registration Agreement.
I nuked most of the busy / lawyer talk in the default one, and added a section at the very top, which explicitly said that they would have to enter the name of both their favorite science fiction TV series and character, into a specific Profile field.

So far, of 657 spammers who have successfully registered, only 47 put anything in that field.
Of those, 10 put a spam URL, and 37 entered one short token with all random characters.

Since most forums have some sort of "about me" field, this is a very easy (no-programming-required) way to help our beloved moderators.

As part of this experiment, I set up a moderator account, and after about a month, went in and tried to nuke individual spam posts.
Dang, that's hard!
Fortunately, Jan had mentioned nuking entire accounts, so I switched to that. Much easier! Thanks Jan, for sharing your tips!

That's where that "required" field is particularly helpful.
It's a quick way to do one final sanity check before going all Thor.

It would be easy to write some custom software that checks all new registrations, and auto upgrades everyone with a "reasonable" value in a specified profile field.
I've added that to my list of Coding projects (won't happen any time soon, but will happen).


2. Use a "strong" CAPTCHA.
A few years ago spammers "broke" CAPTCHAs, however, it does take significant effort, which raises their costs and slows them down.

It's a little early for numbers (I just started a new set of tests on a different server), but their range surprised me.
Some spammers took dozens of tries before they could register.
Others made hundreds of attempts, and never succeeded.

DougO has set up a "strong" CAPTCHA here... if you think things have been bad with that, they'd be much worse with a "weak" or no CAPTCHA.


3. Consider blocking abusive IP ranges and/or major nations!
More than half of the spammers who hit my experiment, were using IPs listed as issued to companies in China.
Of the most pernicious, bandwidth-consuming "sessions", more than three quarters were from China.

Most web hosting software (e.g. Apache) makes it easy to block by IP address, and there are several sites which provide samples of how to block all of China.
I generated my own file, because I'm admin of a major "IP to Nation" database.

Instead of blocking all of China, it's extremely effective to block just the most pernicious IPs.
Here's how (requires the use of a decent log analysis application):

• load one week's worth of your "raw web access logs" into a good logs analysis app
  
• filter on sessions which visited your forum, and which attempted to register
  
• if your analysis app allows it, dump a list of the IPs which had the highest bandwidth or number of hits
  
• convert those IPs to block/deny rules

You'll probably find many of those IPs are close neighbors, so it's a good idea to merge them into as few blocks as practical.

I was able to block about three quarters of my spammer traffic by just blocking a couple dozen IP ranges.

That takes more effort, but makes it easier for legitimate Chinese registrants to join.

For forums who expect Chinese visitors, it's a good idea to redirect banned IPs to an info page, with an alternate means to register.

I've found the most effective way to handle putting an email address on the web, is to use a dedicated email address and embed a required Subject line "token", then add a server rule which discards/quarantines anything that does not have that particular "token" in the Subject.
Here's an example of a "mailto" link with a real email address and an embedded Subject:

<a href="mailto:[email protected]?Subject= Delenn_and_G'Kar_rock!">click here to contact us, but do NOT change the Subject line</a>

Check your webhosting help wiki/forum for info on how to block/pass email based on Subject lines (it's trivially easy, and if your webhost does not allow it, they suck).

I've been using embedded Subject tokens for about a decade, and have had zero spam get thru that defense.

Disclaimer: No offense against the Chinese! Historically, it's been the #1 point of origin of net abuse, but some of that has been sponsored by 'Merican scum. The USA remains the world's #1 reason for spam, and is the #1 source of "snowshoe" spam. I aggressively block huge expanses of 'Merican IP space.


Anyone interested in helping with some "Usability" testing?
Over the next couple of months, I'm planning to install each of the major free forum packages, and evaluate each for security and ease of anti-spam tweaking (e.g. how easy/possible is it to implement the two strategies I described earlier in this thread).

I have a "feed" of live spammers, so it's "easy" to test my countermeasures against the bad guys.

What I need is some humans, preferably of diverse age, tech saviness, and browsers / operating systems.

I figure this is a great place to start looking for some.
All you (and your Muggle friend(s)) would have to do is try to register, make one test post, and give me feedback (preferably in the test forum) as to how easy or confusing registration was, and what you think of the forum software compared to what you're used to. I'll probably set that up as a series of polls.

I've found there are very few objective and technically oriented reviews of most web software and services. A big part of that is that reviewers don't come up with a specific test plan based on real world experience with the problem domain. I've needed to set up a new forum for a work project, for quite some time, so I have a detailed wish list, and a detailed anti-spam / pro-security wish/need list, plus six years of mucking about as admin of two small forums (fora?).

I'll write up and publicly post the results on my anti-spam research/info site.
It would be really cool if I could end it with "and thanks to the great Babylon 5 (greatest TV SF series of all time) fan community for their testing help".

[Jan]

Thanks, Chipmunk! I've passed this all along to DougO for when he has a chance to address this. I appreciate the thought you've put into this. Not able to help out with your testing at this point but I'll keep it in mind.

Jan

[Marsden]

There's me, Omaha and 5 spammers on! D'oh!


I wish I could do something but power corrupts and any given to me would turn me into a tyrant