Spammers and Messaging Vendors in Constant Battle of One-Upmanship

Like comic book super villains, spam kingpins always seem to find new ways to thwart the technology heroes that fight against junk mail. Just as it seems that they’ve finally been vanquished, they manage to elude the traps laid by anti-spam technology vendors in order to flood the inboxes of innocent users.

Last year was a case in point. Throughout much of 2006 there were some in the IT industry that were ready to proclaim victory over the spam problem, only to find a new wave of spam cropping up late in the year that was even tougher and more filter-proof than before.

While the resurgence surprised some, many in the anti-spam community were hardly shocked by the up tick that started at the end of the year. “Spam kind of reemerged with a vengeance last year, but that’s not new,” says Willy Leichter, director of product marketing for Tumbleweed. “Or, it’s not new to us. There’ve been these waves of spam countermeasures that will be effective for a few months, then the spammers adapt. They’re very responsive to whether they’re being blocked and whether something’s getting through. And they’re very much monitoring what the defensive techniques are. So, it’s this ongoing spy-vs-spy game where these cycles have gone on.”

Countering Image Spam Influx

Much of this recent flare-up of spam can be attributed to the latest weapon in the spammer’s arsenal: image spam. Spammers got wise to the last generation of text filters and began pitching their wares by embedding words into images.

“It’s shaken the industry up. A lot of the older filters that are signature base just got blown by,” Leichter says. Security messaging companies such as Tumbleweed fired a shot over the spammers’ ramparts by indexing frequently used spam images and blocking messages embedded with these files. But the spammers have been responding with even more sophisticated methods to push image spam through.

“We’re rapidly getting into the second or third round of image spam innovation,” Leichter says. “You almost have to have a grudging respect these guys. What they’ve very cleverly done is started to randomize them so that every one is different. Like snowflakes.”

For example, some spammers have created programs that will make slight changes to a source image to create thousands of different images out of a single file. “They can change the colors, they can put some random line and patterns and things in the background to make it more difficult for the automated character recognition to try to analyze it,” says Paul Wood, senior analyst for MessageLabs. “Sometimes they have wavy lines or text so that you can read it as a person, but a computer will find it really hard to read.”

These techniques have vendors scrambling around the clock to find ways to win this latest round of one-upmanship. Many have had to gain the right people and resources to better understand image technology, a niche that wasn’t top-of-mind for most messaging vendors until very recently.

“What we had to do to defend against it is to get into image processing technology – kind of analogous to what we do with text,” Leichter says. “But you’ve actually got to dissect the image and tell if you’ve seen a similar one before. That involves very sophisticated state-of-the-art mathematics to determine if this image is new or not. Anything short of that is just guessing around the fringes.”

Looking outside the envelope

However, simply playing the image analysis game may only do so much. Spammers are already reacting to the processing of spam images in a number of ways that will likely flummox filters yet again.

“What we’re starting to see now is, it’s still image spam, but it’s slightly different in that the image isn’t attached to the message itself, it’s hosted on the Internet on free file sharing sites like FreeShare, ImageShack and a number of others,” Wood says. And even beyond the immediate horizon, there will likely be new innovations to get around old filters because that is how spammers have been operating for years.

According to experts such as Leichter and Wood, this is why vendors must take a more rounded approach to the problem, rather than simply reacting to the content of the spam. “Spam is kind of viewed as one monolithic problem but it’s actually many different problems that usually fall into one of two different buckets,” Leichter says. “One is looking inside the envelope, reading the message and trying to figure out whether you want it or not. The other is just looking at the outside of the envelope and looking for other clues that would tell you that this is not something you even need to open. Like you do with junk mail.”

These ‘outside the envelope’ tactics used to simply rely on simple IP reputation filters, but many types of these filters and spam blacklists have been rendered ineffective by spammers’ botnets, which allow them to fly under the radar by only sending a few e-mails per originating IP address. This makes it difficult to spot bad IPs simply by volume of sent e-mail.

As a result, says Wood, vendors have turned to complex scoring systems that not only take sender IP into account but also look at other factors such as headers that a user normally wouldn’t see.

Usually users only see headers such as the “To” and “From” fields, he says. But there are many more behind the scenes that researches can use that will better help identify the mail’s source. “There’s some other patterns that if it was sent from a legitimate mail package, say Outlook or Thunderbird, they have a kind of pattern. Those headers appear in a certain order and you can almost “fingerprint” them to a degree – so you know that if it’s a genuine message from someone using Mozilla Thunderbird it will have these headers and they will appear in this order,” Wood says. “But if it’s come from a botnet, they typically don’t spend that much time making them look that accurate. They will put in the headers to try to spoof, say, Outlook Express message but in fact it won’t necessarily appear to be completely genuine, if you start to analyze it closely. And that’s something that we can then do at a greater level.”

End Game

While all of these techniques used by vendors do a lot to buffet the onslaught of new spam methods, some experts believe there is a major flaw in this approach. Garth Bruen with KnujOn says that the prevailing antispam lines of defense are only treating the symptoms of the real problem.

“Beyond the analysis, you have to ask a simple question. What do the spammers want?” he says. “In addition to looking at the technical aspects of spam we have to look at what’s driving spam, what’s enabling it in the world of crime. We have to partner up with global initiatives to stop traffic of counterfeit goods and pharmaceuticals across international borders. You have to push the issue with your government and say this is an important problem and we’re also concerned about the problems behind it. And you have to provide law enforcement with actionable information.”

That is Bruen’s goal with KnujOn, which is a non-profit side project designed to analyze as many spam messages as he and his volunteers can get their hands on. Rather than simply ignoring the root problem and constantly looking for ways to simply filter spam, businesses can do more for the greater good by helping researchers examine and track spam in order to provide authorities with that information. (Read more on Bruen:Financial Institutions: Fight Back Against Unwanted Email).