A while ago I turned up as much spam and junk mail protection stuff as I could find in my Postfix configuration. Specifically, I had the following options set:

smtpd_helo_restrictions =
    reject_invalid_hostname
    reject_non_fqdn_hostname
    reject_unknown_hostname

Last week, Amy was contacted by somebody who had tried to send her an email but the email was rejected by my mail server. We hadn't yet got back in touch with him to try to find out what the problem was. I started to pay more attention to my spam rejection logs just in case something was amiss. Today, I noticed that my mail server rejected some sort of newsletter from Google. Upon further investigation, it seems that the Google server that sent the mail was using a HELO name that wasn't resolvable. This condition is checked by Postfix on the reject_unknown_hostname restriction.

I decided that if even Google can't always get it right, then I should probably stop rejecting email for an unresolvable HELO name. So I turned off the three restrictions listed above.

Later today, we got a phone call from the company here in New Zealand who is handling the import of our household goods. Apparently, they had tried to email me earlier but the mail had been bounced back, rejected. I looked through the mail server log and found that they, too, had sent from a server with a misconfigured HELO name (it even ended in .local). The mail had come through on a retry after I had removed the HELO name restriction.

That pretty much does it for that filter. If companies as small as a local goods importer and as big as Google can't get it exactly right, I run the risk of rejecting all kinds of legitimate incoming email. As much as I dislike getting spam, I dislike losing real email even more.

decibel45 keeps telling me I should be using one of those scoring filters that doesn't outright block a message for just one SMTP transactional failure. It might take a while for me to get motivated enough to mess with my Postfix configuration again.

I happened to read the disclaimer text at the bottom of a stock spam I got today. I was amused (emphasis mine):

Seems to me that those statements are even more pessimistic than usual.

Further to my previous entry about people responding to spam, I heard back from one of the people I sent a response to. I'll back up a bit and show you the whole exchange. First, his original message to me:

From: "Don" <...@cox.net>
To: "Rosamund Stephenson" <lchta@hewgill.net>
Subject: RE: monsoon

Ok knock your shit off please and stop sending me this

[original spam message quoted in his message]

My response:

From: "Greg Hewgill" <greg@hewgill.com>
To: "Don" <...@cox.net>
Subject: RE: monsoon

On Fri, Apr 28, 2006 at 08:29:26AM -0700, Don wrote:
> Ok knock your shit off please and stop sending me this

I understand you received some spam that appeared to be from an email
address in a domain I own (hewgill.net). Unfortunately, today spammers
are using software that can easily forge the "From" address on email
messages. They don't use their own name to try to hide their identity.
They insert random email addresses from domains they don't own and so it
looks like the spam came from me. I can assure you that it did not.

If your email provider has a spam blocking system, you may want to try
turning that on. On my own email system, I have a number of different
layers of spam protection and it brings the spam down to a manageable
level (I receive thousands of spam messages per day, most of which are
automatically blocked). With luck, people will stop buying products from
spammers and it will no longer be profitable for them to send their junk
mail.

Have a great day.

Greg Hewgill
http://hewgill.com

I thought that was clear, succinct, and quite understandable. However, our friend Don appears to have not read any further than the very first sentence:

From: "Don" <...@cox.net>
To: "Greg Hewgill" <greg@hewgill.com>
Subject: RE: monsoon

Own it or not I don't need it so STOP sending me this

Of course he didn't read my explanation about forging of email headers; he didn't read that I did not personally send him that spam; he did not read about my recommendation regarding spam filters; he did not read that I too am subject to receiving spam from junk mailers; he is probably not even having a great day today. I had originally thought about mentioning SPF and how that would have prevented him receiving the message in the first place, but I'm sort of glad I didn't waste my energy. (Instead, I'm using that energy to rant about this here.)

Perhaps I'm old and jaded (at least in internet years), but it's surprising to me that people using the internet today have no clue how email works. I imagine our friend Don could, perhaps with a bit of effort, understand what happened if he received a forged written letter in his mailbox delivered by the post office. However, the same does not appear to be true about email delivered by his computer. If he hits Reply, the message must go back to the person who sent it, right? I fear for him if he ever starts receiving phishing messages.

Ok, I know this is boring. I'm done for now.

I got the following message in email today. The "phishing" detector in my brain went off immediately, but on further inspection it appears that this message really did come from Paypal.

This message seems non-phishy due to the following aspects:

• My full name is used, not something derived from my email address
• There are no links to click on (ie. "click here to visit the Resolutions Center")
• The message headers (below) indicate no evidence of forgery
• The IP address it was sent from matches the SPF record at paypal.com

Paypal is in the unenviable position of having to fight with all the phishers when they really do want to email a notification to their customers. It appears that this is the best they can do, and of course the phishers will imitate this style of message as closely as possible, which makes Paypal's original message look like junk mail. It's an uphill battle for them.

Now I'm curious about what they've done with my account.

Update: I logged on to Paypal and they appear to only have expired my password. No other outstanding issues were to be found in their "Resolution Center" after logging on. I've found that Paypal seems to expire my password frequently (once every month or two), and asks that I supply a new, different password plus configure two new security questions. Perhaps I've been getting a message like this every time they expire my password but this is the first time I noticed it wasn't a phishing message.

I just checked my mail and one second after receiving confirmation of changing my password and security questions, I also got:

Thanks Paypal. Hardly a pleasure doing business with you.

I've been working again on making sure my mail server does the best job it possibly can at rejecting spam and other unwanted email. I recently wrote a Postfix log file scanner that summarizes the reasons why email is being rejected. You can see the current summary here (this shows the statistics for the current day since 00:00 UTC).

A few things about this summary are interesting:

• I suspect a lot of the entries in the "HELO (unknown)" list are actual SMTP servers, relaying spam, that are misconfigured to send the wrong HELO name. Sorry, I'm not going to accept your mail.
• The "Recipient (local reject)" list of common worm destinations (adam, alex, alice, etc) is still very effective.
• Relay attempts seem rare. I had expected to see more of these, but on the other hand it still gives me a warm fuzzy feeling when relay attempts are rejected.
• I had expected the "DATA pipelining" filter to catch more. This happens when an SMTP sender ignores responses from the my server and just fires the commands through as fast as it can without waiting for acknowledgement. Perhaps all those clients happen to be caught earlier by my rejecting a "HELO hewgill.com" command, who knows.
• There are a lot of different email worms out there! I suspect that many of the previously rejected connections to my mail server would eventually have tried to deliver an email worm. But clamav still does a great job of filtering out that junk.

Even after all this, when my SMTP server rejects over 90% of the connections to it, spamassassin still catches a lot after delivery. I'm going to work on some statistics processing there so I can find out how effective that is.

Finally, I still get an annoying amount of spam landing in my inbox. The war continues.

I received the following message yesterday, and it almost had me fooled:

From: "Jamie" <jamie.andrews@totalbusiness.co.uk>
To: <greg@hewgill.com>
Subject: Photo Approval
Date: Thu, 10 Nov 2005 03:31:07 -0600

Hello,
Your photograph was forwarded to us as part of an article we are publishing for
our December edition of Total Business Monthly.
Can you check over the format and get back to us with your approval or any
changes?
If the picture is not to your liking then please send a preferred one.
We have attached the photo with the article here.
Kind regards,
Jamie Andrews
Editor
www.TotalBusiness.co.uk
**********************************************
The Professional Development Institute
**********************************************

I had received a legitimate message a few days ago from somebody in Germany asking to use one of my pictures in an architectural publication of some kind (I gave them permission). Initially I thought this might be related. Then I noticed the attachment:

[-- Attachment #2: Photo+Article.exe --]
[-- Type: application/octet-stream, Encoding: base64, Size: 13K --]

It's an email virus! Luckily I handle my mail using mutt on freebsd, so it would have been several more steps for me to "open" the attachment, so I didn't run it. It's amazing the lengths to which these virus authors are going to try to trap people.

In related news, I'm still getting fake-rolex spam which spamassassin is still marking as BAYES_50 (which means spamassassin thinks it's middle of the road, no sign of spam). Every one of those damn things I run through sa-learn to teach it, but the people who write those have found that the english language is extremely flexible and there are a million ways to package their sales pitch.

Spam is starting to become overwhelming again. Whatever happened to SPF, anyway?

It seems that spammers have introduced a new twist on an old tactic. When Paul Graham's A Plan for Spam article introduced Bayesian filtering principles to the antispam world, spammers were quick to react to this new threat. Since their spam was now being scored by full content (and not just naive keyword matching), they started including snippets of legitimate text along with their spam messages. This legitimate text, since it wasn't part of their marketing campaign, was typically displayed in an impossibly small font or in invisible (ie. white on white) colors.

Anyway, I recall seeing text pulled from such works as Moby Dick, Ulysses, and various Shakespeare. It didn't matter what the text was, as long as it didn't look very much like spam. As far as I can tell, there are at least two goals involved here:

1. With the inclusion of a lot of non-spam text, there would be a slightly higher probability that the message might look a little bit more like a legitimate message, and would then sneak through a slightly higher percentage of spam filters.
2. Bayesian filters learn patterns from the messages you receive and mark as spam. When you mark a message as spam, each word in the entire message essentially gets a count in the "spam" column. By including a lot of non-spam text, this means that a lot of non-spam words will end up with higher counts in the "spam" column. This has the longer-term effect of decreasing the trustworthiness of the Bayesian filter data, because it may start to mark legitimate messages as spam. If this happens a lot, users may turn off the Bayesian part of the filter.

Recently, several people (cetan, leroy_brown242, Amy) who have journals, have received messages from other Internet users wondering why some of their journal text was included in the spam message. Obviously, the journal authors don't have anything to do with the sending of the spam. It seems that the spammers are now scraping text off the Internet instead of using text from the classics.

Perhaps this approach is intended to more closely match the kind of text that people normally receive in email. Because the text is written by today's Internet users and not 19th century authors, the vocabulary will be better suited to confuse spam filters.

This new technique is surprising and annoying to those users whose text is used in spam. Most recipients of the spam will either not see the message at all, or not see the small/obscured text, or just ignore it. The few who do look at the whole message and google for key words or phrases to find the original author's journal, seem skilled enough at that point to not accuse the user whose journal text was used.

Fortunately, Bayesian filtering techniques are just one weapon in the fight against spam. With blacklists, SPF, virus scanners, and the battery of tests provided by SpamAssassin, I now get, on average, about 5 spam messages in my inbox per day. Since my mail server receives about 1000 spam messages per day, that's less than a 1% miss rate on my spam filters.

I've been using gmail for a while now. I have set up procmail rules that forward all my email to my gmail account for easier reading (I like the way gmail lets me read mailing lists and such). What I've noticed for a while now, is that whenever I make a post to a mailing list, my own message ends up in my gmail Spam folder. Every time this happens, I mark it as "Not spam" and move it back out. It doesn't help. I tried some test messages today to my gmail account and other gmail accounts, and each time my email was marked as spam.

Unfortunately, gmail does not provide a way for me to determine why a given message is marked as spam. But I have my own theory...

Recent generations of email worms (such as Netsky and Bagle) send worm messages using email addresses found on the infected computer. These email engines scan through the computer's hard drive looking for various types of files that contain email addresses, and send email worm messages using those addresses in the To and From field. I get a lot of these messages, usually about a thousand per day (slightly less on weekends). Assuming the worms randomly select addresses, I can assume that at least an equivalent number of worm messages are also sent "From" my email address.

The next question is why does my email address appear on so many computers around the world? I believe the answer lies with VNC. Many years ago I contributed some code to the VNC project, and the VNC authors acknowledged my contribution by including my email address in the VNC "history.txt" file. Since VNC is a very popular program, my email address appears in a .txt file on an unknown number of computers on the Internet, where it can be easily picked up and used by email worms.

I have done some analysis of email worm messages I receive, and a significant number of them (around 20%, I don't remember the exact number) have addresses in the From field that also come from VNC-related files. This lends support to my theory.

Connecting the dots, it seems that gmail's systemwide spam filter has identified <greg@hewgill.com> as a sender whose messages should always be marked as spam. It's quite disappointing for me to reach this conclusion.

I have submitted a problem report to gmail, asking whether they can shed any light on this issue. The ideal solution would be to remove my name from their global address filter list, and instead let the filtering happen by content (gmail rejects worm-infected email). The worst case solution is they ignore the problem or just tell me to change my email address (which I don't consider an acceptable solution).

Another disheartening implication here is that other email providers, large or small, might also automatically consider my email messages spam for the same reasons. This would not make me very happy.

I've noticed a significant increase in worm related email traffic over the last couple of days, apparently related to the "netsky" variant this time (though it's getting harder to tell all the different variants apart). Overnight last night I received nearly 400 new worm-related email messages, all of which successfully passed through qgreylist, spamassassin, and some custom filters that worked for mydoom.

In looking at the incoming messages, I noticed something odd about the HELO command that was used. Normally in an SMTP transaction, the first command that the sending mail server sends to the receiver is something like "HELO example.com", which means the sender is responsible for sending messages from example.com. This HELO name isn't actually used during the delivery of the messages, but is just sort of a polite introduction.

What I noticed was that in most of these worm messages, the sender was using the command "HELO hewgill.com". This is backwards, it should be using the domain name of the sender, not the recipient. So I can be sure that if the sender uses my domain name in HELO, then it is not a legitimate message. By using the following procmail rule:

* ^Received:.*(HELO hewgill.com)

a huge number of worm messages have been caught. In just the last 2.5 hours, 181 messages have been blocked and 33 got through. Of those that got through, virtually all of them are bounce messages from other mail servers, where a message to somebody@example.com with my name forged in the From field, was returned to me as undeliverable. These types of messages, since they are sent by legitimate functional mail servers, have correct HELO commands and are not recognized by this filter.

Unfortunately, the latest variants of these email worms no longer seem to have automatic built-in expiration dates (as reported by mcafee or symantec), like the sobig family did. We can therefore expect to be plagued by these for some time to come.

Finally, this would be a good time to remind everybody who is responsible for any email operations to adopt Sender Policy Framework and publish SPF records. Doing so will help reduce the ability for worms to forge email addresses from your domain, and using SPF filtering against incoming messages will help you reject messages with forged From addresses.

:0B
* ----=_NextPart_..._...._........\........[^0]

From: K Fick <xxxxx@yahoo.com>
Subject: DO NOT SEND ANYMORE MAIL TO THIS ADDRESS AND TAKE ME OFF MAILING LIST!!!Re: Approved