[Looking for Charlie's main web site?]

CFMAIL being detected as spam? Some solutions for CF 6, 7, and 8

Someone asked on a list about how to resolve the problem where messages sent via CFMAIL have a message-id value that can cause mail to be flagged as spam. I mentioned in an earlier entry how to solve this in CF8. In this entry, I offer a couple of solutions for CF 6 and 7.

To backup and explain the problem, some have lamented that in CFMX 6 and 7, CFMAIL used the name of your server where CF is installed, which might be something like "server1" or "bingo", as in:

Message-ID: <23070863.1197039960343.JavaMail.SYSTEM@Server1>

This might cause a recipient mail server to flag the mail as spam, if the mail server was a different domain name (like "yourcompany.com"). The bummer was that many found no way to fix this. Sure, in CF5 you could set it in a CFMAILPARAM to set a mail header, but CF 6 and above ignores that.

The good news for those on CF 8 is that a change makes the problem go away, which I talked about in a blog entry back in December.

But for those still on 6 or 7, I'd mentioned that there's an available code or (preferably) configuration fix to resolve the problem. Apologies for not posting it at the time, so here it is. I also explain what the issue is all about, for those not aware of it.

Simple CF5 solution doesn't work in 6, 7, or 8

First, let me point out that some may propose that one can just use CFMAILPARAM to set that message-id header, as was mentioned in this FAQ:

http://www.developer.be/index.cfm/fuseaction/faqDetail/FaqId/201.htm

Unfortunately, that's a very old FAQ and it no longer applies after CF5.

CFMX 6 (and 7 and 8) ignore that header you set and set the message-id themselves to the name of the physical server from which the mail's sent. Here's an example, from my laptop (which is named charlied620):

Message-ID: <23070863.1197039960343.JavaMail.SYSTEM@CharlieD620>

On your server, it could be that the problem is that the server is a real domain name, but it's not the same name as the SMTP server through which you're sending email via CFMAIL, and some servers won't let mail through with FROM addresses having a different domain name.

Why doesn't CF just use the SERVER set in CFMAIL or the CF Admin?

You might wish/expect it would just use name of the mail server as specified in the CF Admin mail server setting or the CFMAIL SERVER attribute, but it does not (at least, not prior to CF8, as discussed in my earlier entry.)

But while researching this problem for the person, I found this, where a very compelling solution was proposed. I tested it, and it does indeed work.

It turns out that you can get CFMAIL to use a specific mail server, either of 2 ways, for CF 6, 7, or 8 (again, on CF8, the easier solution is that I discussed in the earlier entry). Ken Smith of Adobe offered it also in this Adobe forum thread. If you make this change, either server-wide or per page/app, you will now find that the CFMAIL sets the message-id to use a desired servername.

Solution for 6, 7, or 8: Configuring it at CF Startup

If you're on a server where you want to change this for all CFMAIL (and javamail) done from within CF, you can change the JVM.config for your server to add this to the java.args line:

-Dmail.host=desiredservername

That java.args line is one long line. Don't introduce any line breaks. You need to restart the server after making that change. As always, when editing the JVM.config (in the runtim/bin directory of CF), make a backup first, because if you get it wrong, your server won't restart. (Or leave the file open and be prepared to do an Undo in your editor.)

But if you don't want to (or can't) mess with the startup file, or more important, if you are on a server where each app may need to do its own setting, you're not out of luck.

Solution for 6, 7, or 8: Changing it programatically

You can also make the needed change on the fly, programatically. You could issue this set of code just before your CFMAIL:

<cfscript>
sysObj = CreateObject("java", "java.lang.System");
sysObj.setProperty("mail.host", "desiredservername");
</cfscript>

I tried this per-page setting in in 6 and 7 and it worked fine. For CF8, the code will run, but it won't affect the CFMAIL for reasons I explain in the next section. But there's a real problem with this approach, if you're on a shared CF server. You're still setting the value for the entire CF server, but you're just doing it on the fly. It has at least a few problems you need to think about:

  • you'll affect all who use CFMAIL on this server (at least in 6, and 7, since CF8 ignores this, as discussed below)
  • you'll also affect anyone who uses Javamail on this instance of CF
  • and if someone else on the server issue the same code with a different server name, that of course would override your setting

There are a few things you can do to mitigate the problem, but they're not entirely perfect.

First, you could get the current mail.host value (in case it's different), then change it to what you want, and then change it back. You could do that with this code:

<cfscript>
   sysObj = CreateObject("java", "java.lang.System");
   // if there is no mail.host set already, this next variable simply won't be created. need to know that for later.    oldmailserver = sysObj.getProperty("mail.host");
</cfscript>

Note that if the mail.host property has not been set in the startup config or by someone else running such code, then the getproperty will return nothing, and the oldmailserver variable literally will not be created (a curiosity of working with some java methods).

Once you have the oldmailserver (or know that it did not exist), you can do the set of the property and the cfmail as above, then you could set it back with the following:

<cfscript>
// reverse the setting of the mail.host, so as not to affect others on this server if (not isdefined("oldmailserver")) {
   // if there was no previously set mail.host, remove the property
   sysObj.getProperties().remove("mail.host");
}
else{
   // set it back to what it was before
   sysObj.setProperty("mail.host", oldmailserver);
}
</cfscript>

There's still a problem. Because of CF's multi-threaded nature, it's entirely possible that between your setting the host and doing your CFMAIL, someone else could also set the mail server to something different. (This is called a "race condition".)

Now, Mr. Smith in the threads above suggested that you could wrap the code doing the change and the CFMAIL in a named lock, but that's an incomplete solution. It will prevent other other code (that ALSO does the same named lock) from running until yours is complete, but it won't help if others do the set of the property without bothering to use the same named lock (or use a differently named lock).

That's a frequent misconception about locks. They don't prevent other code doing what you're doing in the lock: they only tell other code that IS using the same lock not to run while this lock is held--and even then, only if you use an EXCLUSIVE lock, which he didn't indicate.

So the bottom line is that as useful as the feature is to set the property dynamically, it's fraught with peril in an environment where multiple apps may try to use the code. Only if you can guarantee that all use the same named lock will you be able to protect against this problem is held up while you have the value changed:

<cflock name="setmail-servername" timeout="10" type="EXCLUSIVE">

   <cfscript>
   sysObj = CreateObject("java", "java.lang.System");
   // if there is no mail.host set already, this next variable simply won't be created. need to know that for later.    oldmailserver = sysObj.getProperty("mail.host");
   sysObj.setProperty("mail.host", "desiredserver");
   </cfscript>

   <cfmail ...>
   ...
   </cfmail>
   
   <cfscript>
   // reverse the setting of the mail.host, so as not to affect others on this server    if (not isdefined("oldmailserver")) {
      // if there was no previously set mail.host, remove the property
      sysObj.getProperties().remove("mail.host");
   }
   else{
      // set it back to what it was before
      sysObj.setProperty("mail.host", oldmailserver);
   }
   </cfscript>
</cflock>

Now, someone may propose that all this could be wrapped up into UDFs or CFC methods, and perhaps it could, but since you need to do the setting of the value, and the mail, and the resetting of the value, all within a CFLOCK, it would be challenging to wrap all this up into a generically callable method (unless you wanted to pass in as well all the CMAIL attribute values and the body). Just seems not worth it, since this is a pretty esoteric problem and solution. But others can comment if they feel differently.

Solution for CF8 is much easier

All this is obviated on CF8, because it now properly uses the name of the server specified in the CF Admin mail server setting (or CFMAIL SERVER attribute, which overrides the CF Admin setting.) This is AWESOME news for those challenged by this, and hasn't gotten much press.

Now, what was the caveat I mentioned above? Well, if you use the approach of setting the mail.host servername in the java property, CF8 no longer pays attention to that. It JUST uses the CF Admin mail server setting, or the CFMAIL SERVER attribute. So that code above "won't work". It will work, but it won't affect CFMAIL. But that was a hack to work around CF not honoring these other attributes. I'm not surprised (or bummed myself) to see that it no longer regards the mail.host property for CFMAIL.

(I should say I'm saying this as of CF8. I've not tested it on 8.0.1.)

Some related notes

If you add the SERVER attribute, you may need to add the PORT, Username, and Password as well.

Here's a little bonus tip, in case you try to use the CFMAIL SERVER attribute for the first time on an existing CFMAIL tag to check this out. Note that using the SERVER attribute on CFMAIL requires you then to specify any other attributes required for the mail connection such as USERNAME and PASSWORD (if needed) and PORT (if not 25). What I mean is that it will no longer pick up the values in the CF Admin. If you override the SERVER, then you override the other config settings as well and need to specify them.

I'm not getting the emails now. Where are they?

If you make a mistake in your setting of the mail server arguments, then CF will move the failed emails to the [coldfusion]/mail/Undelivr directory where CF is installed. They're just plain text files. You can open them to see what got set and perhaps can figure out why they failed. You may also find information in the [coldfusion]/logs/mail.log.

When I look at mail in the spool, it looks fine (if I use CFMAILPARAM). Why doesn't it get through?

Don't be misled. The email you generate in CFMAIL may look fine in the [coldfusion]/mail/spool directory, but when it gets sent to the mail server, CF will change that message-id (and some other headers). You really need to look at the email as it's RECEIVED. You can't even look at the message in the Undelvr folder as an indication. It doesn't have those added headers.

So how DO I observe the mail headers?

You need to receive the email and then look at its headers. I'll show you how to do that in Outlook and gmail in a moment. Let me point out that it can be very englightening to view the message headers: not only the message-id but possibly also other headers as well as messages that CF, your mail server, or the mail server of your recipient may have added, which may include indications of whether your email was detected to be spam (perhaps by tools like SpamAssassin).

In Outlook, you can use View>Options while reading an email, to see the values in the "Internet Headers" box of the window that opens. Scroll down in that to find the message-id and other headers.

In gmail, when you open the message, look to the top right of the pane showing your email, to the right of the indicator of the time the message was received. There's a drop-down box, with options like "reply" and "forward". Choose "show original". That will show the complete message including all the headers at the top.

Other CFMAIL alternatives/replacements

Finally, I'll point out that if you run into other problems with CFMAIL, there are always alternative CF mail server solutions, like those I list in the "CFMAIL replacement/enhancement tools" section of my "Tools to Consider for CFML developers" page.

Hope that helps some.

Comments
So weird, I was just looking at this same issue last night, thanks charlie
# Posted By Big Mad Kev | 4/9/08 1:43 AM
Any idea about Bluedragon 7?

My header says:

Message-ID: <30538613.85251207579451743.JavaMail.root@server.host.com>

That is not my VPS domain name, but rather the host's server name.

Is there a way to improve that?
# Posted By ziggy | 4/9/08 11:55 AM
Ziggy, I'd imagine it's the same. Have you tried the things discussed here? (In case anyone's wondering why I hedge, it's because I left the company that makes BlueDragon 2 years ago last week, so I just don't keep up on things like I used to. I also happen to have reinstalled my development laptop recently and have not yet reinstalled BD, so I can't even test things for Ziggy. If anyone else has info on this challenge, feel free to share.
# Posted By Charlie Arehart | 4/10/08 10:09 AM
I found a jvn.cfg file in the BD folder and added a line at the bottom:

-Dmail.host=site.com

With and without the em tags you show above, figured you really don't want them, but neither worked anyway.
# Posted By ziggy | 4/10/08 2:25 PM
Gads, no, those em tags shouldn't have been there. I had used them to emphasize the significant part of the line, but since I put it in an html code block, that just passed them through for display. Darn. I've corrected and removed that above (it was in 2 places).

As for your continued challenge, Ziggy, I'd say the best thing would be to bring this to the BlueDragon interest forum where both enthusiastic users and New Atlanta employees share answers to such challenges. I'm afraid I just can't look into this for you for now. I'm sure their engineers can sort this out and it may well benefit all BD users to bring it up there. Many just aren't aware of the issue at all (in all the CFML engines).

Finally, I want to say that you don't need to keep putting in private@private.com as a fake email address. That causes people who make other comments to get a bounce back when the system tries to email their comments to your fake address.

I realize you were being forced (by the version of BlogCFC that I run) to always enter an email address, even if you chose not to subscribe to the comment. I'd not noticed that before, but I've corrected it now. I've also removed the email addresses from the 2 comments you've made so far.
# Posted By Charlie Arehart | 4/11/08 11:07 AM
Charlie,

We are trying to implement <cfmail> functionality at my new job but our IT department absolutely refuses to allow it because of relaying and running the risk of being "blacklisted". We used <cfmail> at my previous job and had no problems so I don't really understand their argument. I'm not an IT person so I don't really understand the risks in using smtp. All we developers here know is that we really need <cfmail> and don't know how to argue their points. Does your article address their concerns? Is this the "argument" we've been looking for to counter their concerns with relaying and "blacklisting"?
# Posted By Mary | 4/21/08 9:17 AM
@Mary, this entry may very well be just the ticket for helping resolve your problem. It might be quite an eye-opener for them, at least in terms of why the use of CFMAIL may have been cause of inadvertent blacklisting in the past, and how to resolve it a couple of ways (it's a lot more easily "solved" by going to CF8, though there are solutions for earlier releases, too, as described above).

As for relaying I don't myself see how use of CFMAIL alone would open you to relaying. That could just be an instance of the game of "operator", where the concern someone started with just isn't the same as it's ultimately been understood by you. :-)

Relaying is a matter of control in the SMTP server--CF simply sends to a SMTP server just like any other mail client would when generating outgoing mail, so whatever controls they'd implement to prevent relaying shouldn't hurt (or be hurt by) use of CFMAIL.

I suppose another argument they may have is that CFMAIL does indeed make it so easy for people to send thousands of emails automatically, so some developers may abuse the tool. If recipients choose to mark the mail as junk, or the senders fail to provide adequate opt-out features (as required for many per the CAN-SPAM Act of congress from a few years ago), I think that could increase the likelihood of a server being blacklisted (or "RBL'd").

Finally, you may want to make sure they know that there's a logging feature for CFMAIL which they could enable in the CF Admin, that tracks when emails are sent from CFMAIL. That could give them at least more insight/oversight. (Just a thought.)

Hope any of that helps. If it would help, I'll note that I'm available for consulting in as small a timeframe as you need to resolve a problem. If things remain stalemated and you think they (or your colleagues, or your bosses) would benefit from an outsider perspective, I'm happy to help. See http://www.carehart.... But hopefully the info above will help resolve things for you. Cheers.
# Posted By Charlie Arehart | 4/21/08 12:30 PM
Hi,
For those looking for a resolution for bluedragon i have started a thread on the new atlanta mail list. The new atlanta staff are looking into this issue.

cheers

Marcel
# Posted By Marcel Scherzer | 5/21/08 4:43 PM
Oops i had an itchy finger and forgot the link. Sorry maybe charlie can fix my post.

http://forums.newatl...
# Posted By Marcel Scherzer | 5/21/08 4:45 PM
Thanks, Marcel. We'll let it be. If I modified the first and deleted the second, some may click on links in feed readers or email and be confused. Not a problem.

Anyway, thanks for sharing the link. Folks can keep an eye on that thread to see how things get resolved.
# Posted By Charlie Arehart | 5/22/08 1:08 PM
Here's another way that your CF-generated email could be marked as spam.

Consider sites like evite or egreetings, which lets visiting users to send email to others, where the email goes out as from their own email address (which is typically not the same as the domain of your server). That too could cause such email to be marked as spam.

For more info, and how to fix it (generically), see this article from the SPF project: http://www.openspf.o...

You'd want to use CFMAILPARAM to set the appropriate headers as discussed in the site. I've not actually done it. If anyone else has and wants to share more info, please do. I just saw this site for the first time and thought I'd pass it along with respect to the above.
# Posted By Charlie Arehart | 6/9/08 11:04 PM
Updated information for BlueDragon Server, the issue has been resolved since hotfix nr 701_361 for the JX edition. After applying the patch open the bluedragon.xml file (within the config folder of the bd installation). The find the cfmail section and insert the following element if it does not yet exist:

<domain>mydomain</domain>

cheers

Marcel
# Posted By Marcel Scherzer | 7/8/08 3:25 AM
Thanks for sharing that, Marcel.
# Posted By Charlie Arehart | 7/8/08 9:43 AM
@Charlie,

Thanks for sharing this info. I wrote an article using the recommendations from the SPF project that I thought might also help:
http://www.stephenwi...
# Posted By Steve Withington | 7/10/08 4:26 PM
Great. Thanks, Steve.
# Posted By Charlie Arehart | 7/10/08 4:31 PM
our store emails are not getting through to customers who subscribe to our newsletter or order products from our store and but do get an email confirmation

Our store is on a Windows 2K3 VPS account running CF8, MySQL and our mailserver is SmarterMail 5

I am not sure where to look right now. I have enabled SpamAssassin in SmarterMail for incoming and outgoing mail. But I also tried the setup without enabling SpamAssassin for outgoing mail.

Both do not work with one particular email. Where should I look?
# Posted By AnthonyKatgert | 7/30/08 4:20 AM
Anthony, did you follow the suggestion above under "I'm not getting the emails now. Where are they?"
# Posted By Charlie Arehart | 7/30/08 9:15 AM
Copyright ©2018 Charlie Arehart
Carehart Logo
BlogCFC was created by Raymond Camden. This blog is running version 5.005.
(Want to validate the html in this page?)

Managed Hosting Services provided by
Managed Dedicated Hosting